2024-03-21T08:02:10.042 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T08:02:10.042 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117 --owner scheduled_vshankar@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} -- /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/orig.config.yaml 2024-03-21T08:02:10.109 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:02:10.195 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117 branch: wip-vshankar-testing-20240320.053002 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} email: vshankar@redhat.com first_in_suite: false job_id: '7612117' kernel: flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240320.053002 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240320.053002 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8073 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 sleep_before_teardown: 0 subset: 203/1024 suite: fs suite_branch: wip-vshankar-testing-20240320.053002 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJDj96CaTm2/xZqJDkdoZNnUKpmGwfZz2kpHXR/Y21IrtkN6OGzeD4VzXbZ63tXBerS9FgBv3ayeab/WJ+/FpNE= smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPP0yP7mX4zsSOqBOxp68QobEtnme1Xc9wLUGtwZ2Yl7Za9u+FacaCqBcA5eMm9++w8axt+NJ46jUJYIxo9RShA= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBXpGC40u8YdbIVz5SEG8QaUJr7u3b/aEXSvg9CYeSrYV+uWi7WnDuwILoBGgeERzSUEVTeQnVM7tqs7eJMJotE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-20_11:21:18 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T08:02:10.196 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa; will attempt to use it 2024-03-21T08:02:10.196 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks 2024-03-21T08:02:10.196 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T08:02:10.198 INFO:teuthology.task.internal:Checking packages... 2024-03-21T08:02:10.222 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2edfee68ad7243bed422a049b6f5f58619300f11' 2024-03-21T08:02:10.222 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T08:02:10.222 INFO:teuthology.packaging:ref: None 2024-03-21T08:02:10.222 INFO:teuthology.packaging:tag: None 2024-03-21T08:02:10.222 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240320.053002 2024-03-21T08:02:10.222 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:02:10.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240320.053002 2024-03-21T08:02:10.371 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2231.g2edfee68 2024-03-21T08:02:10.372 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T08:02:10.379 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T08:02:10.379 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T08:02:10.386 INFO:teuthology.task.internal:Saving configuration 2024-03-21T08:02:10.410 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T08:02:10.416 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T08:02:10.442 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 07:56:43.417094', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJDj96CaTm2/xZqJDkdoZNnUKpmGwfZz2kpHXR/Y21IrtkN6OGzeD4VzXbZ63tXBerS9FgBv3ayeab/WJ+/FpNE='} 2024-03-21T08:02:10.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 07:56:43.418897', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPP0yP7mX4zsSOqBOxp68QobEtnme1Xc9wLUGtwZ2Yl7Za9u+FacaCqBcA5eMm9++w8axt+NJ46jUJYIxo9RShA='} 2024-03-21T08:02:10.488 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 07:56:43.420329', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBXpGC40u8YdbIVz5SEG8QaUJr7u3b/aEXSvg9CYeSrYV+uWi7WnDuwILoBGgeERzSUEVTeQnVM7tqs7eJMJotE='} 2024-03-21T08:02:10.488 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T08:02:10.494 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-21T08:02:10.495 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-21T08:02:10.495 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-21T08:02:10.495 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T08:02:10.596 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a87fdd160>, signals=[15]) 2024-03-21T08:02:10.597 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T08:02:10.605 INFO:teuthology.task.internal:Opening connections... 2024-03-21T08:02:10.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:02:10.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:02:10.713 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-03-21T08:02:10.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:02:10.810 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-03-21T08:02:10.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:02:10.905 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T08:02:10.913 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-03-21T08:02:10.932 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-03-21T08:02:10.932 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-03-21T08:02:10.990 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-03-21T08:02:10.990 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-03-21T08:02:10.990 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:02:10.991 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:02:10.992 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-03-21T08:02:11.013 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-03-21T08:02:11.031 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-03-21T08:02:11.031 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-03-21T08:02:11.087 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:02:11.088 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:02:11.089 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-03-21T08:02:11.113 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-03-21T08:02:11.130 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-03-21T08:02:11.130 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-03-21T08:02:11.187 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-03-21T08:02:11.187 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-03-21T08:02:11.187 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-03-21T08:02:11.187 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:02:11.187 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-03-21T08:02:11.187 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:02:11.188 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:02:11.188 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-03-21T08:02:11.212 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T08:02:11.223 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T08:02:11.251 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T08:02:11.251 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:02:11.254 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:02:11.256 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:02:11.273 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T08:02:11.279 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T08:02:11.279 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:02:11.312 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:02:11.315 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:02:11.420 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T08:02:11.497 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T08:02:11.512 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-21T08:02:11.512 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}}, timeout 300 2024-03-21T08:02:11.513 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:02:11.513 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:02:11.513 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:02:11.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:02:11.530 DEBUG:teuthology.task.kernel:sha1 for mon.b is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.531 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T08:02:11.531 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-03-21T08:02:11.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:02:11.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:02:11.532 DEBUG:teuthology.task.kernel:sha1 for mon.a is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.533 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T08:02:11.533 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-03-21T08:02:11.533 DEBUG:teuthology.task.kernel:sha1 for mon.c is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.533 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T08:02:11.533 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-03-21T08:02:11.588 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-361.el9.x86_64 2024-03-21T08:02:11.588 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.588 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T08:02:11.588 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.593 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-361.el9.x86_64 2024-03-21T08:02:11.593 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-361.el9.x86_64 2024-03-21T08:02:11.593 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.593 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T08:02:11.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.594 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.595 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T08:02:11.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:02:11.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum upgrade -y linux-firmware 2024-03-21T08:02:11.746 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum upgrade -y linux-firmware 2024-03-21T08:02:11.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum upgrade -y linux-firmware 2024-03-21T08:02:14.334 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - AppStream 7.0 MB/s | 6.7 MB 00:00 2024-03-21T08:02:14.347 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS-9 - AppStream 7.3 MB/s | 6.7 MB 00:00 2024-03-21T08:02:14.387 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 6.6 MB/s | 6.7 MB 00:01 2024-03-21T08:02:15.834 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-03-21T08:02:15.856 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-03-21T08:02:15.862 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-21T08:02:16.754 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-21T08:02:17.450 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.0 MB 00:01 2024-03-21T08:02:18.044 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-03-21T08:02:45.031 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 727 kB/s | 19 MB 00:26 2024-03-21T08:02:46.382 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 707 kB/s | 19 MB 00:27 2024-03-21T08:02:50.198 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.1 MB 00:01 2024-03-21T08:02:51.590 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 16 kB 00:00 2024-03-21T08:02:51.657 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.1 MB 00:01 2024-03-21T08:02:53.583 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-03-21T08:02:55.109 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-21T08:02:59.478 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 21 MB 00:07 2024-03-21T08:03:00.050 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-03-21T08:03:02.193 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-03-21T08:03:02.204 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-03-21T08:03:02.204 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-03-21T08:03:02.204 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 1 Package 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 375 M 2024-03-21T08:03:02.205 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-03-21T08:03:04.274 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-21T08:03:06.452 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-03-21T08:03:06.462 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-03-21T08:03:06.462 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-03-21T08:03:06.462 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-03-21T08:03:06.462 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 1 Package 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 375 M 2024-03-21T08:03:06.463 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-03-21T08:03:17.174 INFO:teuthology.orchestra.run.smithi171.stdout:linux-firmware-20240219-143.el9.noarch.rpm 35 MB/s | 375 MB 00:10 2024-03-21T08:03:17.175 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:03:17.175 INFO:teuthology.orchestra.run.smithi171.stdout:Total 35 MB/s | 375 MB 00:10 2024-03-21T08:03:17.176 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-03-21T08:03:17.270 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-03-21T08:03:17.270 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-03-21T08:03:19.182 INFO:teuthology.orchestra.run.smithi084.stdout:linux-firmware-20240219-143.el9.noarch.rpm 22 MB/s | 375 MB 00:16 2024-03-21T08:03:19.182 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:03:19.183 INFO:teuthology.orchestra.run.smithi084.stdout:Total 22 MB/s | 375 MB 00:16 2024-03-21T08:03:19.183 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-03-21T08:03:19.277 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-03-21T08:03:19.277 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-03-21T08:03:19.826 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-03-21T08:03:19.826 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-03-21T08:03:21.729 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-03-21T08:03:21.729 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-03-21T08:03:22.764 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 297 kB/s | 19 MB 01:04 2024-03-21T08:03:26.351 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-03-21T08:03:26.606 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:03:26.702 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:03:27.400 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.1 MB 00:00 2024-03-21T08:03:27.961 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-03-21T08:03:28.025 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:03:28.025 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:03:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:03:28.266 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:03:28.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:03:28.422 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:03:28.422 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-03-21T08:03:28.423 INFO:teuthology.orchestra.run.smithi171.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T08:03:28.423 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:03:28.423 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-03-21T08:03:28.710 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.c... 2024-03-21T08:03:28.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:03:28.882 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-03-21T08:03:28.948 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T08:03:28.948 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T08:03:28.952 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi171.front.sepia.ceph.com to complete... 2024-03-21T08:03:29.496 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:03:29.497 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:03:29.882 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:03:29.882 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:03:29.883 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-03-21T08:03:29.883 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T08:03:29.883 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:03:29.883 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-03-21T08:03:30.184 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.b... 2024-03-21T08:03:30.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:03:30.332 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T08:03:30.332 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T08:03:30.334 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi084.front.sepia.ceph.com to complete... 2024-03-21T08:03:31.036 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-03-21T08:03:36.075 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-21T08:03:38.371 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-03-21T08:03:38.387 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-03-21T08:03:38.387 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-03-21T08:03:38.388 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-03-21T08:03:38.388 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-03-21T08:03:38.388 INFO:teuthology.orchestra.run.smithi003.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T08:03:38.388 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:03:38.388 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-03-21T08:03:38.388 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-03-21T08:03:38.389 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 1 Package 2024-03-21T08:03:38.389 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:03:38.389 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 375 M 2024-03-21T08:03:38.389 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-03-21T08:03:43.916 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21 08:03:43 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T08:03:43.916 INFO:teuthology.orchestra.run.smithi171.stderr:FINISHED --2024-03-21 08:03:43-- 2024-03-21T08:03:43.917 INFO:teuthology.orchestra.run.smithi171.stderr:Total wall clock time: 15s 2024-03-21T08:03:43.917 INFO:teuthology.orchestra.run.smithi171.stderr:Downloaded: 1 files, 321M in 15s (21.9 MB/s) 2024-03-21T08:03:43.919 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.c... 2024-03-21T08:03:43.919 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T08:03:45.861 INFO:teuthology.orchestra.run.smithi171.stdout:Verifying... ######################################## 2024-03-21T08:03:45.912 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing... ######################################## 2024-03-21T08:03:45.991 INFO:teuthology.orchestra.run.smithi171.stdout:Updating / installing... 2024-03-21T08:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout:linux-firmware-20240219-143.el9.noarch.rpm 29 MB/s | 375 MB 00:12 2024-03-21T08:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout:Total 29 MB/s | 375 MB 00:12 2024-03-21T08:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-03-21T08:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-03-21T08:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-03-21T08:03:51.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21 08:03:51 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T08:03:51.410 INFO:teuthology.orchestra.run.smithi084.stderr:FINISHED --2024-03-21 08:03:51-- 2024-03-21T08:03:51.410 INFO:teuthology.orchestra.run.smithi084.stderr:Total wall clock time: 21s 2024-03-21T08:03:51.410 INFO:teuthology.orchestra.run.smithi084.stderr:Downloaded: 1 files, 321M in 21s (15.6 MB/s) 2024-03-21T08:03:51.413 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.b... 2024-03-21T08:03:51.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T08:03:53.353 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying... ######################################## 2024-03-21T08:03:53.474 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T08:03:53.483 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing... ######################################## 2024-03-21T08:03:53.564 INFO:teuthology.orchestra.run.smithi084.stdout:Updating / installing... 2024-03-21T08:03:53.762 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-03-21T08:03:53.762 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-03-21T08:04:00.271 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-03-21T08:04:00.502 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:04:00.611 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:04:01.097 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T08:04:01.821 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:04:01.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:04:02.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:04:02.242 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:04:02.242 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-03-21T08:04:02.242 INFO:teuthology.orchestra.run.smithi003.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T08:04:02.242 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:04:02.242 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-03-21T08:04:02.507 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.a... 2024-03-21T08:04:02.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:04:02.657 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T08:04:02.658 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T08:04:02.660 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi003.front.sepia.ceph.com to complete... 2024-03-21T08:04:09.977 INFO:teuthology.orchestra.run.smithi171.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T08:04:14.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21 08:04:14 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T08:04:14.099 INFO:teuthology.orchestra.run.smithi003.stderr:FINISHED --2024-03-21 08:04:14-- 2024-03-21T08:04:14.099 INFO:teuthology.orchestra.run.smithi003.stderr:Total wall clock time: 11s 2024-03-21T08:04:14.099 INFO:teuthology.orchestra.run.smithi003.stderr:Downloaded: 1 files, 321M in 11s (28.8 MB/s) 2024-03-21T08:04:14.102 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.a... 2024-03-21T08:04:14.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T08:04:16.138 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying... ######################################## 2024-03-21T08:04:16.195 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing... ######################################## 2024-03-21T08:04:16.279 INFO:teuthology.orchestra.run.smithi003.stdout:Updating / installing... 2024-03-21T08:04:16.550 INFO:teuthology.orchestra.run.smithi084.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T08:04:24.235 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T08:04:32.517 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T08:04:32.517 DEBUG:teuthology.orchestra.run.smithi171:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T08:04:32.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:32.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T08:04:32.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T08:04:32.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T08:04:32.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T08:04:32.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T08:04:32.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T08:04:32.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T08:04:32.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T08:04:32.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T08:04:32.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T08:04:32.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T08:04:32.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T08:04:32.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T08:04:32.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T08:04:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T08:04:32.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T08:04:32.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T08:04:32.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T08:04:32.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T08:04:32.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T08:04:32.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T08:04:32.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T08:04:32.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T08:04:32.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T08:04:32.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T08:04:32.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T08:04:32.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T08:04:32.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T08:04:32.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T08:04:32.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T08:04:32.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T08:04:32.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T08:04:32.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T08:04:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T08:04:32.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T08:04:32.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T08:04:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T08:04:32.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T08:04:32.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T08:04:32.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T08:04:32.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T08:04:32.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T08:04:32.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T08:04:32.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T08:04:32.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T08:04:32.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T08:04:32.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T08:04:32.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T08:04:32.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T08:04:32.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T08:04:32.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T08:04:32.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T08:04:32.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T08:04:32.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T08:04:32.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T08:04:32.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T08:04:32.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T08:04:32.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T08:04:32.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T08:04:32.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T08:04:32.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T08:04:32.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T08:04:32.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T08:04:32.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T08:04:32.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T08:04:32.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T08:04:32.701 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T08:04:32.702 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T08:04:32.703 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T08:04:32.704 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T08:04:32.705 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T08:04:32.706 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T08:04:32.707 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T08:04:32.708 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T08:04:32.709 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T08:04:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T08:04:32.711 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T08:04:32.712 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T08:04:32.713 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T08:04:32.714 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T08:04:32.715 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T08:04:32.716 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T08:04:32.717 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T08:04:32.718 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T08:04:32.719 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T08:04:32.720 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T08:04:32.721 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T08:04:32.722 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T08:04:32.723 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T08:04:32.724 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T08:04:32.725 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T08:04:32.726 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T08:04:32.727 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T08:04:32.728 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T08:04:32.729 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T08:04:32.730 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T08:04:32.731 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T08:04:32.732 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T08:04:32.733 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T08:04:32.734 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T08:04:32.735 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T08:04:32.736 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T08:04:32.737 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T08:04:32.738 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T08:04:32.739 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T08:04:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T08:04:32.741 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T08:04:32.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T08:04:32.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T08:04:32.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T08:04:32.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T08:04:32.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T08:04:32.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T08:04:32.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T08:04:32.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T08:04:32.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T08:04:32.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T08:04:32.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T08:04:32.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T08:04:32.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T08:04:32.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T08:04:32.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T08:04:32.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T08:04:32.758 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T08:04:32.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T08:04:32.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T08:04:32.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T08:04:32.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T08:04:32.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T08:04:32.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T08:04:32.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T08:04:32.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T08:04:32.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T08:04:32.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T08:04:32.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T08:04:32.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T08:04:32.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T08:04:32.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T08:04:32.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T08:04:32.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T08:04:32.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T08:04:32.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T08:04:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T08:04:32.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T08:04:32.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T08:04:32.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T08:04:32.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T08:04:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T08:04:32.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T08:04:32.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T08:04:32.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T08:04:32.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T08:04:32.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T08:04:32.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T08:04:32.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T08:04:32.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T08:04:32.791 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T08:04:32.792 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T08:04:32.793 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T08:04:32.794 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T08:04:32.795 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T08:04:32.796 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T08:04:32.797 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T08:04:32.798 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T08:04:32.799 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T08:04:32.800 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T08:04:32.801 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T08:04:32.802 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T08:04:32.803 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T08:04:32.804 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T08:04:32.805 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T08:04:32.806 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T08:04:32.807 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T08:04:32.808 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T08:04:32.809 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T08:04:32.810 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T08:04:32.811 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T08:04:32.812 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:04:32.816 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:32.817 DEBUG:teuthology.orchestra.run.smithi171:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:fi 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:for file in vmlinuz System.map config; do 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout: fi 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:done 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T08:04:32.840 INFO:teuthology.orchestra.run.smithi171.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T08:04:32.841 INFO:teuthology.orchestra.run.smithi171.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:04:32.841 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:32.841 INFO:teuthology.orchestra.run.smithi171.stdout:fi 2024-03-21T08:04:32.841 INFO:teuthology.orchestra.run.smithi171.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T08:04:32.841 INFO:teuthology.orchestra.run.smithi171.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T08:04:32.841 INFO:teuthology.orchestra.run.smithi171.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:32.841 INFO:teuthology.orchestra.run.smithi171.stdout:fi 2024-03-21T08:04:32.842 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -qi grub2-tools 2024-03-21T08:04:32.916 INFO:teuthology.orchestra.run.smithi171.stdout:Name : grub2-tools 2024-03-21T08:04:32.916 INFO:teuthology.orchestra.run.smithi171.stdout:Epoch : 1 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Version : 2.06 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Release : 68.el9 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: x86_64 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Group : Unspecified 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Size : 8264254 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:License : GPLv3+ 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Packager : builder@centos.org 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:Vendor : CentOS 2024-03-21T08:04:32.917 INFO:teuthology.orchestra.run.smithi171.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout:Summary : Support tools for GRUB. 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout:Description : 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout:hardware devices. 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:04:32.918 INFO:teuthology.orchestra.run.smithi171.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T08:04:32.919 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T08:04:32.919 INFO:teuthology.task.kernel:Updating grub on smithi171 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:32.919 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T08:04:33.540 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-03-21T08:04:34.970 INFO:teuthology.orchestra.run.smithi171.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T08:04:35.005 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-03-21T08:04:35.007 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-03-21T08:04:35.022 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.lz0RE7NEmQ 2024-03-21T08:04:35.022 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lz0RE7NEmQ 2024-03-21T08:04:35.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.lz0RE7NEmQ 2024-03-21T08:04:35.232 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.lz0RE7NEmQ is 6KB 2024-03-21T08:04:35.280 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.lz0RE7NEmQ 2024-03-21T08:04:35.294 DEBUG:teuthology.orchestra.run.smithi171:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T08:04:35.360 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T08:04:35.360 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T08:04:35.360 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T08:04:35.360 INFO:teuthology.orchestra.run.smithi171.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T08:04:35.361 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:35.518 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-03-21T08:04:39.019 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T08:04:39.019 DEBUG:teuthology.orchestra.run.smithi084:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T08:04:39.096 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T08:04:39.097 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T08:04:39.098 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T08:04:39.099 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T08:04:39.100 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T08:04:39.101 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T08:04:39.102 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T08:04:39.103 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T08:04:39.104 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T08:04:39.105 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T08:04:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T08:04:39.107 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T08:04:39.108 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T08:04:39.109 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T08:04:39.110 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T08:04:39.111 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T08:04:39.112 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T08:04:39.113 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T08:04:39.114 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T08:04:39.115 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T08:04:39.116 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T08:04:39.117 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T08:04:39.118 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T08:04:39.119 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T08:04:39.120 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T08:04:39.121 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T08:04:39.122 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T08:04:39.123 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T08:04:39.124 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T08:04:39.125 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T08:04:39.126 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T08:04:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T08:04:39.128 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T08:04:39.129 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T08:04:39.130 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T08:04:39.131 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T08:04:39.132 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T08:04:39.133 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T08:04:39.134 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T08:04:39.135 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T08:04:39.136 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T08:04:39.137 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T08:04:39.138 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T08:04:39.139 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T08:04:39.140 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T08:04:39.141 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T08:04:39.142 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T08:04:39.143 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T08:04:39.144 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T08:04:39.145 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T08:04:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T08:04:39.147 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T08:04:39.148 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T08:04:39.149 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T08:04:39.150 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T08:04:39.151 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T08:04:39.152 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T08:04:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T08:04:39.154 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T08:04:39.155 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T08:04:39.156 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T08:04:39.157 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T08:04:39.158 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T08:04:39.159 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T08:04:39.160 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T08:04:39.161 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T08:04:39.162 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T08:04:39.163 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T08:04:39.164 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T08:04:39.165 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T08:04:39.166 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T08:04:39.167 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T08:04:39.168 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T08:04:39.169 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T08:04:39.170 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T08:04:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T08:04:39.172 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T08:04:39.173 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T08:04:39.174 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T08:04:39.175 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T08:04:39.176 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T08:04:39.177 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T08:04:39.178 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T08:04:39.179 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T08:04:39.180 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T08:04:39.181 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T08:04:39.182 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T08:04:39.183 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T08:04:39.184 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T08:04:39.185 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T08:04:39.186 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T08:04:39.187 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T08:04:39.188 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T08:04:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T08:04:39.190 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T08:04:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T08:04:39.192 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T08:04:39.193 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T08:04:39.194 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T08:04:39.195 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T08:04:39.196 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T08:04:39.197 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T08:04:39.198 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T08:04:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T08:04:39.200 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T08:04:39.201 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T08:04:39.202 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T08:04:39.203 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T08:04:39.204 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T08:04:39.205 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T08:04:39.206 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T08:04:39.207 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T08:04:39.208 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T08:04:39.209 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T08:04:39.210 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T08:04:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T08:04:39.212 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T08:04:39.213 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T08:04:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T08:04:39.215 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T08:04:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T08:04:39.217 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T08:04:39.218 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T08:04:39.219 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T08:04:39.220 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T08:04:39.221 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T08:04:39.222 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T08:04:39.223 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T08:04:39.224 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T08:04:39.225 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T08:04:39.226 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T08:04:39.227 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T08:04:39.228 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T08:04:39.229 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T08:04:39.230 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T08:04:39.231 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T08:04:39.232 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T08:04:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T08:04:39.234 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:04:39.238 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:39.238 DEBUG:teuthology.orchestra.run.smithi084:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T08:04:39.260 INFO:teuthology.orchestra.run.smithi084.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T08:04:39.260 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:04:39.260 INFO:teuthology.orchestra.run.smithi084.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout:for file in vmlinuz System.map config; do 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout: fi 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout:done 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T08:04:39.261 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T08:04:39.262 INFO:teuthology.orchestra.run.smithi084.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T08:04:39.262 INFO:teuthology.orchestra.run.smithi084.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:04:39.262 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:39.262 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2024-03-21T08:04:39.262 INFO:teuthology.orchestra.run.smithi084.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T08:04:39.262 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T08:04:39.262 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:39.263 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2024-03-21T08:04:39.263 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2024-03-21T08:04:39.342 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2024-03-21T08:04:39.381 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2024-03-21T08:04:39.381 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.06 2024-03-21T08:04:39.381 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 68.el9 2024-03-21T08:04:39.381 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2024-03-21T08:04:39.381 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Group : Unspecified 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 8264254 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : builder@centos.org 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : CentOS 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:04:39.382 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T08:04:39.383 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T08:04:39.383 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T08:04:39.383 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2024-03-21T08:04:39.383 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:04:39.383 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T08:04:39.384 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T08:04:39.384 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:39.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T08:04:39.971 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2024-03-21T08:04:40.032 INFO:teuthology.orchestra.run.smithi003.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T08:04:41.345 INFO:teuthology.orchestra.run.smithi084.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T08:04:41.374 INFO:teuthology.orchestra.run.smithi084.stderr:done 2024-03-21T08:04:41.376 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-03-21T08:04:41.391 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.LrUWPdE7hs 2024-03-21T08:04:41.391 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LrUWPdE7hs 2024-03-21T08:04:41.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.LrUWPdE7hs 2024-03-21T08:04:41.599 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.LrUWPdE7hs is 6KB 2024-03-21T08:04:41.648 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.LrUWPdE7hs 2024-03-21T08:04:41.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T08:04:41.727 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T08:04:41.727 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T08:04:41.727 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T08:04:41.727 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T08:04:41.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T08:04:41.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2024-03-21T08:05:03.256 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T08:05:03.256 DEBUG:teuthology.orchestra.run.smithi003:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T08:05:03.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T08:05:03.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T08:05:03.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T08:05:03.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T08:05:03.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T08:05:03.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T08:05:03.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T08:05:03.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T08:05:03.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T08:05:03.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T08:05:03.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T08:05:03.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T08:05:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T08:05:03.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T08:05:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T08:05:03.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T08:05:03.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T08:05:03.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T08:05:03.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T08:05:03.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T08:05:03.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T08:05:03.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T08:05:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T08:05:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T08:05:03.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T08:05:03.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T08:05:03.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T08:05:03.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T08:05:03.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T08:05:03.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T08:05:03.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T08:05:03.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T08:05:03.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T08:05:03.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T08:05:03.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T08:05:03.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T08:05:03.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T08:05:03.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T08:05:03.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T08:05:03.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T08:05:03.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T08:05:03.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T08:05:03.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T08:05:03.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T08:05:03.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T08:05:03.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T08:05:03.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T08:05:03.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T08:05:03.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T08:05:03.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T08:05:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T08:05:03.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T08:05:03.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T08:05:03.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T08:05:03.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T08:05:03.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T08:05:03.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T08:05:03.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T08:05:03.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T08:05:03.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T08:05:03.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T08:05:03.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T08:05:03.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T08:05:03.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T08:05:03.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T08:05:03.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T08:05:03.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T08:05:03.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T08:05:03.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T08:05:03.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T08:05:03.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T08:05:03.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T08:05:03.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T08:05:03.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T08:05:03.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T08:05:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T08:05:03.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T08:05:03.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T08:05:03.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T08:05:03.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T08:05:03.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T08:05:03.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T08:05:03.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T08:05:03.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T08:05:03.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T08:05:03.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T08:05:03.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T08:05:03.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T08:05:03.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T08:05:03.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T08:05:03.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T08:05:03.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T08:05:03.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T08:05:03.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T08:05:03.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T08:05:03.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T08:05:03.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T08:05:03.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T08:05:03.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T08:05:03.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T08:05:03.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T08:05:03.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T08:05:03.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T08:05:03.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T08:05:03.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T08:05:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T08:05:03.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T08:05:03.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T08:05:03.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T08:05:03.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T08:05:03.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T08:05:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T08:05:03.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T08:05:03.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T08:05:03.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T08:05:03.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T08:05:03.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T08:05:03.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T08:05:03.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T08:05:03.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T08:05:03.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T08:05:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T08:05:03.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T08:05:03.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T08:05:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T08:05:03.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T08:05:03.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T08:05:03.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T08:05:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T08:05:03.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T08:05:03.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T08:05:03.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T08:05:03.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T08:05:03.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T08:05:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T08:05:03.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T08:05:03.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T08:05:03.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T08:05:03.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T08:05:03.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T08:05:03.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T08:05:03.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T08:05:03.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T08:05:03.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T08:05:03.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T08:05:03.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T08:05:03.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T08:05:03.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T08:05:03.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T08:05:03.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T08:05:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T08:05:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:05:03.473 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T08:05:03.473 DEBUG:teuthology.orchestra.run.smithi003:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T08:05:03.495 INFO:teuthology.orchestra.run.smithi003.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:for file in vmlinuz System.map config; do 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout: fi 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:done 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T08:05:03.496 INFO:teuthology.orchestra.run.smithi003.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:05:03.497 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:05:03.497 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-03-21T08:05:03.497 INFO:teuthology.orchestra.run.smithi003.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T08:05:03.497 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T08:05:03.497 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:05:03.497 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-03-21T08:05:03.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 68.el9 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8264254 2024-03-21T08:05:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:05:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T08:05:03.578 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T08:05:03.578 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T08:05:03.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T08:05:04.211 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-03-21T08:05:05.521 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-21T08:05:05.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-21T08:05:05.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:05:05.708 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T08:05:05.741 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-03-21T08:05:05.744 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-03-21T08:05:05.759 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.HDX4pCdcj1 2024-03-21T08:05:05.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HDX4pCdcj1 2024-03-21T08:05:05.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.HDX4pCdcj1 2024-03-21T08:05:05.996 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.HDX4pCdcj1 is 6KB 2024-03-21T08:05:06.045 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.HDX4pCdcj1 2024-03-21T08:05:06.060 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T08:05:06.134 INFO:teuthology.orchestra.run.smithi003.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T08:05:06.134 INFO:teuthology.orchestra.run.smithi003.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T08:05:06.134 INFO:teuthology.orchestra.run.smithi003.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T08:05:06.134 INFO:teuthology.orchestra.run.smithi003.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T08:05:06.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T08:05:06.316 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-03-21T08:05:11.868 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-21T08:05:11.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T08:05:11.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:05:23.871 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-03-21T08:05:30.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T08:05:32.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-21T08:05:32.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:05:35.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-03-21T08:05:36.320 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-21T08:05:36.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-03-21T08:05:36.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:05:39.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T08:05:39.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:05:42.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T08:05:47.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-21T08:05:47.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:05:54.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T08:05:54.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:05:54.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-03-21T08:06:03.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-03-21T08:06:03.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:06:47.987 DEBUG:teuthology.orchestra.remote:timed out 2024-03-21T08:06:54.347 DEBUG:teuthology.orchestra.remote:timed out 2024-03-21T08:07:02.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-21T08:07:02.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:07:03.500 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-03-21T08:07:03.857 DEBUG:teuthology.orchestra.remote:timed out 2024-03-21T08:07:04.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-21T08:07:04.150 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T08:07:04.151 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-03-21T08:07:04.203 INFO:teuthology.orchestra.run.smithi171.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T08:07:04.203 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T08:07:04.203 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T08:07:04.203 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T08:07:05.204 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-21T08:07:05.204 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T08:07:05.360 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-03-21T08:07:05.405 DEBUG:teuthology.parallel:result is None 2024-03-21T08:07:09.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T08:07:09.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:07:09.820 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-03-21T08:07:10.570 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T08:07:10.570 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T08:07:10.570 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-03-21T08:07:10.621 INFO:teuthology.orchestra.run.smithi084.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T08:07:10.622 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T08:07:10.622 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T08:07:10.622 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T08:07:11.623 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-21T08:07:11.623 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T08:07:11.772 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2024-03-21T08:07:11.841 DEBUG:teuthology.parallel:result is None 2024-03-21T08:07:15.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-03-21T08:07:15.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:07:16.235 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-03-21T08:07:16.919 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-03-21T08:07:16.920 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T08:07:16.920 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-03-21T08:07:16.972 INFO:teuthology.orchestra.run.smithi003.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T08:07:16.972 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T08:07:16.972 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T08:07:16.972 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T08:07:17.973 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-21T08:07:17.973 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T08:07:18.110 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2024-03-21T08:07:18.169 DEBUG:teuthology.parallel:result is None 2024-03-21T08:07:18.170 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T08:07:18.178 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T08:07:18.179 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:07:18.186 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:07:18.193 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:07:18.251 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T08:07:18.262 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T08:07:18.276 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T08:07:18.277 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:07:18.284 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:07:18.294 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:07:18.369 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T08:07:18.378 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T08:07:18.378 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:07:18.386 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:07:18.394 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:07:18.488 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:07:18.489 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:07:18.496 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:07:18.531 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:07:18.534 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:07:18.539 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:07:18.543 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T08:07:18.550 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T08:07:18.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:07:18.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:07:18.564 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:07:18.655 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T08:07:18.665 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T08:07:18.665 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:07:18.672 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:07:18.679 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:07:18.734 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:07:18.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:07:18.952 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:07:19.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:07:19.140 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:07:19.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:07:19.242 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:07:19.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:07:19.466 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:07:19.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:07:19.651 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:07:19.652 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:07:19.743 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:07:19.822 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:07:19.947 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:07:20.029 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:07:20.125 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:07:20.125 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:07:20.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-03-21T08:07:20.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-03-21T08:07:20.234 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-03-21T08:07:20.348 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:07:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:07:20.393 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:07:20.717 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T08:07:20.725 INFO:teuthology.task.internal:Starting timer... 2024-03-21T08:07:20.725 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T08:07:20.736 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T08:07:20.744 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-21T08:07:20.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-03-21T08:07:20.899 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-03-21T08:07:20.904 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-03-21T08:07:21.044 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-03-21T08:07:21.049 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-03-21T08:07:21.170 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-03-21T08:07:21.172 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T08:07:21.172 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-03-21T08:07:21.228 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-03-21T08:07:21.229 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-03-21T08:07:21.291 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-03-21T08:07:21.291 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-03-21T08:07:21.344 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-03-21T08:07:21.344 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-03-21T08:07:21.345 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:07:21.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:07:21.435 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:07:21.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:07:21.513 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:07:21.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:07:21.603 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T08:07:21.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:07:21.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:07:21.787 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:07:21.881 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T08:07:21.891 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-21T08:07:21.896 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-21T08:07:22.023 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T08:07:22.040 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T08:07:22.042 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi084.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T08:13:32.913 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-03-21T08:13:32.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-03-21T08:13:32.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:13:33.062 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-03-21T08:13:33.222 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-03-21T08:13:33.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T08:13:33.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:13:33.367 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-03-21T08:13:33.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T08:13:33.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-21T08:13:33.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:13:33.665 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-03-21T08:13:33.817 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-21T08:13:33.818 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T08:13:33.830 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T08:13:33.830 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:13:33.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:13:33.844 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:13:33.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:13:33.851 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:13:33.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:13:33.947 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:13:33.948 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:13:33.951 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:13:34.003 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:13:34.004 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:13:34.004 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:13:34.098 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-03-21T08:13:34.100 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-03-21T08:13:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-03-21T08:13:34.141 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-03-21T08:13:34.142 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-03-21T08:13:34.150 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-03-21T08:13:34.191 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:13:34.193 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:13:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:13:34.245 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:13:34.245 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:13:34.263 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:13:34.371 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:13:34.378 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:13:34.380 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:13:34.380 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-03-21T08:13:34.381 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.381 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.381 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.381 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.386 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:13:34.386 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-03-21T08:13:34.386 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.386 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.386 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.386 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.402 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:13:34.410 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:13:34.410 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-03-21T08:13:34.410 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.410 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.410 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.410 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:13:34.411 INFO:teuthology.run_tasks:Running task install... 2024-03-21T08:13:34.419 DEBUG:teuthology.task.install:project ceph 2024-03-21T08:13:34.420 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'}} 2024-03-21T08:13:34.420 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'} 2024-03-21T08:13:34.420 INFO:teuthology.task.install:Using flavor: default 2024-03-21T08:13:34.425 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T08:13:34.426 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-21T08:13:34.426 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:13:34.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:13:34.429 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:13:34.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:13:34.430 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:13:34.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:13:34.581 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:13:34.581 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:13:34.591 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:13:34.592 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:13:34.608 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:13:34.609 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:13:34.705 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:13:34.705 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:13:34.705 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:13:34.717 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:13:34.717 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:13:34.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:13:34.751 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:13:34.751 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:13:34.751 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:13:34.806 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:13:34.806 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:13:34.824 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:13:34.824 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:13:34.855 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:13:34.856 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:13:34.943 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:13:34.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:13:34.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:13:35.075 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:13:35.110 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:13:35.124 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:13:35.165 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-03-21T08:13:35.170 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-03-21T08:13:35.208 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-03-21T08:13:35.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-03-21T08:13:35.216 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-03-21T08:13:35.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-03-21T08:13:35.534 INFO:teuthology.orchestra.run.smithi171.stdout:74 files removed 2024-03-21T08:13:35.573 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:13:35.581 INFO:teuthology.orchestra.run.smithi003.stdout:74 files removed 2024-03-21T08:13:35.585 INFO:teuthology.orchestra.run.smithi084.stdout:74 files removed 2024-03-21T08:13:35.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:13:35.628 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:13:36.468 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-03-21T08:13:36.520 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 212 kB/s | 83 kB 00:00 2024-03-21T08:13:36.554 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-03-21T08:13:36.968 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-21T08:13:37.004 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-03-21T08:13:37.063 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-03-21T08:13:37.436 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-03-21T08:13:37.446 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-21T08:13:37.555 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-21T08:13:38.040 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-21T08:13:38.122 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-21T08:13:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-21T08:13:39.675 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-21T08:13:39.815 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2024-03-21T08:13:39.908 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-21T08:13:42.580 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.0 MB 00:02 2024-03-21T08:13:44.631 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-21T08:13:44.969 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-03-21T08:13:47.195 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-21T08:13:49.152 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 903 kB/s | 8.0 MB 00:09 2024-03-21T08:13:49.721 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.1 MB 00:01 2024-03-21T08:13:51.120 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-21T08:13:52.341 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-21T08:13:55.562 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.1 MB 00:00 2024-03-21T08:13:56.478 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 1.1 MB/s | 6.1 MB 00:05 2024-03-21T08:13:57.254 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-03-21T08:13:58.145 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-03-21T08:13:59.629 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 21 MB 00:02 2024-03-21T08:14:00.327 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-03-21T08:14:05.347 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-03-21T08:14:06.055 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-21T08:14:07.906 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:14:07.908 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:14:07.930 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:14:07.932 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:14:07.933 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:14:07.935 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:14:07.937 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:14:07.941 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:14:07.943 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:14:08.033 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================================ 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================================ 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:14:08.059 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:14:08.060 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:14:08.061 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:14:08.062 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:14:08.063 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:14:08.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:14:08.065 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:14:08.066 INFO:teuthology.orchestra.run.smithi084.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:14:08.067 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:14:08.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:14:08.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:14:08.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:14:08.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-03-21T08:14:08.072 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:14:08.073 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:14:08.073 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-03-21T08:14:08.073 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================================ 2024-03-21T08:14:08.073 INFO:teuthology.orchestra.run.smithi084.stdout:Install 150 Packages 2024-03-21T08:14:08.073 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 24 Packages 2024-03-21T08:14:08.073 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:14:08.073 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 224 M 2024-03-21T08:14:08.074 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-03-21T08:14:08.707 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:14:08.709 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:14:08.730 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:14:08.731 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:14:08.733 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:14:08.734 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:14:08.737 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:14:08.740 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:14:08.741 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:14:08.830 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-03-21T08:14:08.847 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================ 2024-03-21T08:14:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-03-21T08:14:08.847 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================ 2024-03-21T08:14:08.847 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-03-21T08:14:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:14:08.847 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:14:08.848 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:14:08.849 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:14:08.850 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:14:08.851 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:14:08.852 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:14:08.853 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:14:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:14:08.855 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:14:08.856 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:14:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:14:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:14:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:14:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:14:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:14:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-03-21T08:14:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:14:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:14:08.864 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-03-21T08:14:08.864 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================ 2024-03-21T08:14:08.864 INFO:teuthology.orchestra.run.smithi171.stdout:Install 150 Packages 2024-03-21T08:14:08.864 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 24 Packages 2024-03-21T08:14:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:14:08.865 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 224 M 2024-03-21T08:14:08.865 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-03-21T08:14:09.198 INFO:teuthology.orchestra.run.smithi084.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-21T08:14:09.433 INFO:teuthology.orchestra.run.smithi084.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 3.7 MB/s | 890 kB 00:00 2024-03-21T08:14:09.498 INFO:teuthology.orchestra.run.smithi084.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 2.2 MB/s | 145 kB 00:00 2024-03-21T08:14:09.799 INFO:teuthology.orchestra.run.smithi084.stdout:(4/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 7.7 MB/s | 2.3 MB 00:00 2024-03-21T08:14:09.999 INFO:teuthology.orchestra.run.smithi084.stdout:(5/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-03-21T08:14:10.065 INFO:teuthology.orchestra.run.smithi171.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-03-21T08:14:10.575 INFO:teuthology.orchestra.run.smithi084.stdout:(6/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-03-21T08:14:10.908 INFO:teuthology.orchestra.run.smithi084.stdout:(7/174): ceph-common-19.0.0-2231.g2edfee68.el9. 10 MB/s | 21 MB 00:01 2024-03-21T08:14:11.092 INFO:teuthology.orchestra.run.smithi084.stdout:(8/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-03-21T08:14:11.200 INFO:teuthology.orchestra.run.smithi084.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 234 kB/s | 25 kB 00:00 2024-03-21T08:14:11.801 INFO:teuthology.orchestra.run.smithi084.stdout:(10/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 11 MB/s | 9.9 MB 00:00 2024-03-21T08:14:11.848 INFO:teuthology.orchestra.run.smithi171.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 499 kB/s | 890 kB 00:01 2024-03-21T08:14:11.859 INFO:teuthology.orchestra.run.smithi084.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 3.8 MB/s | 220 kB 00:00 2024-03-21T08:14:11.934 INFO:teuthology.orchestra.run.smithi084.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 365 kB/s | 27 kB 00:00 2024-03-21T08:14:12.009 INFO:teuthology.orchestra.run.smithi084.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 422 kB/s | 31 kB 00:00 2024-03-21T08:14:12.102 INFO:teuthology.orchestra.run.smithi084.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 8.0 MB/s | 747 kB 00:00 2024-03-21T08:14:12.107 INFO:teuthology.orchestra.run.smithi171.stdout:(3/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-03-21T08:14:12.132 INFO:teuthology.orchestra.run.smithi171.stdout:(4/174): ceph-immutable-object-cache-19.0.0-223 514 kB/s | 145 kB 00:00 2024-03-21T08:14:12.193 INFO:teuthology.orchestra.run.smithi084.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 1.8 MB/s | 163 kB 00:00 2024-03-21T08:14:12.416 INFO:teuthology.orchestra.run.smithi171.stdout:(5/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 7.5 MB/s | 2.3 MB 00:00 2024-03-21T08:14:12.436 INFO:teuthology.orchestra.run.smithi084.stdout:(16/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 9.0 MB/s | 17 MB 00:01 2024-03-21T08:14:12.470 INFO:teuthology.orchestra.run.smithi084.stdout:(17/174): librados-devel-19.0.0-2231.g2edfee68. 461 kB/s | 127 kB 00:00 2024-03-21T08:14:12.528 INFO:teuthology.orchestra.run.smithi084.stdout:(18/174): libradosstriper1-19.0.0-2231.g2edfee6 5.4 MB/s | 501 kB 00:00 2024-03-21T08:14:12.620 INFO:teuthology.orchestra.run.smithi084.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 497 kB/s | 45 kB 00:00 2024-03-21T08:14:12.728 INFO:teuthology.orchestra.run.smithi084.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.2 MB/s | 131 kB 00:00 2024-03-21T08:14:12.828 INFO:teuthology.orchestra.run.smithi084.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.6 MB/s | 161 kB 00:00 2024-03-21T08:14:12.920 INFO:teuthology.orchestra.run.smithi084.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 3.4 MB/s | 320 kB 00:00 2024-03-21T08:14:13.029 INFO:teuthology.orchestra.run.smithi084.stdout:(23/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 9.2 MB/s | 5.1 MB 00:00 2024-03-21T08:14:13.090 INFO:teuthology.orchestra.run.smithi084.stdout:(24/174): python3-rbd-19.0.0-2231.g2edfee68.el9 1.7 MB/s | 297 kB 00:00 2024-03-21T08:14:13.099 INFO:teuthology.orchestra.run.smithi171.stdout:(6/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 6.9 MB/s | 4.7 MB 00:00 2024-03-21T08:14:13.113 INFO:teuthology.orchestra.run.smithi084.stdout:(25/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.2 MB/s | 99 kB 00:00 2024-03-21T08:14:13.146 INFO:teuthology.orchestra.run.smithi084.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-21T08:14:13.214 INFO:teuthology.orchestra.run.smithi084.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-21T08:14:13.280 INFO:teuthology.orchestra.run.smithi084.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 516 kB/s | 34 kB 00:00 2024-03-21T08:14:13.356 INFO:teuthology.orchestra.run.smithi084.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.8 MB/s | 140 kB 00:00 2024-03-21T08:14:13.624 INFO:teuthology.orchestra.run.smithi171.stdout:(7/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-03-21T08:14:13.673 INFO:teuthology.orchestra.run.smithi084.stdout:(30/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 5.5 MB/s | 3.1 MB 00:00 2024-03-21T08:14:13.731 INFO:teuthology.orchestra.run.smithi084.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 4.8 MB/s | 1.8 MB 00:00 2024-03-21T08:14:13.841 INFO:teuthology.orchestra.run.smithi084.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.2 MB/s | 248 kB 00:00 2024-03-21T08:14:13.923 INFO:teuthology.orchestra.run.smithi084.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 608 kB/s | 49 kB 00:00 2024-03-21T08:14:13.999 INFO:teuthology.orchestra.run.smithi084.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 222 kB/s | 16 kB 00:00 2024-03-21T08:14:14.091 INFO:teuthology.orchestra.run.smithi084.stdout:(35/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.8 MB/s | 264 kB 00:00 2024-03-21T08:14:14.258 INFO:teuthology.orchestra.run.smithi084.stdout:(36/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 4.4 MB/s | 750 kB 00:00 2024-03-21T08:14:14.349 INFO:teuthology.orchestra.run.smithi084.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 201 kB/s | 18 kB 00:00 2024-03-21T08:14:14.633 INFO:teuthology.orchestra.run.smithi084.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 662 kB 00:00 2024-03-21T08:14:14.758 INFO:teuthology.orchestra.run.smithi084.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-03-21T08:14:14.842 INFO:teuthology.orchestra.run.smithi084.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2024-03-21T08:14:14.859 INFO:teuthology.orchestra.run.smithi084.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-03-21T08:14:14.934 INFO:teuthology.orchestra.run.smithi084.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-21T08:14:14.962 INFO:teuthology.orchestra.run.smithi084.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 317 kB/s | 32 kB 00:00 2024-03-21T08:14:15.110 INFO:teuthology.orchestra.run.smithi084.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 115 kB/s | 17 kB 00:00 2024-03-21T08:14:15.343 INFO:teuthology.orchestra.run.smithi084.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-03-21T08:14:15.393 INFO:teuthology.orchestra.run.smithi084.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 36 kB/s | 10 kB 00:00 2024-03-21T08:14:15.428 INFO:teuthology.orchestra.run.smithi084.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-21T08:14:15.453 INFO:teuthology.orchestra.run.smithi084.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 44 kB 00:00 2024-03-21T08:14:15.508 INFO:teuthology.orchestra.run.smithi171.stdout:(8/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x86 7.0 MB/s | 17 MB 00:02 2024-03-21T08:14:15.553 INFO:teuthology.orchestra.run.smithi084.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 661 kB 00:00 2024-03-21T08:14:15.576 INFO:teuthology.orchestra.run.smithi171.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 384 kB/s | 25 kB 00:00 2024-03-21T08:14:15.620 INFO:teuthology.orchestra.run.smithi084.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 578 kB 00:00 2024-03-21T08:14:15.645 INFO:teuthology.orchestra.run.smithi084.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 527 kB/s | 46 kB 00:00 2024-03-21T08:14:15.712 INFO:teuthology.orchestra.run.smithi084.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 282 kB 00:00 2024-03-21T08:14:15.746 INFO:teuthology.orchestra.run.smithi084.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2024-03-21T08:14:15.796 INFO:teuthology.orchestra.run.smithi084.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-03-21T08:14:15.822 INFO:teuthology.orchestra.run.smithi084.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 460 kB/s | 34 kB 00:00 2024-03-21T08:14:16.505 INFO:teuthology.orchestra.run.smithi084.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 2.2 MB/s | 1.5 MB 00:00 2024-03-21T08:14:16.697 INFO:teuthology.orchestra.run.smithi084.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 5.3 MB/s | 4.7 MB 00:00 2024-03-21T08:14:16.764 INFO:teuthology.orchestra.run.smithi084.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 3.1 MB/s | 811 kB 00:00 2024-03-21T08:14:16.789 INFO:teuthology.orchestra.run.smithi084.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-21T08:14:17.423 INFO:teuthology.orchestra.run.smithi084.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 1.6 MB/s | 1.0 MB 00:00 2024-03-21T08:14:17.583 INFO:teuthology.orchestra.run.smithi171.stdout:(10/174): ceph-common-19.0.0-2231.g2edfee68.el9 2.6 MB/s | 21 MB 00:07 2024-03-21T08:14:17.700 INFO:teuthology.orchestra.run.smithi084.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 6.1 MB/s | 5.7 MB 00:00 2024-03-21T08:14:17.714 INFO:teuthology.orchestra.run.smithi171.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 1.7 MB/s | 220 kB 00:00 2024-03-21T08:14:17.731 INFO:teuthology.orchestra.run.smithi084.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 254 kB/s | 78 kB 00:00 2024-03-21T08:14:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 522 kB/s | 27 kB 00:00 2024-03-21T08:14:17.834 INFO:teuthology.orchestra.run.smithi171.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 477 kB/s | 31 kB 00:00 2024-03-21T08:14:17.915 INFO:teuthology.orchestra.run.smithi084.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 135 kB/s | 24 kB 00:00 2024-03-21T08:14:18.050 INFO:teuthology.orchestra.run.smithi171.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 3.4 MB/s | 747 kB 00:00 2024-03-21T08:14:18.126 INFO:teuthology.orchestra.run.smithi171.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.2 MB/s | 163 kB 00:00 2024-03-21T08:14:18.132 INFO:teuthology.orchestra.run.smithi084.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2024-03-21T08:14:18.149 INFO:teuthology.orchestra.run.smithi084.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 80 kB/s | 18 kB 00:00 2024-03-21T08:14:18.184 INFO:teuthology.orchestra.run.smithi171.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 2.2 MB/s | 127 kB 00:00 2024-03-21T08:14:18.199 INFO:teuthology.orchestra.run.smithi084.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-21T08:14:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 487 kB/s | 32 kB 00:00 2024-03-21T08:14:18.334 INFO:teuthology.orchestra.run.smithi171.stdout:(17/174): libradosstriper1-19.0.0-2231.g2edfee6 3.3 MB/s | 501 kB 00:00 2024-03-21T08:14:18.433 INFO:teuthology.orchestra.run.smithi084.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-21T08:14:18.492 INFO:teuthology.orchestra.run.smithi084.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 795 kB/s | 217 kB 00:00 2024-03-21T08:14:18.525 INFO:teuthology.orchestra.run.smithi084.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 174 kB 00:00 2024-03-21T08:14:18.551 INFO:teuthology.orchestra.run.smithi084.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-21T08:14:18.576 INFO:teuthology.orchestra.run.smithi084.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 41 kB 00:00 2024-03-21T08:14:18.610 INFO:teuthology.orchestra.run.smithi084.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-21T08:14:18.635 INFO:teuthology.orchestra.run.smithi084.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-21T08:14:18.669 INFO:teuthology.orchestra.run.smithi084.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-21T08:14:18.694 INFO:teuthology.orchestra.run.smithi084.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 842 kB/s | 47 kB 00:00 2024-03-21T08:14:18.719 INFO:teuthology.orchestra.run.smithi084.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 874 kB/s | 43 kB 00:00 2024-03-21T08:14:18.895 INFO:teuthology.orchestra.run.smithi084.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 235 kB/s | 39 kB 00:00 2024-03-21T08:14:19.201 INFO:teuthology.orchestra.run.smithi171.stdout:(18/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 1.8 MB/s | 9.9 MB 00:05 2024-03-21T08:14:19.268 INFO:teuthology.orchestra.run.smithi171.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 685 kB/s | 45 kB 00:00 2024-03-21T08:14:19.278 INFO:teuthology.orchestra.run.smithi084.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-03-21T08:14:19.303 INFO:teuthology.orchestra.run.smithi084.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 104 kB/s | 41 kB 00:00 2024-03-21T08:14:19.368 INFO:teuthology.orchestra.run.smithi171.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.3 MB/s | 131 kB 00:00 2024-03-21T08:14:19.409 INFO:teuthology.orchestra.run.smithi084.stdout:(81/174): uuid-1.6.2-55.el9.x86_64.rpm 539 kB/s | 56 kB 00:00 2024-03-21T08:14:19.495 INFO:teuthology.orchestra.run.smithi084.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-21T08:14:19.510 INFO:teuthology.orchestra.run.smithi171.stdout:(21/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 4.4 MB/s | 5.1 MB 00:01 2024-03-21T08:14:19.520 INFO:teuthology.orchestra.run.smithi084.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 585 kB/s | 63 kB 00:00 2024-03-21T08:14:19.544 INFO:teuthology.orchestra.run.smithi171.stdout:(22/174): python3-cephfs-19.0.0-2231.g2edfee68. 926 kB/s | 161 kB 00:00 2024-03-21T08:14:19.546 INFO:teuthology.orchestra.run.smithi084.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 901 kB/s | 44 kB 00:00 2024-03-21T08:14:19.571 INFO:teuthology.orchestra.run.smithi084.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-21T08:14:19.605 INFO:teuthology.orchestra.run.smithi084.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-21T08:14:19.611 INFO:teuthology.orchestra.run.smithi171.stdout:(23/174): python3-rados-19.0.0-2231.g2edfee68.e 3.2 MB/s | 320 kB 00:00 2024-03-21T08:14:19.672 INFO:teuthology.orchestra.run.smithi084.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-21T08:14:19.678 INFO:teuthology.orchestra.run.smithi171.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.5 MB/s | 99 kB 00:00 2024-03-21T08:14:19.697 INFO:teuthology.orchestra.run.smithi084.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-21T08:14:19.711 INFO:teuthology.orchestra.run.smithi171.stdout:(25/174): python3-rbd-19.0.0-2231.g2edfee68.el9 1.7 MB/s | 297 kB 00:00 2024-03-21T08:14:19.722 INFO:teuthology.orchestra.run.smithi084.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-21T08:14:19.737 INFO:teuthology.orchestra.run.smithi171.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-21T08:14:19.756 INFO:teuthology.orchestra.run.smithi084.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-21T08:14:19.804 INFO:teuthology.orchestra.run.smithi171.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-21T08:14:19.862 INFO:teuthology.orchestra.run.smithi171.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 590 kB/s | 34 kB 00:00 2024-03-21T08:14:19.929 INFO:teuthology.orchestra.run.smithi171.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 2.1 MB/s | 140 kB 00:00 2024-03-21T08:14:20.307 INFO:teuthology.orchestra.run.smithi084.stdout:(91/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 5.2 MB/s | 48 MB 00:09 2024-03-21T08:14:20.507 INFO:teuthology.orchestra.run.smithi084.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 239 kB/s | 187 kB 00:00 2024-03-21T08:14:20.540 INFO:teuthology.orchestra.run.smithi084.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 841 kB/s | 195 kB 00:00 2024-03-21T08:14:20.574 INFO:teuthology.orchestra.run.smithi084.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-21T08:14:20.616 INFO:teuthology.orchestra.run.smithi084.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 935 kB/s | 803 kB 00:00 2024-03-21T08:14:20.633 INFO:teuthology.orchestra.run.smithi084.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 235 kB/s | 21 kB 00:00 2024-03-21T08:14:20.655 INFO:teuthology.orchestra.run.smithi171.stdout:(30/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 9.4 MB/s | 48 MB 00:05 2024-03-21T08:14:20.674 INFO:teuthology.orchestra.run.smithi084.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-03-21T08:14:20.738 INFO:teuthology.orchestra.run.smithi171.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 2.2 MB/s | 1.8 MB 00:00 2024-03-21T08:14:20.750 INFO:teuthology.orchestra.run.smithi084.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-03-21T08:14:20.783 INFO:teuthology.orchestra.run.smithi084.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-03-21T08:14:20.817 INFO:teuthology.orchestra.run.smithi084.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-21T08:14:20.867 INFO:teuthology.orchestra.run.smithi084.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 422 kB/s | 35 kB 00:00 2024-03-21T08:14:20.880 INFO:teuthology.orchestra.run.smithi171.stdout:(32/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 2.6 MB/s | 3.1 MB 00:01 2024-03-21T08:14:20.914 INFO:teuthology.orchestra.run.smithi171.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2231.g2e 1.4 MB/s | 248 kB 00:00 2024-03-21T08:14:20.940 INFO:teuthology.orchestra.run.smithi171.stdout:(34/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 851 kB/s | 49 kB 00:00 2024-03-21T08:14:20.943 INFO:teuthology.orchestra.run.smithi084.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 356 kB/s | 44 kB 00:00 2024-03-21T08:14:20.964 INFO:teuthology.orchestra.run.smithi171.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2231.g2 334 kB/s | 16 kB 00:00 2024-03-21T08:14:20.968 INFO:teuthology.orchestra.run.smithi084.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 940 kB/s | 94 kB 00:00 2024-03-21T08:14:21.031 INFO:teuthology.orchestra.run.smithi171.stdout:(36/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.8 MB/s | 264 kB 00:00 2024-03-21T08:14:21.110 INFO:teuthology.orchestra.run.smithi084.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 8.4 MB/s | 4.1 MB 00:00 2024-03-21T08:14:21.115 INFO:teuthology.orchestra.run.smithi171.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 221 kB/s | 18 kB 00:00 2024-03-21T08:14:21.251 INFO:teuthology.orchestra.run.smithi084.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 73 kB/s | 22 kB 00:00 2024-03-21T08:14:21.386 INFO:teuthology.orchestra.run.smithi171.stdout:(38/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 1.7 MB/s | 750 kB 00:00 2024-03-21T08:14:21.566 INFO:teuthology.orchestra.run.smithi171.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 8.1 MB/s | 7.4 MB 00:00 2024-03-21T08:14:21.632 INFO:teuthology.orchestra.run.smithi171.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2024-03-21T08:14:21.674 INFO:teuthology.orchestra.run.smithi171.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 662 kB 00:00 2024-03-21T08:14:21.691 INFO:teuthology.orchestra.run.smithi171.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 888 kB/s | 106 kB 00:00 2024-03-21T08:14:21.725 INFO:teuthology.orchestra.run.smithi171.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.5 MB/s | 307 kB 00:00 2024-03-21T08:14:21.758 INFO:teuthology.orchestra.run.smithi171.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 500 kB/s | 32 kB 00:00 2024-03-21T08:14:21.810 INFO:teuthology.orchestra.run.smithi084.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 784 kB/s | 548 kB 00:00 2024-03-21T08:14:21.844 INFO:teuthology.orchestra.run.smithi084.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 522 kB/s | 309 kB 00:00 2024-03-21T08:14:21.875 INFO:teuthology.orchestra.run.smithi171.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-03-21T08:14:21.900 INFO:teuthology.orchestra.run.smithi171.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 101 kB/s | 17 kB 00:00 2024-03-21T08:14:21.911 INFO:teuthology.orchestra.run.smithi084.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-03-21T08:14:21.925 INFO:teuthology.orchestra.run.smithi171.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 65 kB/s | 10 kB 00:00 2024-03-21T08:14:21.997 INFO:teuthology.orchestra.run.smithi171.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 631 kB/s | 44 kB 00:00 2024-03-21T08:14:22.034 INFO:teuthology.orchestra.run.smithi171.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-21T08:14:22.036 INFO:teuthology.orchestra.run.smithi084.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 209 kB/s | 26 kB 00:00 2024-03-21T08:14:22.068 INFO:teuthology.orchestra.run.smithi171.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 661 kB 00:00 2024-03-21T08:14:22.110 INFO:teuthology.orchestra.run.smithi171.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.4 MB/s | 578 kB 00:00 2024-03-21T08:14:22.135 INFO:teuthology.orchestra.run.smithi171.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 759 kB/s | 46 kB 00:00 2024-03-21T08:14:22.160 INFO:teuthology.orchestra.run.smithi171.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 282 kB 00:00 2024-03-21T08:14:22.161 INFO:teuthology.orchestra.run.smithi084.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 393 kB/s | 49 kB 00:00 2024-03-21T08:14:22.194 INFO:teuthology.orchestra.run.smithi171.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 187 kB 00:00 2024-03-21T08:14:22.219 INFO:teuthology.orchestra.run.smithi171.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 614 kB/s | 34 kB 00:00 2024-03-21T08:14:22.261 INFO:teuthology.orchestra.run.smithi171.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.2 MB/s | 217 kB 00:00 2024-03-21T08:14:22.270 INFO:teuthology.orchestra.run.smithi084.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 626 kB/s | 67 kB 00:00 2024-03-21T08:14:22.370 INFO:teuthology.orchestra.run.smithi084.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-03-21T08:14:22.428 INFO:teuthology.orchestra.run.smithi171.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-21T08:14:22.429 INFO:teuthology.orchestra.run.smithi084.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 957 kB/s | 151 kB 00:00 2024-03-21T08:14:22.495 INFO:teuthology.orchestra.run.smithi171.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 5.8 MB/s | 1.5 MB 00:00 2024-03-21T08:14:22.504 INFO:teuthology.orchestra.run.smithi084.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-03-21T08:14:22.537 INFO:teuthology.orchestra.run.smithi171.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.0 MB/s | 811 kB 00:00 2024-03-21T08:14:22.562 INFO:teuthology.orchestra.run.smithi171.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2024-03-21T08:14:22.588 INFO:teuthology.orchestra.run.smithi084.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-03-21T08:14:22.613 INFO:teuthology.orchestra.run.smithi084.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-03-21T08:14:22.646 INFO:teuthology.orchestra.run.smithi171.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 9.5 MB/s | 1.0 MB 00:00 2024-03-21T08:14:22.689 INFO:teuthology.orchestra.run.smithi084.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-03-21T08:14:22.714 INFO:teuthology.orchestra.run.smithi084.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-03-21T08:14:22.813 INFO:teuthology.orchestra.run.smithi171.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-21T08:14:22.814 INFO:teuthology.orchestra.run.smithi084.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-21T08:14:22.847 INFO:teuthology.orchestra.run.smithi084.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-21T08:14:22.854 INFO:teuthology.orchestra.run.smithi171.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 274 kB/s | 78 kB 00:00 2024-03-21T08:14:22.939 INFO:teuthology.orchestra.run.smithi084.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-21T08:14:22.965 INFO:teuthology.orchestra.run.smithi084.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-03-21T08:14:22.988 INFO:teuthology.orchestra.run.smithi171.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-03-21T08:14:23.013 INFO:teuthology.orchestra.run.smithi171.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 131 kB/s | 24 kB 00:00 2024-03-21T08:14:23.038 INFO:teuthology.orchestra.run.smithi171.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 106 kB/s | 18 kB 00:00 2024-03-21T08:14:23.065 INFO:teuthology.orchestra.run.smithi084.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-21T08:14:23.081 INFO:teuthology.orchestra.run.smithi171.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.5 MB/s | 144 kB 00:00 2024-03-21T08:14:23.105 INFO:teuthology.orchestra.run.smithi171.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 485 kB/s | 32 kB 00:00 2024-03-21T08:14:23.172 INFO:teuthology.orchestra.run.smithi084.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.9 MB/s | 2.0 MB 00:00 2024-03-21T08:14:23.207 INFO:teuthology.orchestra.run.smithi084.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-03-21T08:14:23.247 INFO:teuthology.orchestra.run.smithi171.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-03-21T08:14:23.272 INFO:teuthology.orchestra.run.smithi171.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-21T08:14:23.274 INFO:teuthology.orchestra.run.smithi084.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 105 kB/s | 11 kB 00:00 2024-03-21T08:14:23.297 INFO:teuthology.orchestra.run.smithi171.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 988 kB/s | 174 kB 00:00 2024-03-21T08:14:23.308 INFO:teuthology.orchestra.run.smithi084.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-03-21T08:14:23.340 INFO:teuthology.orchestra.run.smithi171.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-21T08:14:23.365 INFO:teuthology.orchestra.run.smithi171.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 630 kB/s | 41 kB 00:00 2024-03-21T08:14:23.375 INFO:teuthology.orchestra.run.smithi084.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-03-21T08:14:23.390 INFO:teuthology.orchestra.run.smithi171.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-03-21T08:14:23.400 INFO:teuthology.orchestra.run.smithi084.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 213 kB/s | 19 kB 00:00 2024-03-21T08:14:23.424 INFO:teuthology.orchestra.run.smithi171.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-21T08:14:23.449 INFO:teuthology.orchestra.run.smithi171.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-21T08:14:23.467 INFO:teuthology.orchestra.run.smithi084.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-03-21T08:14:23.474 INFO:teuthology.orchestra.run.smithi171.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 591 kB/s | 47 kB 00:00 2024-03-21T08:14:23.533 INFO:teuthology.orchestra.run.smithi171.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 522 kB/s | 43 kB 00:00 2024-03-21T08:14:23.584 INFO:teuthology.orchestra.run.smithi084.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-03-21T08:14:23.626 INFO:teuthology.orchestra.run.smithi084.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-03-21T08:14:23.649 INFO:teuthology.orchestra.run.smithi171.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 240 kB/s | 39 kB 00:00 2024-03-21T08:14:23.685 INFO:teuthology.orchestra.run.smithi084.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 792 kB/s | 79 kB 00:00 2024-03-21T08:14:23.718 INFO:teuthology.orchestra.run.smithi084.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 632 kB/s | 58 kB 00:00 2024-03-21T08:14:23.794 INFO:teuthology.orchestra.run.smithi084.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-03-21T08:14:23.819 INFO:teuthology.orchestra.run.smithi084.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-03-21T08:14:23.875 INFO:teuthology.orchestra.run.smithi171.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-21T08:14:23.894 INFO:teuthology.orchestra.run.smithi084.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 904 kB/s | 90 kB 00:00 2024-03-21T08:14:23.900 INFO:teuthology.orchestra.run.smithi171.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 116 kB/s | 41 kB 00:00 2024-03-21T08:14:23.952 INFO:teuthology.orchestra.run.smithi084.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 214 kB/s | 28 kB 00:00 2024-03-21T08:14:23.955 INFO:teuthology.orchestra.run.smithi171.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-21T08:14:23.975 INFO:teuthology.orchestra.run.smithi171.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 795 kB/s | 56 kB 00:00 2024-03-21T08:14:23.995 INFO:teuthology.orchestra.run.smithi084.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-03-21T08:14:24.000 INFO:teuthology.orchestra.run.smithi171.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 653 kB/s | 63 kB 00:00 2024-03-21T08:14:24.026 INFO:teuthology.orchestra.run.smithi171.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 640 kB/s | 44 kB 00:00 2024-03-21T08:14:24.051 INFO:teuthology.orchestra.run.smithi171.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 241 kB 00:00 2024-03-21T08:14:24.062 INFO:teuthology.orchestra.run.smithi084.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-21T08:14:24.076 INFO:teuthology.orchestra.run.smithi171.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-21T08:14:24.095 INFO:teuthology.orchestra.run.smithi084.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 593 kB/s | 59 kB 00:00 2024-03-21T08:14:24.118 INFO:teuthology.orchestra.run.smithi171.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-21T08:14:24.143 INFO:teuthology.orchestra.run.smithi171.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-21T08:14:24.162 INFO:teuthology.orchestra.run.smithi084.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-03-21T08:14:24.177 INFO:teuthology.orchestra.run.smithi171.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-21T08:14:24.196 INFO:teuthology.orchestra.run.smithi084.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 512 kB/s | 51 kB 00:00 2024-03-21T08:14:24.202 INFO:teuthology.orchestra.run.smithi171.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-21T08:14:24.271 INFO:teuthology.orchestra.run.smithi084.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-21T08:14:24.297 INFO:teuthology.orchestra.run.smithi084.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 900 kB/s | 90 kB 00:00 2024-03-21T08:14:24.397 INFO:teuthology.orchestra.run.smithi084.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-21T08:14:24.422 INFO:teuthology.orchestra.run.smithi084.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-03-21T08:14:24.503 INFO:teuthology.orchestra.run.smithi084.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.8 MB/s | 192 kB 00:00 2024-03-21T08:14:24.544 INFO:teuthology.orchestra.run.smithi171.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 574 kB/s | 195 kB 00:00 2024-03-21T08:14:24.569 INFO:teuthology.orchestra.run.smithi171.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 443 kB/s | 187 kB 00:00 2024-03-21T08:14:24.570 INFO:teuthology.orchestra.run.smithi084.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-21T08:14:24.620 INFO:teuthology.orchestra.run.smithi171.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-03-21T08:14:24.654 INFO:teuthology.orchestra.run.smithi171.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-21T08:14:24.679 INFO:teuthology.orchestra.run.smithi171.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-03-21T08:14:24.723 INFO:teuthology.orchestra.run.smithi084.stdout:(150/174): protobuf-compiler-3.14.0-13.el9.x86_ 230 kB/s | 863 kB 00:03 2024-03-21T08:14:24.744 INFO:teuthology.orchestra.run.smithi171.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 206 kB/s | 25 kB 00:00 2024-03-21T08:14:24.855 INFO:teuthology.orchestra.run.smithi171.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-21T08:14:24.888 INFO:teuthology.orchestra.run.smithi084.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-03-21T08:14:24.913 INFO:teuthology.orchestra.run.smithi171.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-03-21T08:14:24.946 INFO:teuthology.orchestra.run.smithi171.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-21T08:14:24.947 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 654 kB/s | 21 MB 00:32 2024-03-21T08:14:24.972 INFO:teuthology.orchestra.run.smithi171.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 607 kB/s | 35 kB 00:00 2024-03-21T08:14:25.005 INFO:teuthology.orchestra.run.smithi171.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 774 kB/s | 44 kB 00:00 2024-03-21T08:14:25.039 INFO:teuthology.orchestra.run.smithi171.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-21T08:14:25.063 INFO:teuthology.orchestra.run.smithi084.stdout:(152/174): librados2-19.0.0-2231.g2edfee68.el9. 6.1 MB/s | 3.4 MB 00:00 2024-03-21T08:14:25.139 INFO:teuthology.orchestra.run.smithi171.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 168 kB/s | 22 kB 00:00 2024-03-21T08:14:25.180 INFO:teuthology.orchestra.run.smithi084.stdout:(153/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-03-21T08:14:25.347 INFO:teuthology.orchestra.run.smithi084.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.0 MB/s | 2.3 MB 00:00 2024-03-21T08:14:25.372 INFO:teuthology.orchestra.run.smithi084.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 220 kB/s | 42 kB 00:00 2024-03-21T08:14:25.414 INFO:teuthology.orchestra.run.smithi084.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.7 MB/s | 248 kB 00:00 2024-03-21T08:14:25.488 INFO:teuthology.orchestra.run.smithi084.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 619 kB/s | 71 kB 00:00 2024-03-21T08:14:25.517 INFO:teuthology.orchestra.run.smithi171.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-03-21T08:14:25.523 INFO:teuthology.orchestra.run.smithi084.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 338 kB/s | 36 kB 00:00 2024-03-21T08:14:25.556 INFO:teuthology.orchestra.run.smithi171.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-03-21T08:14:25.573 INFO:teuthology.orchestra.run.smithi084.stdout:(159/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.6 MB/s | 1.8 MB 00:00 2024-03-21T08:14:25.607 INFO:teuthology.orchestra.run.smithi084.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-03-21T08:14:25.632 INFO:teuthology.orchestra.run.smithi084.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 399 kB/s | 23 kB 00:00 2024-03-21T08:14:25.666 INFO:teuthology.orchestra.run.smithi084.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-21T08:14:25.682 INFO:teuthology.orchestra.run.smithi084.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-21T08:14:25.708 INFO:teuthology.orchestra.run.smithi084.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-21T08:14:25.766 INFO:teuthology.orchestra.run.smithi171.stdout:(107/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.9 MB/s | 4.1 MB 00:01 2024-03-21T08:14:25.766 INFO:teuthology.orchestra.run.smithi084.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-21T08:14:25.766 INFO:teuthology.orchestra.run.smithi084.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 852 kB/s | 64 kB 00:00 2024-03-21T08:14:25.784 INFO:teuthology.orchestra.run.smithi084.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 383 kB/s | 29 kB 00:00 2024-03-21T08:14:25.926 INFO:teuthology.orchestra.run.smithi084.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.3 MB/s | 1.2 MB 00:00 2024-03-21T08:14:26.168 INFO:teuthology.orchestra.run.smithi084.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 1.9 MB/s | 465 kB 00:00 2024-03-21T08:14:26.269 INFO:teuthology.orchestra.run.smithi084.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.7 MB/s | 2.3 MB 00:00 2024-03-21T08:14:26.293 INFO:teuthology.orchestra.run.smithi084.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 250 kB/s | 31 kB 00:00 2024-03-21T08:14:26.360 INFO:teuthology.orchestra.run.smithi084.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 2.2 MB 00:00 2024-03-21T08:14:26.385 INFO:teuthology.orchestra.run.smithi084.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 815 kB/s | 74 kB 00:00 2024-03-21T08:14:26.477 INFO:teuthology.orchestra.run.smithi084.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.2 MB/s | 1.1 MB 00:00 2024-03-21T08:14:26.485 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:14:26.486 INFO:teuthology.orchestra.run.smithi084.stdout:Total 12 MB/s | 224 MB 00:18 2024-03-21T08:14:26.590 INFO:teuthology.orchestra.run.smithi171.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 289 kB/s | 309 kB 00:01 2024-03-21T08:14:26.615 INFO:teuthology.orchestra.run.smithi171.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 18 kB/s | 19 kB 00:01 2024-03-21T08:14:27.940 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-03-21T08:14:28.315 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-03-21T08:14:28.315 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-03-21T08:14:30.669 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-03-21T08:14:30.669 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-03-21T08:14:31.021 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-03-21T08:14:32.473 INFO:teuthology.orchestra.run.smithi171.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 4.4 kB/s | 26 kB 00:05 2024-03-21T08:14:32.740 INFO:teuthology.orchestra.run.smithi171.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 7.9 kB/s | 49 kB 00:06 2024-03-21T08:14:33.648 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:14:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:14:33.672 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:14:33.673 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:14:33.675 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:14:33.676 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:14:33.679 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:14:33.682 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:14:33.684 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:14:33.776 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-03-21T08:14:33.803 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================================ 2024-03-21T08:14:33.803 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================================ 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:14:33.804 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:14:33.805 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:14:33.806 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:14:33.807 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:14:33.808 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:14:33.809 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:14:33.810 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:14:33.811 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:14:33.812 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:14:33.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:14:33.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:14:33.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:14:33.816 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:14:33.817 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================================ 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout:Install 150 Packages 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 24 Packages 2024-03-21T08:14:33.818 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:14:33.819 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 224 M 2024-03-21T08:14:33.819 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-03-21T08:14:35.224 INFO:teuthology.orchestra.run.smithi003.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-21T08:14:35.444 INFO:teuthology.orchestra.run.smithi003.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 4.0 MB/s | 890 kB 00:00 2024-03-21T08:14:35.507 INFO:teuthology.orchestra.run.smithi003.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 2.3 MB/s | 145 kB 00:00 2024-03-21T08:14:35.849 INFO:teuthology.orchestra.run.smithi003.stdout:(4/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-03-21T08:14:35.924 INFO:teuthology.orchestra.run.smithi003.stdout:(5/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 5.6 MB/s | 2.3 MB 00:00 2024-03-21T08:14:36.058 INFO:teuthology.orchestra.run.smithi003.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-03-21T08:14:36.608 INFO:teuthology.orchestra.run.smithi003.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 6.9 MB/s | 4.7 MB 00:00 2024-03-21T08:14:37.350 INFO:teuthology.orchestra.run.smithi003.stdout:(8/174): ceph-common-19.0.0-2231.g2edfee68.el9. 8.5 MB/s | 21 MB 00:02 2024-03-21T08:14:37.417 INFO:teuthology.orchestra.run.smithi003.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 384 kB/s | 25 kB 00:00 2024-03-21T08:14:37.818 INFO:teuthology.orchestra.run.smithi003.stdout:(10/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 9.6 MB/s | 17 MB 00:01 2024-03-21T08:14:37.893 INFO:teuthology.orchestra.run.smithi003.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 2.9 MB/s | 220 kB 00:00 2024-03-21T08:14:38.060 INFO:teuthology.orchestra.run.smithi003.stdout:(12/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 6.8 MB/s | 9.9 MB 00:01 2024-03-21T08:14:38.085 INFO:teuthology.orchestra.run.smithi003.stdout:(13/174): cephfs-shell-19.0.0-2231.g2edfee68.el 141 kB/s | 27 kB 00:00 2024-03-21T08:14:38.110 INFO:teuthology.orchestra.run.smithi003.stdout:(14/174): libcephfs-devel-19.0.0-2231.g2edfee68 638 kB/s | 31 kB 00:00 2024-03-21T08:14:38.177 INFO:teuthology.orchestra.run.smithi003.stdout:(15/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 8.0 MB/s | 747 kB 00:00 2024-03-21T08:14:38.211 INFO:teuthology.orchestra.run.smithi003.stdout:(16/174): libcephsqlite-19.0.0-2231.g2edfee68.e 1.6 MB/s | 163 kB 00:00 2024-03-21T08:14:38.244 INFO:teuthology.orchestra.run.smithi003.stdout:(17/174): librados-devel-19.0.0-2231.g2edfee68. 1.9 MB/s | 127 kB 00:00 2024-03-21T08:14:38.272 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-03-21T08:14:38.303 INFO:teuthology.orchestra.run.smithi003.stdout:(18/174): libradosstriper1-19.0.0-2231.g2edfee6 5.4 MB/s | 501 kB 00:00 2024-03-21T08:14:38.370 INFO:teuthology.orchestra.run.smithi003.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 687 kB/s | 45 kB 00:00 2024-03-21T08:14:38.443 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:14:38.453 INFO:teuthology.orchestra.run.smithi003.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.6 MB/s | 131 kB 00:00 2024-03-21T08:14:38.499 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:14:38.537 INFO:teuthology.orchestra.run.smithi003.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.9 MB/s | 161 kB 00:00 2024-03-21T08:14:38.545 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:14:38.589 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:14:38.637 INFO:teuthology.orchestra.run.smithi003.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 3.2 MB/s | 320 kB 00:00 2024-03-21T08:14:38.695 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:14:38.779 INFO:teuthology.orchestra.run.smithi003.stdout:(23/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 9.6 MB/s | 5.1 MB 00:00 2024-03-21T08:14:38.790 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:14:38.821 INFO:teuthology.orchestra.run.smithi003.stdout:(24/174): python3-rbd-19.0.0-2231.g2edfee68.el9 1.6 MB/s | 297 kB 00:00 2024-03-21T08:14:38.855 INFO:teuthology.orchestra.run.smithi003.stdout:(25/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.3 MB/s | 99 kB 00:00 2024-03-21T08:14:38.888 INFO:teuthology.orchestra.run.smithi003.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-21T08:14:38.930 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:14:38.955 INFO:teuthology.orchestra.run.smithi003.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-21T08:14:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:14:39.030 INFO:teuthology.orchestra.run.smithi003.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 459 kB/s | 34 kB 00:00 2024-03-21T08:14:39.097 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:14:39.156 INFO:teuthology.orchestra.run.smithi003.stdout:(29/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 10 MB/s | 3.1 MB 00:00 2024-03-21T08:14:39.189 INFO:teuthology.orchestra.run.smithi003.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 886 kB/s | 140 kB 00:00 2024-03-21T08:14:39.390 INFO:teuthology.orchestra.run.smithi003.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 7.7 MB/s | 1.8 MB 00:00 2024-03-21T08:14:39.482 INFO:teuthology.orchestra.run.smithi003.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.7 MB/s | 248 kB 00:00 2024-03-21T08:14:39.523 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:14:39.548 INFO:teuthology.orchestra.run.smithi003.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 752 kB/s | 49 kB 00:00 2024-03-21T08:14:39.570 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:14:39.615 INFO:teuthology.orchestra.run.smithi003.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 250 kB/s | 16 kB 00:00 2024-03-21T08:14:39.699 INFO:teuthology.orchestra.run.smithi003.stdout:(35/174): ceph-volume-19.0.0-2231.g2edfee68.el9 3.1 MB/s | 264 kB 00:00 2024-03-21T08:14:39.720 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:14:39.764 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:14:39.866 INFO:teuthology.orchestra.run.smithi003.stdout:(36/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 4.4 MB/s | 750 kB 00:00 2024-03-21T08:14:39.894 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:14:39.936 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:14:39.941 INFO:teuthology.orchestra.run.smithi003.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 246 kB/s | 18 kB 00:00 2024-03-21T08:14:40.001 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:14:40.134 INFO:teuthology.orchestra.run.smithi003.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-03-21T08:14:40.234 INFO:teuthology.orchestra.run.smithi003.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-21T08:14:40.276 INFO:teuthology.orchestra.run.smithi003.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 920 kB 00:00 2024-03-21T08:14:40.301 INFO:teuthology.orchestra.run.smithi003.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 106 kB 00:00 2024-03-21T08:14:40.301 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:14:40.335 INFO:teuthology.orchestra.run.smithi003.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-21T08:14:40.360 INFO:teuthology.orchestra.run.smithi003.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2024-03-21T08:14:40.367 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:14:40.392 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:14:40.469 INFO:teuthology.orchestra.run.smithi003.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-03-21T08:14:40.494 INFO:teuthology.orchestra.run.smithi003.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 146 kB/s | 17 kB 00:00 2024-03-21T08:14:40.523 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:14:40.536 INFO:teuthology.orchestra.run.smithi003.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 249 kB/s | 10 kB 00:00 2024-03-21T08:14:40.561 INFO:teuthology.orchestra.run.smithi003.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 159 kB 00:00 2024-03-21T08:14:40.577 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:14:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 924 kB/s | 44 kB 00:00 2024-03-21T08:14:40.628 INFO:teuthology.orchestra.run.smithi003.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-03-21T08:14:40.667 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:14:40.687 INFO:teuthology.orchestra.run.smithi003.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.0 MB/s | 578 kB 00:00 2024-03-21T08:14:40.712 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:14:40.720 INFO:teuthology.orchestra.run.smithi003.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 540 kB/s | 46 kB 00:00 2024-03-21T08:14:40.745 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:14:40.754 INFO:teuthology.orchestra.run.smithi003.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-21T08:14:40.788 INFO:teuthology.orchestra.run.smithi003.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-21T08:14:40.808 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:14:40.821 INFO:teuthology.orchestra.run.smithi003.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-21T08:14:40.846 INFO:teuthology.orchestra.run.smithi003.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 616 kB/s | 34 kB 00:00 2024-03-21T08:14:40.989 INFO:teuthology.orchestra.run.smithi003.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2024-03-21T08:14:41.056 INFO:teuthology.orchestra.run.smithi003.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 7.9 MB/s | 1.5 MB 00:00 2024-03-21T08:14:41.097 INFO:teuthology.orchestra.run.smithi003.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.4 MB/s | 811 kB 00:00 2024-03-21T08:14:41.123 INFO:teuthology.orchestra.run.smithi003.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2024-03-21T08:14:41.128 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:14:41.178 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:14:41.220 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:14:41.273 INFO:teuthology.orchestra.run.smithi003.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-03-21T08:14:41.323 INFO:teuthology.orchestra.run.smithi003.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-03-21T08:14:41.348 INFO:teuthology.orchestra.run.smithi003.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-21T08:14:41.507 INFO:teuthology.orchestra.run.smithi003.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-03-21T08:14:41.532 INFO:teuthology.orchestra.run.smithi003.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 147 kB/s | 24 kB 00:00 2024-03-21T08:14:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:14:41.583 INFO:teuthology.orchestra.run.smithi003.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 369 kB/s | 18 kB 00:00 2024-03-21T08:14:41.608 INFO:teuthology.orchestra.run.smithi003.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-21T08:14:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 1.3 MB/s | 32 kB 00:00 2024-03-21T08:14:41.735 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:14:41.767 INFO:teuthology.orchestra.run.smithi003.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-21T08:14:41.800 INFO:teuthology.orchestra.run.smithi003.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-21T08:14:41.824 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:14:41.834 INFO:teuthology.orchestra.run.smithi003.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-21T08:14:41.859 INFO:teuthology.orchestra.run.smithi003.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-21T08:14:41.885 INFO:teuthology.orchestra.run.smithi003.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 41 kB 00:00 2024-03-21T08:14:41.910 INFO:teuthology.orchestra.run.smithi003.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-03-21T08:14:41.935 INFO:teuthology.orchestra.run.smithi003.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2024-03-21T08:14:41.969 INFO:teuthology.orchestra.run.smithi003.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-21T08:14:41.994 INFO:teuthology.orchestra.run.smithi003.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 848 kB/s | 47 kB 00:00 2024-03-21T08:14:42.019 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:14:42.019 INFO:teuthology.orchestra.run.smithi003.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 876 kB/s | 43 kB 00:00 2024-03-21T08:14:42.080 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:14:42.436 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:14:42.553 INFO:teuthology.orchestra.run.smithi003.stdout:(78/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 9.3 MB/s | 48 MB 00:05 2024-03-21T08:14:42.633 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:14:42.687 INFO:teuthology.orchestra.run.smithi003.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 59 kB/s | 39 kB 00:00 2024-03-21T08:14:42.762 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:14:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:14:42.858 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:14:42.912 INFO:teuthology.orchestra.run.smithi003.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-03-21T08:14:42.937 INFO:teuthology.orchestra.run.smithi003.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 111 kB/s | 41 kB 00:00 2024-03-21T08:14:42.960 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:14:42.964 INFO:teuthology.orchestra.run.smithi171.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 6.4 kB/s | 67 kB 00:10 2024-03-21T08:14:42.971 INFO:teuthology.orchestra.run.smithi003.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-03-21T08:14:42.996 INFO:teuthology.orchestra.run.smithi003.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-21T08:14:43.021 INFO:teuthology.orchestra.run.smithi003.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 786 kB/s | 63 kB 00:00 2024-03-21T08:14:43.046 INFO:teuthology.orchestra.run.smithi003.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 599 kB/s | 44 kB 00:00 2024-03-21T08:14:43.080 INFO:teuthology.orchestra.run.smithi003.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-21T08:14:43.084 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:14:43.105 INFO:teuthology.orchestra.run.smithi003.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-03-21T08:14:43.155 INFO:teuthology.orchestra.run.smithi003.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-21T08:14:43.163 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:14:43.172 INFO:teuthology.orchestra.run.smithi003.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-21T08:14:43.197 INFO:teuthology.orchestra.run.smithi003.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-21T08:14:43.212 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:14:43.233 INFO:teuthology.orchestra.run.smithi003.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 187 kB 00:00 2024-03-21T08:14:43.306 INFO:teuthology.orchestra.run.smithi003.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-03-21T08:14:43.340 INFO:teuthology.orchestra.run.smithi003.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-21T08:14:43.342 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:14:43.390 INFO:teuthology.orchestra.run.smithi003.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-03-21T08:14:43.392 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:14:43.432 INFO:teuthology.orchestra.run.smithi003.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-03-21T08:14:43.457 INFO:teuthology.orchestra.run.smithi003.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 185 kB/s | 21 kB 00:00 2024-03-21T08:14:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:14:43.512 INFO:teuthology.orchestra.run.smithi003.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 211 kB/s | 25 kB 00:00 2024-03-21T08:14:43.530 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:14:43.566 INFO:teuthology.orchestra.run.smithi003.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-21T08:14:43.570 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:14:43.634 INFO:teuthology.orchestra.run.smithi003.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-21T08:14:43.667 INFO:teuthology.orchestra.run.smithi003.stdout:(100/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.6 MB/s | 248 kB 00:00 2024-03-21T08:14:43.690 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:14:43.692 INFO:teuthology.orchestra.run.smithi003.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 604 kB/s | 35 kB 00:00 2024-03-21T08:14:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-21T08:14:43.734 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:14:43.743 INFO:teuthology.orchestra.run.smithi003.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-03-21T08:14:43.783 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:14:43.821 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:14:43.852 INFO:teuthology.orchestra.run.smithi003.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.9 MB/s | 4.1 MB 00:00 2024-03-21T08:14:43.877 INFO:teuthology.orchestra.run.smithi003.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-03-21T08:14:43.942 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:14:43.997 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:14:44.096 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:14:44.119 INFO:teuthology.orchestra.run.smithi003.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-03-21T08:14:44.152 INFO:teuthology.orchestra.run.smithi003.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-21T08:14:44.178 INFO:teuthology.orchestra.run.smithi003.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 336 kB/s | 19 kB 00:00 2024-03-21T08:14:44.228 INFO:teuthology.orchestra.run.smithi003.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-03-21T08:14:44.245 INFO:teuthology.orchestra.run.smithi003.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-03-21T08:14:44.284 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:14:44.286 INFO:teuthology.orchestra.run.smithi003.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 842 kB/s | 49 kB 00:00 2024-03-21T08:14:44.312 INFO:teuthology.orchestra.run.smithi003.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-21T08:14:44.345 INFO:teuthology.orchestra.run.smithi003.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-03-21T08:14:44.352 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:14:44.438 INFO:teuthology.orchestra.run.smithi003.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-03-21T08:14:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-03-21T08:14:44.496 INFO:teuthology.orchestra.run.smithi003.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 752 kB/s | 43 kB 00:00 2024-03-21T08:14:44.530 INFO:teuthology.orchestra.run.smithi003.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 558 kB/s | 32 kB 00:00 2024-03-21T08:14:44.555 INFO:teuthology.orchestra.run.smithi003.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-03-21T08:14:44.588 INFO:teuthology.orchestra.run.smithi003.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-21T08:14:44.622 INFO:teuthology.orchestra.run.smithi003.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-03-21T08:14:44.656 INFO:teuthology.orchestra.run.smithi003.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-03-21T08:14:44.681 INFO:teuthology.orchestra.run.smithi003.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 604 kB/s | 35 kB 00:00 2024-03-21T08:14:44.706 INFO:teuthology.orchestra.run.smithi003.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-03-21T08:14:44.790 INFO:teuthology.orchestra.run.smithi003.stdout:(124/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-03-21T08:14:44.824 INFO:teuthology.orchestra.run.smithi003.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-21T08:14:44.849 INFO:teuthology.orchestra.run.smithi003.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-03-21T08:14:44.883 INFO:teuthology.orchestra.run.smithi003.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 305 kB/s | 18 kB 00:00 2024-03-21T08:14:44.939 INFO:teuthology.orchestra.run.smithi003.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 261 kB/s | 23 kB 00:00 2024-03-21T08:14:44.965 INFO:teuthology.orchestra.run.smithi003.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 241 kB/s | 19 kB 00:00 2024-03-21T08:14:44.997 INFO:teuthology.orchestra.run.smithi003.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 340 kB/s | 20 kB 00:00 2024-03-21T08:14:45.048 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:14:45.090 INFO:teuthology.orchestra.run.smithi003.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-03-21T08:14:45.115 INFO:teuthology.orchestra.run.smithi003.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-03-21T08:14:45.139 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:14:45.157 INFO:teuthology.orchestra.run.smithi003.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-03-21T08:14:45.182 INFO:teuthology.orchestra.run.smithi003.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 872 kB/s | 58 kB 00:00 2024-03-21T08:14:45.208 INFO:teuthology.orchestra.run.smithi003.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 675 kB/s | 79 kB 00:00 2024-03-21T08:14:45.233 INFO:teuthology.orchestra.run.smithi003.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-03-21T08:14:45.258 INFO:teuthology.orchestra.run.smithi003.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-03-21T08:14:45.264 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:14:45.283 INFO:teuthology.orchestra.run.smithi003.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-21T08:14:45.317 INFO:teuthology.orchestra.run.smithi003.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 494 kB/s | 28 kB 00:00 2024-03-21T08:14:45.342 INFO:teuthology.orchestra.run.smithi003.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 374 kB/s | 31 kB 00:00 2024-03-21T08:14:45.349 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:14:45.375 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:14:45.384 INFO:teuthology.orchestra.run.smithi003.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-21T08:14:45.409 INFO:teuthology.orchestra.run.smithi003.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 540 kB/s | 36 kB 00:00 2024-03-21T08:14:45.426 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:14:45.435 INFO:teuthology.orchestra.run.smithi003.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 507 kB/s | 59 kB 00:00 2024-03-21T08:14:45.460 INFO:teuthology.orchestra.run.smithi003.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 682 kB/s | 51 kB 00:00 2024-03-21T08:14:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-21T08:14:45.504 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:14:45.519 INFO:teuthology.orchestra.run.smithi003.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-21T08:14:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-21T08:14:45.561 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:14:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-03-21T08:14:45.626 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:14:45.636 INFO:teuthology.orchestra.run.smithi003.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-21T08:14:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-21T08:14:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-03-21T08:14:45.946 INFO:teuthology.orchestra.run.smithi003.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-21T08:14:46.029 INFO:teuthology.orchestra.run.smithi003.stdout:(153/174): librados2-19.0.0-2231.g2edfee68.el9. 7.7 MB/s | 3.4 MB 00:00 2024-03-21T08:14:46.030 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:14:46.105 INFO:teuthology.orchestra.run.smithi003.stdout:(154/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 6.7 MB/s | 3.1 MB 00:00 2024-03-21T08:14:46.130 INFO:teuthology.orchestra.run.smithi003.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 425 kB/s | 42 kB 00:00 2024-03-21T08:14:46.214 INFO:teuthology.orchestra.run.smithi003.stdout:(156/174): NetworkManager-libnm-1.47.2-1.el9.x8 6.9 MB/s | 1.8 MB 00:00 2024-03-21T08:14:46.239 INFO:teuthology.orchestra.run.smithi003.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 665 kB/s | 71 kB 00:00 2024-03-21T08:14:46.264 INFO:teuthology.orchestra.run.smithi003.stdout:(158/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.5 MB/s | 248 kB 00:00 2024-03-21T08:14:46.270 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:14:46.290 INFO:teuthology.orchestra.run.smithi003.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 488 kB/s | 36 kB 00:00 2024-03-21T08:14:46.348 INFO:teuthology.orchestra.run.smithi003.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-21T08:14:46.373 INFO:teuthology.orchestra.run.smithi003.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 280 kB/s | 23 kB 00:00 2024-03-21T08:14:46.402 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:14:46.407 INFO:teuthology.orchestra.run.smithi003.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-21T08:14:46.449 INFO:teuthology.orchestra.run.smithi003.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-21T08:14:46.455 INFO:teuthology.orchestra.run.smithi171.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 11 kB/s | 151 kB 00:13 2024-03-21T08:14:46.469 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:14:46.474 INFO:teuthology.orchestra.run.smithi003.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-21T08:14:46.499 INFO:teuthology.orchestra.run.smithi003.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 136 kB 00:00 2024-03-21T08:14:46.541 INFO:teuthology.orchestra.run.smithi003.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 701 kB/s | 64 kB 00:00 2024-03-21T08:14:46.566 INFO:teuthology.orchestra.run.smithi003.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 315 kB/s | 29 kB 00:00 2024-03-21T08:14:46.647 INFO:teuthology.orchestra.run.smithi171.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 216 kB/s | 4.4 MB 00:20 2024-03-21T08:14:46.676 INFO:teuthology.orchestra.run.smithi003.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.9 MB/s | 1.2 MB 00:00 2024-03-21T08:14:46.687 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:14:46.741 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:14:46.775 INFO:teuthology.orchestra.run.smithi003.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.3 MB/s | 2.2 MB 00:00 2024-03-21T08:14:46.842 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:14:46.843 INFO:teuthology.orchestra.run.smithi003.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.3 MB/s | 2.3 MB 00:00 2024-03-21T08:14:46.884 INFO:teuthology.orchestra.run.smithi003.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.2 MB/s | 465 kB 00:00 2024-03-21T08:14:46.909 INFO:teuthology.orchestra.run.smithi003.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 235 kB/s | 31 kB 00:00 2024-03-21T08:14:46.938 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:14:46.968 INFO:teuthology.orchestra.run.smithi003.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 903 kB/s | 74 kB 00:00 2024-03-21T08:14:46.982 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:14:47.021 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:14:47.051 INFO:teuthology.orchestra.run.smithi003.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.2 MB/s | 1.1 MB 00:00 2024-03-21T08:14:47.060 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:14:47.060 INFO:teuthology.orchestra.run.smithi003.stdout:Total 17 MB/s | 224 MB 00:13 2024-03-21T08:14:47.089 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:14:47.129 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:14:47.215 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:14:47.284 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:14:47.474 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:14:47.566 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:14:47.609 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:14:47.671 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:14:47.809 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:14:47.856 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:14:47.914 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:14:47.955 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:14:47.988 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:14:48.162 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:14:48.339 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:14:48.479 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:14:48.546 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-03-21T08:14:48.551 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:14:48.612 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:14:48.695 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:14:48.943 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-03-21T08:14:48.943 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-03-21T08:14:48.949 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:14:49.009 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:14:49.187 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:14:49.240 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:14:49.280 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:14:49.318 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:14:49.359 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:14:49.404 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:14:49.462 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:14:49.521 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:14:49.570 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:14:49.706 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:14:49.749 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:14:49.792 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:14:49.844 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:14:49.895 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:14:50.037 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:14:50.155 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:14:50.224 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:14:50.277 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:14:50.335 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:14:50.475 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:14:50.543 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:14:50.597 INFO:teuthology.orchestra.run.smithi171.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 110 kB/s | 838 kB 00:07 2024-03-21T08:14:51.418 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-03-21T08:14:51.418 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-03-21T08:14:51.496 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:14:51.558 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:14:51.592 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:14:51.648 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:14:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:14:51.800 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:14:52.024 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:14:52.056 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:14:52.156 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:14:52.714 INFO:teuthology.orchestra.run.smithi171.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 88 kB/s | 548 kB 00:06 2024-03-21T08:14:52.814 INFO:teuthology.orchestra.run.smithi171.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 7.0 kB/s | 43 kB 00:06 2024-03-21T08:14:52.906 INFO:teuthology.orchestra.run.smithi171.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 14 kB/s | 32 kB 00:02 2024-03-21T08:14:52.956 INFO:teuthology.orchestra.run.smithi171.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 62 kB/s | 15 kB 00:00 2024-03-21T08:14:52.965 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:14:53.193 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:14:53.406 INFO:teuthology.orchestra.run.smithi171.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 290 kB/s | 172 kB 00:00 2024-03-21T08:14:54.265 INFO:teuthology.orchestra.run.smithi171.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 264 kB/s | 358 kB 00:01 2024-03-21T08:14:54.726 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:14:54.817 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:14:54.872 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:14:54.903 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:14:54.953 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:14:55.006 INFO:teuthology.orchestra.run.smithi171.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 148 kB/s | 303 kB 00:02 2024-03-21T08:14:55.057 INFO:teuthology.orchestra.run.smithi171.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 21 kB/s | 35 kB 00:01 2024-03-21T08:14:55.327 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:14:55.358 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:14:55.477 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:14:55.505 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:14:55.567 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:14:56.032 INFO:teuthology.orchestra.run.smithi171.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 115 kB/s | 203 kB 00:01 2024-03-21T08:14:56.098 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:14:56.136 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:14:56.226 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:14:56.254 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:14:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-03-21T08:14:58.885 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:14:58.917 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:14:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:14:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:14:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:14:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:14:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:14:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:14:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:14:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:14:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:14:59.536 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:14:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:14:59.741 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:14:59.741 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:14:59.741 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:14:59.741 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:14:59.779 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:14:59.924 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:14:59.970 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:15:00.130 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:15:00.162 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:15:00.286 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:15:00.331 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:15:00.402 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:15:00.741 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:15:00.800 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:15:00.827 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:15:00.952 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:15:00.984 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:15:01.074 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:15:01.144 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:15:01.170 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:15:01.241 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:15:01.525 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:15:01.578 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:15:01.629 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:15:01.942 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:15:02.151 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:15:02.224 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:15:02.431 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:15:02.479 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:15:02.829 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:15:03.016 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:15:03.134 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:15:03.177 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:15:03.206 INFO:teuthology.orchestra.run.smithi171.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 1.5 kB/s | 11 kB 00:07 2024-03-21T08:15:03.232 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:15:03.302 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:15:03.426 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:15:03.496 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:15:03.546 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:15:03.648 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:15:03.691 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:15:03.751 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:15:03.794 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:15:03.827 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:15:03.942 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:15:03.981 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:15:04.037 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:15:04.071 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:15:04.073 INFO:teuthology.orchestra.run.smithi171.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 230 kB/s | 2.0 MB 00:09 2024-03-21T08:15:04.132 INFO:teuthology.orchestra.run.smithi171.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 16 kB/s | 144 kB 00:09 2024-03-21T08:15:04.148 INFO:teuthology.orchestra.run.smithi171.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 19 kB/s | 18 kB 00:00 2024-03-21T08:15:04.182 INFO:teuthology.orchestra.run.smithi171.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-03-21T08:15:04.191 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:15:04.199 INFO:teuthology.orchestra.run.smithi171.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 293 kB/s | 19 kB 00:00 2024-03-21T08:15:04.224 INFO:teuthology.orchestra.run.smithi171.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-03-21T08:15:04.252 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:15:04.367 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:15:04.563 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:15:04.659 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:15:05.383 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:15:05.481 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:15:05.615 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:15:05.691 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:15:05.720 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:15:05.772 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:15:05.893 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:15:06.278 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:15:06.337 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:15:06.782 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:15:06.883 INFO:teuthology.orchestra.run.smithi171.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 387 kB/s | 1.0 MB 00:02 2024-03-21T08:15:07.024 INFO:teuthology.orchestra.run.smithi171.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 16 kB/s | 46 kB 00:02 2024-03-21T08:15:07.026 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:15:07.146 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:15:07.166 INFO:teuthology.orchestra.run.smithi171.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 27 kB/s | 79 kB 00:02 2024-03-21T08:15:07.216 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:15:07.266 INFO:teuthology.orchestra.run.smithi171.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 150 kB/s | 58 kB 00:00 2024-03-21T08:15:07.437 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:15:07.492 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:15:07.593 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:15:07.694 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:15:07.734 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:15:07.773 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:15:07.844 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:15:07.907 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:15:07.972 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:15:08.040 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:15:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:15:08.295 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:15:08.333 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:15:08.392 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:15:08.442 INFO:teuthology.orchestra.run.smithi171.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 192 kB/s | 272 kB 00:01 2024-03-21T08:15:08.492 INFO:teuthology.orchestra.run.smithi171.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 12 kB/s | 16 kB 00:01 2024-03-21T08:15:08.539 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:15:08.590 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:15:08.639 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:15:08.673 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:15:08.706 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:15:08.809 INFO:teuthology.orchestra.run.smithi171.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 58 kB/s | 90 kB 00:01 2024-03-21T08:15:08.880 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:15:08.934 INFO:teuthology.orchestra.run.smithi171.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 57 kB/s | 28 kB 00:00 2024-03-21T08:15:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:15:09.084 INFO:teuthology.orchestra.run.smithi171.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 52 kB/s | 31 kB 00:00 2024-03-21T08:15:09.224 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:15:09.283 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:15:09.345 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:15:09.421 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:15:09.674 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:15:09.744 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:15:09.835 INFO:teuthology.orchestra.run.smithi171.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 184 kB/s | 188 kB 00:01 2024-03-21T08:15:09.931 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:15:09.982 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:15:10.023 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:15:10.043 INFO:teuthology.orchestra.run.smithi171.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 53 kB/s | 59 kB 00:01 2024-03-21T08:15:10.061 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:15:10.102 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:15:10.148 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:15:10.215 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:15:10.251 INFO:teuthology.orchestra.run.smithi171.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 31 kB/s | 36 kB 00:01 2024-03-21T08:15:10.283 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:15:10.358 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:15:10.485 INFO:teuthology.orchestra.run.smithi171.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 78 kB/s | 51 kB 00:00 2024-03-21T08:15:10.505 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:15:10.553 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:15:10.597 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:15:10.662 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:15:10.707 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:15:10.862 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:15:10.977 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:15:10.993 INFO:teuthology.orchestra.run.smithi171.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 242 kB/s | 230 kB 00:00 2024-03-21T08:15:11.066 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:15:11.114 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:15:11.175 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:15:11.331 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:15:11.397 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:15:11.427 INFO:teuthology.orchestra.run.smithi171.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 76 kB/s | 90 kB 00:01 2024-03-21T08:15:12.396 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:15:12.470 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:15:12.505 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:15:12.568 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-03-21T08:15:12.573 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:12.691 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:15:12.729 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:15:12.791 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:15:12.838 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:15:12.838 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:12.838 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:15:12.838 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:12.978 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:15:13.008 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:15:13.108 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:15:13.135 INFO:teuthology.orchestra.run.smithi171.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 161 kB/s | 427 kB 00:02 2024-03-21T08:15:13.186 INFO:teuthology.orchestra.run.smithi171.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 9.1 kB/s | 20 kB 00:02 2024-03-21T08:15:13.528 INFO:teuthology.orchestra.run.smithi171.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 91 kB/s | 192 kB 00:02 2024-03-21T08:15:13.745 INFO:teuthology.orchestra.run.smithi171.stdout:(150/174): librados2-19.0.0-2231.g2edfee68.el9. 6.1 MB/s | 3.4 MB 00:00 2024-03-21T08:15:13.912 INFO:teuthology.orchestra.run.smithi171.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.9 MB/s | 321 kB 00:00 2024-03-21T08:15:13.954 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:15:14.094 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:15:14.104 INFO:teuthology.orchestra.run.smithi171.stdout:(152/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-03-21T08:15:14.390 INFO:teuthology.orchestra.run.smithi171.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.8 MB/s | 2.3 MB 00:00 2024-03-21T08:15:14.393 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:15:14.463 INFO:teuthology.orchestra.run.smithi171.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 580 kB/s | 42 kB 00:00 2024-03-21T08:15:14.469 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:15:14.469 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:14.469 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:15:14.469 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:15:14.469 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:15:14.469 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:14.539 INFO:teuthology.orchestra.run.smithi171.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.2 MB/s | 1.8 MB 00:00 2024-03-21T08:15:14.581 INFO:teuthology.orchestra.run.smithi171.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-03-21T08:15:14.631 INFO:teuthology.orchestra.run.smithi171.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 785 kB/s | 71 kB 00:00 2024-03-21T08:15:14.656 INFO:teuthology.orchestra.run.smithi171.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 491 kB/s | 36 kB 00:00 2024-03-21T08:15:14.707 INFO:teuthology.orchestra.run.smithi171.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-21T08:15:14.709 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:15:14.749 INFO:teuthology.orchestra.run.smithi171.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-21T08:15:14.776 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:15:14.776 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:14.776 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:15:14.776 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:15:14.776 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:15:14.777 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:14.807 INFO:teuthology.orchestra.run.smithi171.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 231 kB/s | 23 kB 00:00 2024-03-21T08:15:14.841 INFO:teuthology.orchestra.run.smithi171.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 108 kB 00:00 2024-03-21T08:15:14.891 INFO:teuthology.orchestra.run.smithi171.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-21T08:15:14.925 INFO:teuthology.orchestra.run.smithi171.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-21T08:15:14.975 INFO:teuthology.orchestra.run.smithi171.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 772 kB/s | 64 kB 00:00 2024-03-21T08:15:14.992 INFO:teuthology.orchestra.run.smithi171.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 435 kB/s | 29 kB 00:00 2024-03-21T08:15:15.101 INFO:teuthology.orchestra.run.smithi171.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-03-21T08:15:15.218 INFO:teuthology.orchestra.run.smithi171.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.6 MB/s | 2.2 MB 00:00 2024-03-21T08:15:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:15:15.301 INFO:teuthology.orchestra.run.smithi171.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-21T08:15:15.343 INFO:teuthology.orchestra.run.smithi171.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.7 MB/s | 465 kB 00:00 2024-03-21T08:15:15.352 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:15:15.353 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:15.353 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:15:15.353 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:15:15.353 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:15:15.353 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:15.376 INFO:teuthology.orchestra.run.smithi171.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 418 kB/s | 31 kB 00:00 2024-03-21T08:15:15.452 INFO:teuthology.orchestra.run.smithi171.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 10 MB/s | 1.1 MB 00:00 2024-03-21T08:15:15.470 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:15:15.519 INFO:teuthology.orchestra.run.smithi171.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 527 kB/s | 74 kB 00:00 2024-03-21T08:15:15.702 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:15:15.787 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:15:15.825 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:15:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:15:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:15:16.152 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:15:16.188 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:15:16.238 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:15:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:15:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:15:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:15:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:15:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:15:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:15:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:15:16.711 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:15:16.767 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:15:17.110 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:15:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:15:17.239 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:15:17.266 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:15:17.473 INFO:teuthology.orchestra.run.smithi171.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 373 kB/s | 1.6 MB 00:04 2024-03-21T08:15:17.487 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:15:17.488 INFO:teuthology.orchestra.run.smithi171.stdout:Total 3.3 MB/s | 224 MB 01:08 2024-03-21T08:15:17.886 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:15:17.926 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:15:18.070 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:15:18.239 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:15:18.294 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:15:18.363 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:15:18.363 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:18.363 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:15:18.363 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:15:18.363 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:15:18.364 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:15:18.457 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:15:18.542 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:15:18.902 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-03-21T08:15:19.284 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-03-21T08:15:19.284 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-03-21T08:15:19.539 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:15:19.572 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:15:19.659 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:15:19.659 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:19.660 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:15:19.660 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:15:19.660 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:15:19.660 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:19.714 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:15:19.804 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:15:19.804 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:19.804 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:15:19.804 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:19.890 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:15:19.994 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:15:19.994 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:19.994 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:15:19.994 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:15:19.994 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:15:19.994 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:15:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:15:20.382 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:15:20.466 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:15:20.466 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:20.466 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:15:20.466 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:15:20.467 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:15:20.467 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:15:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:15:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:15:20.977 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:15:20.977 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:15:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:21.025 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:15:21.629 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-03-21T08:15:21.629 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-03-21T08:15:25.289 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:15:25.335 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:15:25.371 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:15:25.408 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:15:25.683 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:15:25.808 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:15:25.863 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:15:25.913 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:15:26.011 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:15:26.059 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:15:26.095 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:15:26.139 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:15:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:15:26.274 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:15:26.386 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:15:26.438 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:15:27.336 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:15:27.385 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:15:27.442 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:15:28.770 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:15:28.856 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:15:28.913 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:15:29.042 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:15:29.109 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:15:29.151 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:15:29.185 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:15:29.193 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-03-21T08:15:29.228 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:15:29.228 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:15:29.320 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:15:29.365 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:15:29.372 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:15:29.414 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:15:29.414 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:15:29.432 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:15:29.477 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:15:29.560 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:15:29.568 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:15:29.609 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:15:29.668 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:15:29.679 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:15:29.726 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:15:29.738 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:15:29.758 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:15:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:15:29.831 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:15:29.896 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:15:29.914 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:15:29.963 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:15:29.996 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:15:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:15:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:15:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:15:30.140 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:15:30.140 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:15:30.308 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:15:30.375 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:15:30.472 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:15:30.473 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:15:30.526 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:15:30.551 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:15:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:15:30.674 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:15:30.688 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:15:30.728 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:15:30.733 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:15:30.761 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:15:30.798 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:15:30.801 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:15:30.929 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:15:31.112 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:15:31.163 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:15:31.189 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:15:31.204 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:15:31.205 INFO:teuthology.orchestra.run.smithi084.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T08:15:31.205 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:31.357 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:15:31.390 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:15:31.495 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:15:31.525 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:15:31.550 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:15:31.611 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:15:31.948 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:15:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:15:32.076 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:15:32.431 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:15:32.667 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:15:32.749 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:15:32.944 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:15:33.014 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:15:33.335 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:15:33.519 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:15:33.648 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:15:33.695 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:15:33.734 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:15:33.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:15:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:15:34.004 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:15:34.055 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:15:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:15:34.217 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:15:34.285 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:15:34.334 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:15:34.382 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:15:34.515 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:15:34.561 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-03-21T08:15:34.569 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:34.644 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:15:34.678 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:15:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:15:34.801 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:15:34.843 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:15:34.844 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:34.844 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:15:34.844 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:34.866 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:15:34.974 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:15:35.160 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:15:35.224 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:15:35.935 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:15:36.024 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:15:36.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:15:36.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:15:36.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:15:36.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:15:36.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:15:36.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:15:36.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:15:36.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:15:36.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:15:36.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:15:36.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:15:36.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:15:36.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:15:36.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:15:36.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:15:36.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:15:36.041 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:15:36.147 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:15:36.225 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:15:36.250 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:15:36.299 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:15:36.381 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:15:36.447 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:15:36.493 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:15:36.514 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:15:36.605 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:15:36.605 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:36.605 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:15:36.605 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:15:36.606 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:15:36.606 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:36.864 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:15:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:15:36.950 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:15:36.950 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:36.950 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:15:36.950 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:15:36.950 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:15:36.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:37.164 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:15:37.289 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:15:37.358 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:15:37.456 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:15:37.539 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:15:37.539 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:37.539 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:15:37.539 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:15:37.539 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:15:37.539 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:37.570 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:15:37.631 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:15:37.658 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:15:37.723 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:15:37.820 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:15:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:15:37.895 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:15:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:15:38.005 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:15:38.091 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:15:38.163 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:15:38.310 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:15:38.342 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:15:38.362 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:15:38.393 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:15:38.441 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:15:38.484 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:15:38.497 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:15:38.529 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:15:38.543 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:15:38.686 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:15:38.725 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:15:38.745 INFO:teuthology.orchestra.run.smithi084.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.746 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:15:38.747 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:15:38.747 INFO:teuthology.orchestra.run.smithi084.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.747 INFO:teuthology.orchestra.run.smithi084.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.747 INFO:teuthology.orchestra.run.smithi084.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.748 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.749 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:15:38.750 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.751 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:15:38.752 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:15:38.753 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:15:38.754 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:15:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:15:38.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:15:38.757 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:15:38.758 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:15:38.759 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:15:38.759 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-03-21T08:15:38.759 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-03-21T08:15:38.770 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:15:38.833 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:15:38.866 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:15:38.873 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:15:38.914 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:15:39.044 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:15:39.229 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:15:39.342 DEBUG:teuthology.parallel:result is None 2024-03-21T08:15:39.374 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:15:39.436 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:15:39.499 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:15:39.574 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:15:39.840 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:15:39.900 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:15:40.075 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:15:40.092 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:15:40.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:15:40.124 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:15:40.162 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:15:40.193 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:15:40.242 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:15:40.274 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:15:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:15:40.347 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:15:40.418 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:15:40.460 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:15:40.462 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:15:40.493 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:15:40.572 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:15:40.572 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:40.572 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:15:40.572 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:15:40.572 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:15:40.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:40.588 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:15:40.631 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:15:40.635 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:15:40.667 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:15:40.675 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:15:40.730 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:15:40.767 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:15:40.788 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:15:40.935 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:15:41.048 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:15:41.109 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:15:41.170 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:15:41.237 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:15:41.379 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:15:41.444 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:15:41.791 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:15:41.829 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:15:41.910 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:15:41.911 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:41.911 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:15:41.911 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:15:41.911 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:15:41.911 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:41.963 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:15:42.061 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:15:42.062 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:42.062 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:15:42.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:42.139 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:15:42.242 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:15:42.242 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:42.243 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:15:42.243 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:15:42.243 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:15:42.243 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:42.386 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:15:42.445 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:15:42.487 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:15:42.553 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:15:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:15:42.665 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:15:42.714 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:15:42.714 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:15:42.715 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:15:42.715 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:15:42.715 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:15:42.715 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:15:42.715 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:42.945 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:15:42.978 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:15:43.069 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:15:43.869 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:15:44.007 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:15:45.532 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:15:45.697 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:15:45.736 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:15:45.767 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:15:45.834 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:15:46.216 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:15:46.247 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:15:46.349 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:15:46.378 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:15:46.432 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:15:46.943 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:15:46.993 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:15:47.067 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:15:47.103 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:15:47.796 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:15:47.842 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:15:47.877 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:15:47.914 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:15:48.198 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:15:48.336 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:15:48.387 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:15:48.449 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:15:48.511 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:15:48.557 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:15:48.591 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:15:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:15:48.680 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:15:48.769 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:15:48.870 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:15:48.910 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:15:49.769 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:15:49.802 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:15:49.965 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:15:50.016 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:15:50.074 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:15:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:15:50.465 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:15:50.641 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:15:50.642 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:15:50.642 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:15:50.642 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:15:50.682 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:15:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:15:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:15:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:15:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:15:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:15:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:15:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:15:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:15:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:15:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:15:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:15:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:15:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:15:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:15:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:15:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:15:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:15:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:15:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:15:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:15:52.684 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:15:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:15:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:15:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:15:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:15:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:15:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:15:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:15:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:15:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:15:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:15:53.457 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:15:53.497 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:15:53.620 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:15:53.886 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:15:53.886 INFO:teuthology.orchestra.run.smithi003.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T08:15:53.887 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:15:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:15:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:15:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:15:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:15:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:15:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:15:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:15:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:15:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:15:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:15:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:15:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:15:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:15:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:15:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:15:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:16:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:16:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:16:01.435 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-03-21T08:16:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:01.435 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.436 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:16:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.438 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:16:01.439 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.440 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:16:01.441 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:16:01.442 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:16:01.444 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:16:01.445 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:16:01.446 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:16:01.447 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:16:01.448 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:16:01.449 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:16:01.449 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-03-21T08:16:02.268 DEBUG:teuthology.parallel:result is None 2024-03-21T08:16:03.762 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:16:03.762 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-03-21T08:16:03.762 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-03-21T08:16:03.762 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-03-21T08:16:03.763 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-03-21T08:16:03.763 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-03-21T08:16:03.763 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-03-21T08:16:03.763 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-03-21T08:16:03.763 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:03.985 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:16:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:16:04.035 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:04.035 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:16:04.035 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:05.623 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:16:05.711 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:16:05.711 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:05.712 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:16:05.712 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:16:05.712 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:16:05.712 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:05.956 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:16:06.024 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:16:06.024 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:06.024 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:16:06.024 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:16:06.024 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:16:06.024 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:06.508 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:16:06.580 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:16:06.580 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:06.580 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:16:06.580 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:16:06.580 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:16:06.581 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:06.716 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:16:07.359 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:16:07.396 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:16:07.446 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:16:07.579 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:16:07.616 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:16:07.953 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:16:07.992 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:16:09.111 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:16:09.143 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:16:09.283 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:16:09.465 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:16:09.496 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:16:09.592 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:16:09.592 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:09.592 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:16:09.592 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:16:09.593 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:16:09.593 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:09.647 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:16:09.685 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:16:09.773 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:16:10.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:16:10.824 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:16:10.892 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:16:10.892 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:10.893 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:16:10.893 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:16:10.893 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:16:10.893 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:10.976 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:16:11.044 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:16:11.044 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:11.045 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:16:11.045 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:11.118 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:16:11.190 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:16:11.190 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:11.190 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:16:11.190 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:16:11.190 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:16:11.190 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:11.561 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:16:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:16:11.630 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:16:11.630 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:16:11.630 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:16:11.630 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:16:11.630 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:16.436 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:16:16.480 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:16:16.517 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:16:16.554 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:16:16.837 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:16:16.963 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:16:17.020 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:16:17.074 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:16:17.162 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:16:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:16:17.253 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:16:17.297 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:16:17.348 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:16:17.446 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:16:17.581 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:16:17.622 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:16:18.530 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:16:18.580 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:16:18.648 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:16:19.969 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:16:20.055 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:16:20.103 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:16:20.245 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:16:20.284 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:16:20.317 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:16:20.352 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:16:20.393 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:16:20.393 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:16:20.539 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:16:20.598 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:16:20.598 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:16:20.736 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:16:20.785 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:16:20.826 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:16:20.869 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:16:20.901 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:16:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:16:21.066 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:16:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:16:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:16:21.183 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:16:21.216 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:16:21.250 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:16:21.287 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:16:21.288 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:16:21.610 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:16:21.610 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:16:21.765 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:16:21.795 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:16:21.850 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:16:21.890 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:16:21.939 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:16:22.060 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:16:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:16:22.316 INFO:teuthology.orchestra.run.smithi171.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T08:16:22.316 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:27.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:16:27.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:16:27.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:16:27.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:16:27.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:16:27.418 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:16:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:16:27.420 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:16:27.421 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:16:27.422 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:16:27.423 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:16:27.424 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:16:27.425 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:16:27.426 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:16:27.427 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:16:27.428 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:16:27.429 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:16:27.430 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:16:27.431 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:16:27.432 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:16:27.433 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:16:27.434 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:16:27.435 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:16:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:16:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:30.068 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-03-21T08:16:30.068 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.069 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:16:30.070 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.071 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.072 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:16:30.073 INFO:teuthology.orchestra.run.smithi171.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.074 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:16:30.075 INFO:teuthology.orchestra.run.smithi171.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:16:30.076 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:16:30.077 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:16:30.078 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:16:30.079 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:16:30.080 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:16:30.081 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:16:30.082 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:16:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:16:30.083 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:16:30.083 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-03-21T08:16:30.747 DEBUG:teuthology.parallel:result is None 2024-03-21T08:16:30.747 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:16:30.908 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:16:30.986 INFO:teuthology.orchestra.run.smithi003.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:16:30.987 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:16:30.987 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:16:30.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:16:31.159 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:16:31.233 INFO:teuthology.orchestra.run.smithi084.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:16:31.233 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:16:31.233 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:16:31.235 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:16:31.382 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:16:31.448 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:16:31.448 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:16:31.448 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:16:31.450 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T08:16:31.450 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:16:31.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:16:31.559 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:16:31.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:16:31.658 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:16:31.659 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:16:31.746 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T08:16:31.747 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:16:31.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:16:31.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:16:31.992 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:16:31.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:16:32.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:16:32.216 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:16:32.216 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:16:32.296 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:16:32.406 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T08:16:32.407 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:16:32.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:16:32.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:16:32.657 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:16:32.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:16:32.745 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:16:32.884 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:16:32.884 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:16:32.952 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:16:33.062 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T08:16:33.062 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:16:33.062 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:16:33.167 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:16:33.271 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:16:33.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:16:33.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:16:33.463 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:16:33.463 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:16:33.536 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:16:33.647 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T08:16:33.760 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'subvols': {'create': 2}} 2024-03-21T08:16:33.761 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:16:33.762 INFO:tasks.cephadm:Cluster fsid is 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:16:33.762 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T08:16:33.762 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.3', 'mon.b': '172.21.15.84', 'mon.c': '172.21.15.171'} 2024-03-21T08:16:33.762 INFO:tasks.cephadm:First mon is mon.a on smithi003 2024-03-21T08:16:33.762 INFO:tasks.cephadm:First mgr is x 2024-03-21T08:16:33.762 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T08:16:33.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname $(hostname -s) 2024-03-21T08:16:33.870 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname $(hostname -s) 2024-03-21T08:16:33.971 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-03-21T08:16:34.048 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-21T08:16:34.048 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:16:34.290 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77791/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2231-g2edfee68', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2231.g2edfee68'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 07:25:08.579979', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240320.053002', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-21T08:16:34.530 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240320.053002, sha1 2edfee68ad7243bed422a049b6f5f58619300f11 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:16:34.532 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm 2024-03-21T08:16:34.532 DEBUG:teuthology.orchestra.run.smithi003:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:35.547 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:16 /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:35.547 DEBUG:teuthology.orchestra.run.smithi084:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:36.691 INFO:teuthology.orchestra.run.smithi084.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:16 /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:36.691 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:37.898 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:16 /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:37.899 DEBUG:teuthology.orchestra.run.smithi003:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:37.963 DEBUG:teuthology.orchestra.run.smithi084:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:38.025 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:16:38.130 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 on all hosts... 2024-03-21T08:16:38.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:16:38.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:16:38.148 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:16:39.273 INFO:teuthology.orchestra.run.smithi084.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:16:39.354 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:16:39.387 INFO:teuthology.orchestra.run.smithi003.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:17:17.113 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-21T08:17:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:17:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:17:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-03-21T08:17:17.113 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:17:17.114 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-03-21T08:17:17.114 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-21T08:17:35.021 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-03-21T08:17:35.022 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:17:35.022 INFO:teuthology.orchestra.run.smithi084.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:17:35.022 INFO:teuthology.orchestra.run.smithi084.stdout: "repo_digests": [ 2024-03-21T08:17:35.022 INFO:teuthology.orchestra.run.smithi084.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:17:35.022 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-03-21T08:17:35.022 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-03-21T08:17:38.249 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:17:38.249 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:17:38.249 INFO:teuthology.orchestra.run.smithi003.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:17:38.250 INFO:teuthology.orchestra.run.smithi003.stdout: "repo_digests": [ 2024-03-21T08:17:38.250 INFO:teuthology.orchestra.run.smithi003.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:17:38.250 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:17:38.250 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:17:38.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph 2024-03-21T08:17:38.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph 2024-03-21T08:17:38.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-03-21T08:17:38.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /etc/ceph 2024-03-21T08:17:38.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /etc/ceph 2024-03-21T08:17:38.806 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-03-21T08:17:38.906 INFO:tasks.cephadm:Writing seed config... 2024-03-21T08:17:38.907 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-21T08:17:38.907 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-21T08:17:38.908 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-21T08:17:38.909 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-21T08:17:38.910 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-21T08:17:38.911 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:17:38.911 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T08:17:38.971 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 544fc91e-e75b-11ee-95cd-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-21T08:17:38.971 DEBUG:teuthology.orchestra.run.smithi003:mon.a> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a.service 2024-03-21T08:17:39.019 DEBUG:teuthology.orchestra.run.smithi003:mgr.x> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.x.service 2024-03-21T08:17:39.065 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T08:17:39.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 -v bootstrap --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.3 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:17:39.342 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:17:39.342 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11', '-v', 'bootstrap', '--fsid', '544fc91e-e75b-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.3', '--skip-admin-label'] 2024-03-21T08:17:39.343 INFO:teuthology.orchestra.run.smithi003.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T08:17:39.343 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying podman|docker is present... 2024-03-21T08:17:39.398 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T08:17:39.399 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying lvm2 is present... 2024-03-21T08:17:39.399 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying time synchronization is in place... 2024-03-21T08:17:39.414 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T08:17:39.414 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T08:17:39.428 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T08:17:39.428 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-03-21T08:17:39.448 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-03-21T08:17:39.469 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-03-21T08:17:39.469 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-03-21T08:17:39.469 INFO:teuthology.orchestra.run.smithi003.stdout:Repeating the final host check... 2024-03-21T08:17:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T08:17:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T08:17:39.524 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl is present 2024-03-21T08:17:39.524 INFO:teuthology.orchestra.run.smithi003.stdout:lvcreate is present 2024-03-21T08:17:39.538 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T08:17:39.538 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T08:17:39.555 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T08:17:39.555 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-03-21T08:17:39.578 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-03-21T08:17:39.600 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-03-21T08:17:39.601 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-03-21T08:17:39.601 INFO:teuthology.orchestra.run.smithi003.stdout:Host looks OK 2024-03-21T08:17:39.602 INFO:teuthology.orchestra.run.smithi003.stdout:Cluster fsid: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:39.602 INFO:teuthology.orchestra.run.smithi003.stdout:Acquiring lock 140188280550352 on /run/cephadm/544fc91e-e75b-11ee-95cd-87774f69a715.lock 2024-03-21T08:17:39.603 INFO:teuthology.orchestra.run.smithi003.stdout:Lock 140188280550352 acquired on /run/cephadm/544fc91e-e75b-11ee-95cd-87774f69a715.lock 2024-03-21T08:17:39.604 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 3300 ... 2024-03-21T08:17:39.605 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 6789 ... 2024-03-21T08:17:39.606 INFO:teuthology.orchestra.run.smithi003.stdout:Base mon IP(s) is [172.21.15.3:3300, 172.21.15.3:6789], mon addrv is [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-03-21T08:17:39.623 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.3 metric 100 2024-03-21T08:17:39.623 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.3 metric 100 2024-03-21T08:17:39.636 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T08:17:39.636 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-21T08:17:39.636 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-21T08:17:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T08:17:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T08:17:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T08:17:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-21T08:17:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-03-21T08:17:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T08:17:39.651 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-03-21T08:17:39.652 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-03-21T08:17:39.652 INFO:teuthology.orchestra.run.smithi003.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-21T08:17:39.655 INFO:teuthology.orchestra.run.smithi003.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T08:17:39.657 INFO:teuthology.orchestra.run.smithi003.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:17:40.430 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T08:17:40.430 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:17:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T08:17:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:96a207f569d15e36f8e9e3b2feafd8af7b93861f24c2b60ada6c9667dd5405be 2024-03-21T08:17:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-21T08:17:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying config sha256:237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T08:17:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T08:17:41.549 INFO:teuthology.orchestra.run.smithi003.stdout:ceph: stdout ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T08:17:41.550 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph version: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T08:17:41.550 INFO:teuthology.orchestra.run.smithi003.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T08:17:42.855 INFO:teuthology.orchestra.run.smithi003.stdout:stat: stdout 167 167 2024-03-21T08:17:42.855 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial keys... 2024-03-21T08:17:43.957 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAn7ftlZTqmEhAAu3Vi8XLoV+4lFERRH9vMRQ== 2024-03-21T08:17:45.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAo7ftlya9mGhAA1TaGp04kGdU8s1+cxcCXqw== 2024-03-21T08:17:46.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAp7ftlXVMyIRAAhDACtCjAPRWtRn9ygoLgeA== 2024-03-21T08:17:46.172 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial monmap... 2024-03-21T08:17:47.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T08:17:47.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-21T08:17:47.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:47.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T08:17:47.253 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool for a [v2:172.21.15.3:3300,v1:172.21.15.3:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T08:17:47.254 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-03-21T08:17:47.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: set fsid to 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:47.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T08:17:47.254 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:17:47.254 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mon... 2024-03-21T08:17:48.316 INFO:teuthology.orchestra.run.smithi003.stdout:create mon.a on 2024-03-21T08:17:48.629 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T08:17:48.870 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T08:17:49.164 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-544fc91e-e75b-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-544fc91e-e75b-11ee-95cd-87774f69a715.target. 2024-03-21T08:17:49.164 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-544fc91e-e75b-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-544fc91e-e75b-11ee-95cd-87774f69a715.target. 2024-03-21T08:17:49.569 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a 2024-03-21T08:17:49.569 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-21T08:17:49.868 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-544fc91e-e75b-11ee-95cd-87774f69a715.target.wants/ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-544fc91e-e75b-11ee-95cd-87774f69a715@.service. 2024-03-21T08:17:50.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:50 smithi003 podman[31289]: 2024-03-21 08:17:50.506124962 +0000 UTC m=+0.365521937 container init 4062767ccc36a9a50bf4e465aa918e8db2e03ef4d021049aa3535fe4d835117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-21T08:17:50.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:50 smithi003 podman[31289]: 2024-03-21 08:17:50.516700378 +0000 UTC m=+0.376097351 container start 4062767ccc36a9a50bf4e465aa918e8db2e03ef4d021049aa3535fe4d835117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:17:50.647 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:17:50.647 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-03-21T08:17:50.665 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:17:50.665 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-03-21T08:17:50.665 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-03-21T08:17:50.665 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T08:17:50.667 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon to start... 2024-03-21T08:17:50.667 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon... 2024-03-21T08:17:50.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:50 smithi003 ceph-mon[31308]: mkfs 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:50.820 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:50 smithi003 ceph-mon[31308]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:17:50.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:50 smithi003 bash[31289]: 4062767ccc36a9a50bf4e465aa918e8db2e03ef4d021049aa3535fe4d835117b 2024-03-21T08:17:50.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:50 smithi003 systemd[1]: Started Ceph mon.a for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:17:51.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:51 smithi003 ceph-mon[31308]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:17:51.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:51 smithi003 ceph-mon[31308]: monmap e1: 1 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:17:51.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:51 smithi003 ceph-mon[31308]: fsmap 2024-03-21T08:17:51.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:51 smithi003 ceph-mon[31308]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T08:17:51.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:51 smithi003 ceph-mon[31308]: mgrmap e1: no daemons active 2024-03-21T08:17:51.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:51 smithi003 ceph-mon[31308]: from='client.? 172.21.15.3:0/888024785' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout id: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout services: 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.757092s) 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T08:17:52.023 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:17:52.024 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout data: 2024-03-21T08:17:52.024 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T08:17:52.024 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T08:17:52.024 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T08:17:52.024 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T08:17:52.025 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:17:52.025 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.259+0000 7fc9b3600700 1 Processor -- start 2024-03-21T08:17:52.025 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.259+0000 7fc9b3600700 1 -- start start 2024-03-21T08:17:52.026 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.259+0000 7fc9b3600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac106560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:17:52.026 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.259+0000 7fc9b3600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac07ae30 con 0x7fc9ac106140 2024-03-21T08:17:52.026 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.260+0000 7fc9b0c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:17:52.026 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.260+0000 7fc9b0c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55080/0 (socket says 172.21.15.3:55080) 2024-03-21T08:17:52.026 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.260+0000 7fc9b0c00700 1 -- 172.21.15.3:0/2097925510 learned_addr learned my addr 172.21.15.3:0/2097925510 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.261+0000 7fc9b0c00700 1 -- 172.21.15.3:0/2097925510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac106aa0 con 0x7fc9ac106140 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.261+0000 7fc9b0c00700 1 --2- 172.21.15.3:0/2097925510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac106560 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00098a0 tx=0x7fc9a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf74f5c4d1ba261 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.261+0000 7fc9ab400700 1 -- 172.21.15.3:0/2097925510 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9a0004160 con 0x7fc9ac106140 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.261+0000 7fc9ab400700 1 -- 172.21.15.3:0/2097925510 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0004300 con 0x7fc9ac106140 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.261+0000 7fc9ab400700 1 -- 172.21.15.3:0/2097925510 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9a00036a0 con 0x7fc9ac106140 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.262+0000 7fc9b3600700 1 -- 172.21.15.3:0/2097925510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 msgr2=0x7fc9ac106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.262+0000 7fc9b3600700 1 --2- 172.21.15.3:0/2097925510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac106560 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00098a0 tx=0x7fc9a0009bb0 comp rx=0 tx=0).stop 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.262+0000 7fc9b3600700 1 -- 172.21.15.3:0/2097925510 shutdown_connections 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.262+0000 7fc9b3600700 1 --2- 172.21.15.3:0/2097925510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac106560 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.262+0000 7fc9b3600700 1 -- 172.21.15.3:0/2097925510 >> 172.21.15.3:0/2097925510 conn(0x7fc9ac101de0 msgr2=0x7fc9ac104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.262+0000 7fc9b3600700 1 -- 172.21.15.3:0/2097925510 shutdown_connections 2024-03-21T08:17:52.027 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.262+0000 7fc9b3600700 1 -- 172.21.15.3:0/2097925510 wait complete. 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.263+0000 7fc9b3600700 1 Processor -- start 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.264+0000 7fc9b3600700 1 -- start start 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.264+0000 7fc9b3600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.264+0000 7fc9b3600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac106fd0 con 0x7fc9ac106140 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.264+0000 7fc9b0c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac19d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.264+0000 7fc9b0c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac19d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55096/0 (socket says 172.21.15.3:55096) 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.264+0000 7fc9b0c00700 1 -- 172.21.15.3:0/888024785 learned_addr learned my addr 172.21.15.3:0/888024785 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.265+0000 7fc9b0c00700 1 -- 172.21.15.3:0/888024785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac19d6c0 con 0x7fc9ac106140 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.265+0000 7fc9b0c00700 1 --2- 172.21.15.3:0/888024785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac19d180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0000c00 tx=0x7fc9a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.265+0000 7fc9a9600700 1 -- 172.21.15.3:0/888024785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9a0003db0 con 0x7fc9ac106140 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.265+0000 7fc9a9600700 1 -- 172.21.15.3:0/888024785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0011630 con 0x7fc9ac106140 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.265+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac19d950 con 0x7fc9ac106140 2024-03-21T08:17:52.028 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.265+0000 7fc9a9600700 1 -- 172.21.15.3:0/888024785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9a0011950 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.265+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac1a0640 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.266+0000 7fc9a9600700 1 -- 172.21.15.3:0/888024785 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc9a001a460 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.266+0000 7fc9a9600700 1 -- 172.21.15.3:0/888024785 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc9a001b780 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.266+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac0637c0 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.270+0000 7fc9a9600700 1 -- 172.21.15.3:0/888024785 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc9a001b980 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.350+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc9ac1a0860 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.351+0000 7fc9a9600700 1 -- 172.21.15.3:0/888024785 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fc9ac1a0860 con 0x7fc9ac106140 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.352+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 msgr2=0x7fc9ac19d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.352+0000 7fc9b3600700 1 --2- 172.21.15.3:0/888024785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac19d180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0000c00 tx=0x7fc9a0004750 comp rx=0 tx=0).stop 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.353+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 shutdown_connections 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.353+0000 7fc9b3600700 1 --2- 172.21.15.3:0/888024785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac106140 0x7fc9ac19d180 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.353+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 >> 172.21.15.3:0/888024785 conn(0x7fc9ac101de0 msgr2=0x7fc9ac1937c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:17:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.353+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 shutdown_connections 2024-03-21T08:17:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:51.353+0000 7fc9b3600700 1 -- 172.21.15.3:0/888024785 wait complete. 2024-03-21T08:17:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:mon is available 2024-03-21T08:17:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T08:17:52.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:52 smithi003 ceph-mon[31308]: from='client.? 172.21.15.3:0/4005221341' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T08:17:52.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:52 smithi003 ceph-mon[31308]: from='client.? 172.21.15.3:0/4005221341' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T08:17:53.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T08:17:53.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.581+0000 7f1c03800700 1 Processor -- start 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.581+0000 7f1c03800700 1 -- start start 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.582+0000 7f1c03800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.582+0000 7f1c03800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc107170 con 0x7f1bfc106780 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.582+0000 7f1c00e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.582+0000 7f1c00e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55102/0 (socket says 172.21.15.3:55102) 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.582+0000 7f1c00e00700 1 -- 172.21.15.3:0/3042817007 learned_addr learned my addr 172.21.15.3:0/3042817007 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:17:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.583+0000 7f1c00e00700 1 -- 172.21.15.3:0/3042817007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc107980 con 0x7f1bfc106780 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.583+0000 7f1c00e00700 1 --2- 172.21.15.3:0/3042817007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc106ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00098a0 tx=0x7f1bf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df45658a0d08af0c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.583+0000 7f1bfb400700 1 -- 172.21.15.3:0/3042817007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf0004160 con 0x7f1bfc106780 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1bfb400700 1 -- 172.21.15.3:0/3042817007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf0004300 con 0x7f1bfc106780 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1bfb400700 1 -- 172.21.15.3:0/3042817007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf00036a0 con 0x7f1bfc106780 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1c03800700 1 -- 172.21.15.3:0/3042817007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 msgr2=0x7f1bfc106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1c03800700 1 --2- 172.21.15.3:0/3042817007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc106ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00098a0 tx=0x7f1bf0009bb0 comp rx=0 tx=0).stop 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1c03800700 1 -- 172.21.15.3:0/3042817007 shutdown_connections 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1c03800700 1 --2- 172.21.15.3:0/3042817007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc106ba0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1c03800700 1 -- 172.21.15.3:0/3042817007 >> 172.21.15.3:0/3042817007 conn(0x7f1bfc101de0 msgr2=0x7f1bfc104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.584+0000 7f1c03800700 1 -- 172.21.15.3:0/3042817007 shutdown_connections 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.585+0000 7f1c03800700 1 -- 172.21.15.3:0/3042817007 wait complete. 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.586+0000 7f1c03800700 1 Processor -- start 2024-03-21T08:17:53.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.586+0000 7f1c03800700 1 -- start start 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.586+0000 7f1c03800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.586+0000 7f1c03800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc107df0 con 0x7f1bfc106780 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.587+0000 7f1c00e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.587+0000 7f1c00e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc19d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55118/0 (socket says 172.21.15.3:55118) 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.587+0000 7f1c00e00700 1 -- 172.21.15.3:0/4005221341 learned_addr learned my addr 172.21.15.3:0/4005221341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.587+0000 7f1c00e00700 1 -- 172.21.15.3:0/4005221341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc19d710 con 0x7f1bfc106780 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.588+0000 7f1c00e00700 1 --2- 172.21.15.3:0/4005221341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc19d1d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0000c00 tx=0x7f1bf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.588+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf0003db0 con 0x7f1bfc106780 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.588+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf0011630 con 0x7f1bfc106780 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.588+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc19d9a0 con 0x7f1bfc106780 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.588+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf0011950 con 0x7f1bfc106780 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.588+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc1a0690 con 0x7f1bfc106780 2024-03-21T08:17:53.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.588+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1bf001a460 con 0x7f1bfc106780 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.589+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1bf001b780 con 0x7f1bfc106780 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.589+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bfc0637c0 con 0x7f1bfc106780 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.592+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1bf001b980 con 0x7f1bfc106780 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.670+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1bfc19de80 con 0x7f1bfc106780 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.678+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f1bfc19de80 con 0x7f1bfc106780 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.678+0000 7f1bf9600700 1 -- 172.21.15.3:0/4005221341 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1bf001a8e0 con 0x7f1bfc106780 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.679+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 msgr2=0x7f1bfc19d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.679+0000 7f1c03800700 1 --2- 172.21.15.3:0/4005221341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc19d1d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0000c00 tx=0x7f1bf0004750 comp rx=0 tx=0).stop 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.680+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 shutdown_connections 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.680+0000 7f1c03800700 1 --2- 172.21.15.3:0/4005221341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1bfc106780 0x7f1bfc19d1d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.680+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 >> 172.21.15.3:0/4005221341 conn(0x7f1bfc101de0 msgr2=0x7f1bfc193e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.680+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 shutdown_connections 2024-03-21T08:17:53.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:52.680+0000 7f1c03800700 1 -- 172.21.15.3:0/4005221341 wait complete. 2024-03-21T08:17:53.307 INFO:teuthology.orchestra.run.smithi003.stdout:Generating new minimal ceph.conf... 2024-03-21T08:17:54.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:54 smithi003 ceph-mon[31308]: from='client.? 172.21.15.3:0/1563545004' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.900+0000 7effd0200700 1 Processor -- start 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.900+0000 7effd0200700 1 -- start start 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.900+0000 7effd0200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc8109de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.900+0000 7effd0200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc810a3b0 con 0x7effc81099c0 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.900+0000 7effcd800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc8109de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.901+0000 7effcd800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc8109de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55124/0 (socket says 172.21.15.3:55124) 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.901+0000 7effcd800700 1 -- 172.21.15.3:0/3792343666 learned_addr learned my addr 172.21.15.3:0/3792343666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.901+0000 7effcd800700 1 -- 172.21.15.3:0/3792343666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc810ab20 con 0x7effc81099c0 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.901+0000 7effcd800700 1 --2- 172.21.15.3:0/3792343666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc8109de0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7effbc0098a0 tx=0x7effbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55d7bb12355d69 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:17:54.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.902+0000 7effc7e00700 1 -- 172.21.15.3:0/3792343666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effbc004160 con 0x7effc81099c0 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.902+0000 7effc7e00700 1 -- 172.21.15.3:0/3792343666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7effbc004300 con 0x7effc81099c0 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.902+0000 7effc7e00700 1 -- 172.21.15.3:0/3792343666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effbc011a80 con 0x7effc81099c0 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.902+0000 7effd0200700 1 -- 172.21.15.3:0/3792343666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 msgr2=0x7effc8109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.902+0000 7effd0200700 1 --2- 172.21.15.3:0/3792343666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc8109de0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7effbc0098a0 tx=0x7effbc009bb0 comp rx=0 tx=0).stop 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.902+0000 7effd0200700 1 -- 172.21.15.3:0/3792343666 shutdown_connections 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.903+0000 7effd0200700 1 --2- 172.21.15.3:0/3792343666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc8109de0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.903+0000 7effd0200700 1 -- 172.21.15.3:0/3792343666 >> 172.21.15.3:0/3792343666 conn(0x7effc807ce00 msgr2=0x7effc81074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.903+0000 7effd0200700 1 -- 172.21.15.3:0/3792343666 shutdown_connections 2024-03-21T08:17:54.604 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.903+0000 7effd0200700 1 -- 172.21.15.3:0/3792343666 wait complete. 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effd0200700 1 Processor -- start 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effd0200700 1 -- start start 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effd0200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effd0200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc810ded0 con 0x7effc81099c0 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effcd800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effcd800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55128/0 (socket says 172.21.15.3:55128) 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effcd800700 1 -- 172.21.15.3:0/1563545004 learned_addr learned my addr 172.21.15.3:0/1563545004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effcd800700 1 -- 172.21.15.3:0/1563545004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc819dcf0 con 0x7effc81099c0 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effcd800700 1 --2- 172.21.15.3:0/1563545004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc819d7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effc6000700 1 -- 172.21.15.3:0/1563545004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effbc0039f0 con 0x7effc81099c0 2024-03-21T08:17:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effc6000700 1 -- 172.21.15.3:0/1563545004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7effbc003b90 con 0x7effc81099c0 2024-03-21T08:17:54.606 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.905+0000 7effc6000700 1 -- 172.21.15.3:0/1563545004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effbc01a740 con 0x7effc81099c0 2024-03-21T08:17:54.606 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.906+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc819df80 con 0x7effc81099c0 2024-03-21T08:17:54.608 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.906+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc81a0c70 con 0x7effc81099c0 2024-03-21T08:17:54.608 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.906+0000 7effc6000700 1 -- 172.21.15.3:0/1563545004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7effbc02d3f0 con 0x7effc81099c0 2024-03-21T08:17:54.608 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.906+0000 7effc6000700 1 -- 172.21.15.3:0/1563545004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7effbc019a90 con 0x7effc81099c0 2024-03-21T08:17:54.608 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.907+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc80637c0 con 0x7effc81099c0 2024-03-21T08:17:54.608 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.911+0000 7effc6000700 1 -- 172.21.15.3:0/1563545004 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7effbc019c90 con 0x7effc81099c0 2024-03-21T08:17:54.608 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.987+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7effc81a1110 con 0x7effc81099c0 2024-03-21T08:17:54.608 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.988+0000 7effc6000700 1 -- 172.21.15.3:0/1563545004 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7effbc032030 con 0x7effc81099c0 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.990+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 msgr2=0x7effc819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.990+0000 7effd0200700 1 --2- 172.21.15.3:0/1563545004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc819d7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc011b00 comp rx=0 tx=0).stop 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.991+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 shutdown_connections 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.991+0000 7effd0200700 1 --2- 172.21.15.3:0/1563545004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc81099c0 0x7effc819d7b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.991+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 >> 172.21.15.3:0/1563545004 conn(0x7effc807ce00 msgr2=0x7effc8106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.991+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 shutdown_connections 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:53.991+0000 7effd0200700 1 -- 172.21.15.3:0/1563545004 wait complete. 2024-03-21T08:17:54.609 INFO:teuthology.orchestra.run.smithi003.stdout:Restarting the monitor... 2024-03-21T08:17:54.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:54 smithi003 systemd[1]: Stopping Ceph mon.a for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:17:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:55 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31304]: 2024-03-21T08:17:55.152+0000 7f8505000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:17:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:55 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31304]: 2024-03-21T08:17:55.152+0000 7f8505000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T08:17:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:55 smithi003 podman[31550]: 2024-03-21 08:17:55.234606169 +0000 UTC m=+0.227609555 container died 4062767ccc36a9a50bf4e465aa918e8db2e03ef4d021049aa3535fe4d835117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T08:17:55.854 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:55 smithi003 podman[31550]: 2024-03-21 08:17:55.634879053 +0000 UTC m=+0.627882487 container cleanup 4062767ccc36a9a50bf4e465aa918e8db2e03ef4d021049aa3535fe4d835117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:17:55.854 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:55 smithi003 bash[31550]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a 2024-03-21T08:17:56.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:55 smithi003 podman[31564]: 2024-03-21 08:17:55.852281304 +0000 UTC m=+0.596391904 container remove 4062767ccc36a9a50bf4e465aa918e8db2e03ef4d021049aa3535fe4d835117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T08:17:56.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:56 smithi003 systemd[1]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-21T08:17:56.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:56 smithi003 systemd[1]: Stopped Ceph mon.a for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:17:56.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:56 smithi003 systemd[1]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a.service: Consumed 2.058s CPU time. 2024-03-21T08:17:56.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:56 smithi003 systemd[1]: Starting Ceph mon.a for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:17:57.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:56 smithi003 podman[31656]: 2024-03-21 08:17:56.752997388 +0000 UTC m=+0.126062325 container create 0b9774172d25aec8d35683bbbba921245e6dac9379211ddf8365022e10c874f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-21T08:17:57.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:56 smithi003 podman[31656]: 2024-03-21 08:17:56.661758286 +0000 UTC m=+0.034823169 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:17:57.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:56 smithi003 podman[31656]: 2024-03-21 08:17:56.98977612 +0000 UTC m=+0.362841080 container init 0b9774172d25aec8d35683bbbba921245e6dac9379211ddf8365022e10c874f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-21T08:17:57.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 podman[31656]: 2024-03-21 08:17:57.001107861 +0000 UTC m=+0.374172751 container start 0b9774172d25aec8d35683bbbba921245e6dac9379211ddf8365022e10c874f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-21T08:17:57.090 INFO:teuthology.orchestra.run.smithi003.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T08:17:57.374 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: pidfile_write: ignore empty --pid-file 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: load: jerasure load: lrc 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: RocksDB version: 7.9.2 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Git sha 0 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: DB SUMMARY 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: DB Session ID: ISAFMRPCELI0JXXE19RW 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: CURRENT file: CURRENT 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T08:17:57.375 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98928 ; 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.error_if_exists: 0 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.create_if_missing: 0 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.paranoid_checks: 1 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.env: 0x561cb5751b20 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.info_log: 0x561cb7f33960 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.statistics: (nil) 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.use_fsync: 0 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_log_file_size: 0 2024-03-21T08:17:57.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.allow_fallocate: 1 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.use_direct_reads: 0 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.db_log_dir: 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.wal_dir: 2024-03-21T08:17:57.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T08:17:57.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T08:17:57.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T08:17:57.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T08:17:57.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T08:17:57.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T08:17:57.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T08:17:57.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.write_buffer_manager: 0x561cb7f061e0 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.unordered_write: 0 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T08:17:57.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.row_cache: None 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.wal_filter: None 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.two_write_queues: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.wal_compression: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.atomic_flush: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T08:17:57.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.log_readahead_size: 0 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_background_jobs: 2 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_background_compactions: -1 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_subcompactions: 1 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T08:17:57.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_open_files: -1 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_background_flushes: -1 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Compression algorithms supported: 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kZSTD supported: 0 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kXpressCompression supported: 0 2024-03-21T08:17:57.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kZlibCompression supported: 1 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kSnappyCompression supported: 1 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kLZ4Compression supported: 1 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: kBZip2Compression supported: 0 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.merge_operator: 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_filter: None 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T08:17:57.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561cb7f33a60) 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks: 1 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_top_level_index_and_filter: 1 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_type: 0 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_index_type: 0 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_shortening: 1 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: checksum: 4 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: no_block_cache: 0 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache: 0x561cb7f39090 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_name: BinnedLRUCache 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_options: 2024-03-21T08:17:57.384 INFO:journalctl@ceph.mon.a.smithi003.stdout: capacity : 536870912 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_shard_bits : 4 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: strict_capacity_limit : 0 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: high_pri_pool_ratio: 0.000 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_compressed: (nil) 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: persistent_cache: (nil) 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size: 4096 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size_deviation: 10 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_restart_interval: 16 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_block_restart_interval: 1 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: metadata_block_size: 4096 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: partition_filters: 0 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: use_delta_encoding: 1 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: filter_policy: bloomfilter 2024-03-21T08:17:57.385 INFO:journalctl@ceph.mon.a.smithi003.stdout: whole_key_filtering: 1 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: verify_compression: 0 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: read_amp_bytes_per_bit: 0 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: format_version: 5 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_index_compression: 1 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_align: 0 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: max_auto_readahead_size: 262144 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: prepopulate_block_cache: 0 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: initial_auto_readahead_size: 8192 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression: NoCompression 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T08:17:57.386 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.num_levels: 7 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T08:17:57.387 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:17:57.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T08:17:57.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T08:17:57.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:17:57.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:17:57.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T08:17:57.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T08:17:57.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T08:17:57.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.table_properties_collectors: 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.inplace_update_support: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.bloom_locality: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.max_successive_merges: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.ttl: 2592000 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T08:17:57.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.enable_blob_files: false 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.min_blob_size: 0 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T08:17:57.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7fb4a10-6513-4554-ad5a-a055260ad87b 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009077059605, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009077061516, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 92230, "index_size": 254, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13505, "raw_average_key_size": 49, "raw_value_size": 84866, "raw_average_value_size": 313, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711009077, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7fb4a10-6513-4554-ad5a-a055260ad87b", "db_session_id": "ISAFMRPCELI0JXXE19RW", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009077061768, "job": 1, "event": "recovery_finished"} 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561cb802c000 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: DB pointer 0x561cb8016000 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** DB Stats ** 2024-03-21T08:17:57.394 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: L0 2/0 93.83 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Sum 2/0 93.83 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:17:57.395 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 69.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative compaction: 0.00 GB write, 13.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:17:57.396 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval compaction: 0.00 GB write, 13.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache BinnedLRUCache@0x561cb7f39090#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 3.4e-05 secs_since: 0 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: starting mon.a rank 0 at public addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] at bind addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???) e1 preinit fsid 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???).mds e1 new map 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???).mds e1 print_map 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: e1 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: legacy client fscid: -1 2024-03-21T08:17:57.397 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout: No filesystems configured 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 bash[31656]: 0b9774172d25aec8d35683bbbba921245e6dac9379211ddf8365022e10c874f2 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 systemd[1]: Started Ceph mon.a for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: monmap e1: 1 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: fsmap 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T08:17:57.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:57 smithi003 ceph-mon[31675]: mgrmap e1: no daemons active 2024-03-21T08:17:59.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:17:58 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2921325057' entity='client.admin' 2024-03-21T08:18:00.383 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.691+0000 7f15eee00700 1 Processor -- start 2024-03-21T08:18:00.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.691+0000 7f15eee00700 1 -- start start 2024-03-21T08:18:00.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.692+0000 7f15eee00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e807b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:00.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.692+0000 7f15eee00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e807b870 con 0x7f15e807ce40 2024-03-21T08:18:00.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.693+0000 7f15e7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e807b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:00.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.693+0000 7f15e7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e807b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55132/0 (socket says 172.21.15.3:55132) 2024-03-21T08:18:00.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.693+0000 7f15e7e00700 1 -- 172.21.15.3:0/2844996084 learned_addr learned my addr 172.21.15.3:0/2844996084 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.693+0000 7f15e7e00700 1 -- 172.21.15.3:0/2844996084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e807b9f0 con 0x7f15e807ce40 2024-03-21T08:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.694+0000 7f15e7e00700 1 --2- 172.21.15.3:0/2844996084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e807b2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15d80098a0 tx=0x7f15d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143f1917a4149db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.694+0000 7f15e7400700 1 -- 172.21.15.3:0/2844996084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15d8004160 con 0x7f15e807ce40 2024-03-21T08:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.694+0000 7f15e7400700 1 -- 172.21.15.3:0/2844996084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f15d8004300 con 0x7f15e807ce40 2024-03-21T08:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.694+0000 7f15e7400700 1 -- 172.21.15.3:0/2844996084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15d8011a80 con 0x7f15e807ce40 2024-03-21T08:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.695+0000 7f15eee00700 1 -- 172.21.15.3:0/2844996084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 msgr2=0x7f15e807b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.695+0000 7f15eee00700 1 --2- 172.21.15.3:0/2844996084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e807b2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15d80098a0 tx=0x7f15d8009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.695+0000 7f15eee00700 1 -- 172.21.15.3:0/2844996084 shutdown_connections 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.695+0000 7f15eee00700 1 --2- 172.21.15.3:0/2844996084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e807b2a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.695+0000 7f15eee00700 1 -- 172.21.15.3:0/2844996084 >> 172.21.15.3:0/2844996084 conn(0x7f15e8102f00 msgr2=0x7f15e8105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.695+0000 7f15eee00700 1 -- 172.21.15.3:0/2844996084 shutdown_connections 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.695+0000 7f15eee00700 1 -- 172.21.15.3:0/2844996084 wait complete. 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.696+0000 7f15eee00700 1 Processor -- start 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.697+0000 7f15eee00700 1 -- start start 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.697+0000 7f15eee00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.697+0000 7f15eee00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e810de90 con 0x7f15e807ce40 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.697+0000 7f15e7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.697+0000 7f15e7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55134/0 (socket says 172.21.15.3:55134) 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.697+0000 7f15e7e00700 1 -- 172.21.15.3:0/2921325057 learned_addr learned my addr 172.21.15.3:0/2921325057 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.698+0000 7f15e7e00700 1 -- 172.21.15.3:0/2921325057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e819dec0 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.698+0000 7f15e7e00700 1 --2- 172.21.15.3:0/2921325057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e819d980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f15d8011a50 tx=0x7f15d8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.698+0000 7f15e5600700 1 -- 172.21.15.3:0/2921325057 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15d80039f0 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.698+0000 7f15e5600700 1 -- 172.21.15.3:0/2921325057 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f15d8003b90 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.698+0000 7f15e5600700 1 -- 172.21.15.3:0/2921325057 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15d801a600 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.698+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e819e150 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.698+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e81a0e40 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.699+0000 7f15e5600700 1 -- 172.21.15.3:0/2921325057 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f15d801ac90 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.699+0000 7f15e5600700 1 -- 172.21.15.3:0/2921325057 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f15d80193c0 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.700+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e80637c0 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.703+0000 7f15e5600700 1 -- 172.21.15.3:0/2921325057 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f15d8024780 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.780+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f15e81a1010 con 0x7f15e807ce40 2024-03-21T08:18:00.390 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.785+0000 7f15e5600700 1 -- 172.21.15.3:0/2921325057 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f15d8024a40 con 0x7f15e807ce40 2024-03-21T08:18:00.391 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.786+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 msgr2=0x7f15e819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:00.391 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.786+0000 7f15eee00700 1 --2- 172.21.15.3:0/2921325057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e819d980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f15d8011a50 tx=0x7f15d8004290 comp rx=0 tx=0).stop 2024-03-21T08:18:00.391 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.786+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 shutdown_connections 2024-03-21T08:18:00.391 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.786+0000 7f15eee00700 1 --2- 172.21.15.3:0/2921325057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15e807ce40 0x7f15e819d980 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:00.391 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.786+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 >> 172.21.15.3:0/2921325057 conn(0x7f15e8102f00 msgr2=0x7f15e8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:00.391 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.786+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 shutdown_connections 2024-03-21T08:18:00.391 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:17:57.786+0000 7f15eee00700 1 -- 172.21.15.3:0/2921325057 wait complete. 2024-03-21T08:18:00.392 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T08:18:01.279 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:18:01.279 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mgr... 2024-03-21T08:18:01.281 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T08:18:01.282 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T08:18:04.336 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.x 2024-03-21T08:18:04.336 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-21T08:18:05.150 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-544fc91e-e75b-11ee-95cd-87774f69a715.target.wants/ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-544fc91e-e75b-11ee-95cd-87774f69a715@.service. 2024-03-21T08:18:05.545 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:05 smithi003 systemd[1]: Starting Ceph mgr.x for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:18:05.928 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:05 smithi003 podman[31886]: 2024-03-21 08:18:05.45099104 +0000 UTC m=+0.039516103 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:18:06.296 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:05 smithi003 podman[31886]: 2024-03-21 08:18:05.984888075 +0000 UTC m=+0.573413161 container create 4a0ec4f047a25633a3bad7a7e56516352534808ca967d4e493f69eaddc22b992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T08:18:06.297 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:06 smithi003 podman[31886]: 2024-03-21 08:18:06.213438337 +0000 UTC m=+0.801963430 container init 4a0ec4f047a25633a3bad7a7e56516352534808ca967d4e493f69eaddc22b992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-21T08:18:06.297 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:06 smithi003 podman[31886]: 2024-03-21 08:18:06.224609491 +0000 UTC m=+0.813134569 container start 4a0ec4f047a25633a3bad7a7e56516352534808ca967d4e493f69eaddc22b992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T08:18:06.333 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:18:06.333 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-03-21T08:18:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:18:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-03-21T08:18:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-03-21T08:18:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T08:18:06.370 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:18:06.370 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-03-21T08:18:06.388 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:18:06.388 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-03-21T08:18:06.388 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-03-21T08:18:06.389 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-21T08:18:06.389 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr to start... 2024-03-21T08:18:06.389 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr... 2024-03-21T08:18:06.647 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:06 smithi003 bash[31886]: 4a0ec4f047a25633a3bad7a7e56516352534808ca967d4e493f69eaddc22b992 2024-03-21T08:18:06.647 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:06 smithi003 systemd[1]: Started Ceph mgr.x for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:18:07.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:07 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/650013182' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:18:07.423 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:07.046+0000 7fe62a5b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:18:07.423 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:07.180+0000 7fe62a5b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:18:07.763 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:18:07.765 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:18:07.766 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:18:07.767 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:18:07.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:18:07.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:18:07.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:17:50.597390+0000", 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.942+0000 7f680f800700 1 Processor -- start 2024-03-21T08:18:07.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.943+0000 7f680f800700 1 -- start start 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.943+0000 7f680f800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680810a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.943+0000 7f680f800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680807ae50 con 0x7f6808108330 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.944+0000 7f680ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680810a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.944+0000 7f680ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680810a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37742/0 (socket says 172.21.15.3:37742) 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.944+0000 7f680ce00700 1 -- 172.21.15.3:0/1752708095 learned_addr learned my addr 172.21.15.3:0/1752708095 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.944+0000 7f680ce00700 1 -- 172.21.15.3:0/1752708095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680810ac60 con 0x7f6808108330 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.945+0000 7f680ce00700 1 --2- 172.21.15.3:0/1752708095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680810a720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0098a0 tx=0x7f67fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ce8c486780723d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.945+0000 7f6807400700 1 -- 172.21.15.3:0/1752708095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67fc004160 con 0x7f6808108330 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.945+0000 7f6807400700 1 -- 172.21.15.3:0/1752708095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f67fc004300 con 0x7f6808108330 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.945+0000 7f6807400700 1 -- 172.21.15.3:0/1752708095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67fc011a80 con 0x7f6808108330 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.946+0000 7f680f800700 1 -- 172.21.15.3:0/1752708095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 msgr2=0x7f680810a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.946+0000 7f680f800700 1 --2- 172.21.15.3:0/1752708095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680810a720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0098a0 tx=0x7f67fc009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:07.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.946+0000 7f680f800700 1 -- 172.21.15.3:0/1752708095 shutdown_connections 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.946+0000 7f680f800700 1 --2- 172.21.15.3:0/1752708095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680810a720 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.946+0000 7f680f800700 1 -- 172.21.15.3:0/1752708095 >> 172.21.15.3:0/1752708095 conn(0x7f6808101de0 msgr2=0x7f6808104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.946+0000 7f680f800700 1 -- 172.21.15.3:0/1752708095 shutdown_connections 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.946+0000 7f680f800700 1 -- 172.21.15.3:0/1752708095 wait complete. 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.947+0000 7f680f800700 1 Processor -- start 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.947+0000 7f680f800700 1 -- start start 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.948+0000 7f680f800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.948+0000 7f680f800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680810bc20 con 0x7f6808108330 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.948+0000 7f680ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.948+0000 7f680ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680819d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37758/0 (socket says 172.21.15.3:37758) 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.948+0000 7f680ce00700 1 -- 172.21.15.3:0/650013182 learned_addr learned my addr 172.21.15.3:0/650013182 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.948+0000 7f680ce00700 1 -- 172.21.15.3:0/650013182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680819df00 con 0x7f6808108330 2024-03-21T08:18:07.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.949+0000 7f680ce00700 1 --2- 172.21.15.3:0/650013182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680819d9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67fc011a50 tx=0x7f67fc004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.949+0000 7f6805600700 1 -- 172.21.15.3:0/650013182 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67fc0039f0 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.949+0000 7f6805600700 1 -- 172.21.15.3:0/650013182 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f67fc003b90 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.949+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680819e190 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.949+0000 7f6805600700 1 -- 172.21.15.3:0/650013182 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67fc01a600 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.949+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68081a0e80 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.950+0000 7f6805600700 1 -- 172.21.15.3:0/650013182 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f67fc01ac80 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.950+0000 7f6805600700 1 -- 172.21.15.3:0/650013182 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f67fc019700 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.950+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68080637c0 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:06.953+0000 7f6805600700 1 -- 172.21.15.3:0/650013182 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f67fc024a40 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.033+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f68081a1430 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.033+0000 7f6805600700 1 -- 172.21.15.3:0/650013182 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f67fc024440 con 0x7f6808108330 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.035+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 msgr2=0x7f680819d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:07.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.035+0000 7f680f800700 1 --2- 172.21.15.3:0/650013182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680819d9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67fc011a50 tx=0x7f67fc004290 comp rx=0 tx=0).stop 2024-03-21T08:18:07.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.035+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 shutdown_connections 2024-03-21T08:18:07.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.035+0000 7f680f800700 1 --2- 172.21.15.3:0/650013182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808108330 0x7f680819d9c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:07.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.035+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 >> 172.21.15.3:0/650013182 conn(0x7f6808101de0 msgr2=0x7f68081028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:07.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.035+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 shutdown_connections 2024-03-21T08:18:07.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:07.035+0000 7f680f800700 1 -- 172.21.15.3:0/650013182 wait complete. 2024-03-21T08:18:07.782 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (1/15)... 2024-03-21T08:18:08.360 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:08.040+0000 7fe62a5b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:18:08.360 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:08.169+0000 7fe62a5b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:18:08.361 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:08.358+0000 7fe62a5b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:18:08.659 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:08.487+0000 7fe62a5b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:18:08.659 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:08.602+0000 7fe62a5b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:18:09.208 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:08.946+0000 7fe62a5b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:18:09.208 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:09.091+0000 7fe62a5b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:18:09.558 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:09.205+0000 7fe62a5b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:18:09.558 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:09.333+0000 7fe62a5b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:18:09.558 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:09.555+0000 7fe62a5b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:18:10.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:10 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3762986912' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:18:11.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:10.962+0000 7fe62a5b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:18:11.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:11.141+0000 7fe62a5b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:18:11.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:11.572+0000 7fe62a5b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:18:11.742 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:11.743 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:18:11.744 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:18:11.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:18:11.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:11.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:17:50.597390+0000", 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.326+0000 7f353de00700 1 Processor -- start 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.326+0000 7f353de00700 1 -- start start 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.326+0000 7f353de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353810a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.326+0000 7f353de00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353807ae70 con 0x7f3538108310 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.327+0000 7f3536a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353810a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.327+0000 7f3536a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353810a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35278/0 (socket says 172.21.15.3:35278) 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.327+0000 7f3536a00700 1 -- 172.21.15.3:0/3855268844 learned_addr learned my addr 172.21.15.3:0/3855268844 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:11.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.327+0000 7f3536a00700 1 -- 172.21.15.3:0/3855268844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353810ac40 con 0x7f3538108310 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.328+0000 7f3536a00700 1 --2- 172.21.15.3:0/3855268844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353810a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35280098a0 tx=0x7f3528009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50bc7443c6322bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.328+0000 7f3535600700 1 -- 172.21.15.3:0/3855268844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3528004160 con 0x7f3538108310 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.328+0000 7f3535600700 1 -- 172.21.15.3:0/3855268844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3528004300 con 0x7f3538108310 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.328+0000 7f3535600700 1 -- 172.21.15.3:0/3855268844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3528011a80 con 0x7f3538108310 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.329+0000 7f353de00700 1 -- 172.21.15.3:0/3855268844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 msgr2=0x7f353810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.329+0000 7f353de00700 1 --2- 172.21.15.3:0/3855268844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353810a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35280098a0 tx=0x7f3528009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.329+0000 7f353de00700 1 -- 172.21.15.3:0/3855268844 shutdown_connections 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.329+0000 7f353de00700 1 --2- 172.21.15.3:0/3855268844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353810a700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.329+0000 7f353de00700 1 -- 172.21.15.3:0/3855268844 >> 172.21.15.3:0/3855268844 conn(0x7f3538101de0 msgr2=0x7f3538104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:11.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.329+0000 7f353de00700 1 -- 172.21.15.3:0/3855268844 shutdown_connections 2024-03-21T08:18:11.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.329+0000 7f353de00700 1 -- 172.21.15.3:0/3855268844 wait complete. 2024-03-21T08:18:11.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.330+0000 7f353de00700 1 Processor -- start 2024-03-21T08:18:11.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.330+0000 7f353de00700 1 -- start start 2024-03-21T08:18:11.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.331+0000 7f353de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:11.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.331+0000 7f353de00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353810bb90 con 0x7f3538108310 2024-03-21T08:18:11.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.331+0000 7f3536a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:11.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.331+0000 7f3536a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35282/0 (socket says 172.21.15.3:35282) 2024-03-21T08:18:11.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.331+0000 7f3536a00700 1 -- 172.21.15.3:0/3762986912 learned_addr learned my addr 172.21.15.3:0/3762986912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:11.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.332+0000 7f3536a00700 1 -- 172.21.15.3:0/3762986912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353819bcb0 con 0x7f3538108310 2024-03-21T08:18:11.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.332+0000 7f3536a00700 1 --2- 172.21.15.3:0/3762986912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353819b770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3528006af0 tx=0x7f3528011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:11.759 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.332+0000 7f353ca00700 1 -- 172.21.15.3:0/3762986912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3528003930 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.332+0000 7f353ca00700 1 -- 172.21.15.3:0/3762986912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3528003ad0 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.332+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353819bfa0 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.332+0000 7f353ca00700 1 -- 172.21.15.3:0/3762986912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3528019590 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.332+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353819ec30 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.333+0000 7f353ca00700 1 -- 172.21.15.3:0/3762986912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3528019c40 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.333+0000 7f353ca00700 1 -- 172.21.15.3:0/3762986912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3528024730 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.333+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35380637c0 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.336+0000 7f353ca00700 1 -- 172.21.15.3:0/3762986912 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3528024930 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.417+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f353819f2a0 con 0x7f3538108310 2024-03-21T08:18:11.760 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.418+0000 7f353ca00700 1 -- 172.21.15.3:0/3762986912 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f352802d440 con 0x7f3538108310 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.419+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 msgr2=0x7f353819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.419+0000 7f353de00700 1 --2- 172.21.15.3:0/3762986912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353819b770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3528006af0 tx=0x7f3528011b70 comp rx=0 tx=0).stop 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.419+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 shutdown_connections 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.419+0000 7f353de00700 1 --2- 172.21.15.3:0/3762986912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3538108310 0x7f353819b770 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.419+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 >> 172.21.15.3:0/3762986912 conn(0x7f3538101de0 msgr2=0x7f35381028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.419+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 shutdown_connections 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:10.420+0000 7f353de00700 1 -- 172.21.15.3:0/3762986912 wait complete. 2024-03-21T08:18:11.761 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (2/15)... 2024-03-21T08:18:11.963 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:11.694+0000 7fe62a5b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:18:11.963 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:11.960+0000 7fe62a5b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:18:12.601 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:12.213+0000 7fe62a5b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:18:12.601 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:12.598+0000 7fe62a5b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:18:13.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:12.921+0000 7fe62a5b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:18:13.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:13 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:13.234+0000 7fe62a5b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:18:14.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:14 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1295865191' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:18:14.678 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:14 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:14.317+0000 7fe62a5b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:18:15.084 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:15.043+0000 7fe62a5b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:18:15.095 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:18:15.095 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:18:15.095 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:18:15.095 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:18:15.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:15.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:18:15.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:18:15.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:15.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:18:15.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:15.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:18:15.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:18:15.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:18:15.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:18:15.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:18:15.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:18:15.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:18:15.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:18:15.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:18:15.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:18:15.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:18:15.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:18:15.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:15.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:15.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:15.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:18:15.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:15.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:17:50.597390+0000", 2024-03-21T08:18:15.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:15.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:15.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:18:15.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:18:15.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.294+0000 7f7c0a200700 1 Processor -- start 2024-03-21T08:18:15.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.294+0000 7f7c0a200700 1 -- start start 2024-03-21T08:18:15.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.295+0000 7f7c0a200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0407b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:15.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.295+0000 7f7c0a200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0407b7e0 con 0x7f7c0407cdf0 2024-03-21T08:18:15.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.295+0000 7f7c03400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0407b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.295+0000 7f7c03400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0407b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35288/0 (socket says 172.21.15.3:35288) 2024-03-21T08:18:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.295+0000 7f7c03400700 1 -- 172.21.15.3:0/3172496124 learned_addr learned my addr 172.21.15.3:0/3172496124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.296+0000 7f7c03400700 1 -- 172.21.15.3:0/3172496124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0407b960 con 0x7f7c0407cdf0 2024-03-21T08:18:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.296+0000 7f7c03400700 1 --2- 172.21.15.3:0/3172496124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0407b2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009e80 tx=0x7f7bf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f40bcbf694d5209 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.296+0000 7f7c02a00700 1 -- 172.21.15.3:0/3172496124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4017070 con 0x7f7c0407cdf0 2024-03-21T08:18:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.296+0000 7f7c02a00700 1 -- 172.21.15.3:0/3172496124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f7bf400b8c0 con 0x7f7c0407cdf0 2024-03-21T08:18:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.296+0000 7f7c02a00700 1 -- 172.21.15.3:0/3172496124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4004520 con 0x7f7c0407cdf0 2024-03-21T08:18:15.114 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.297+0000 7f7c0a200700 1 -- 172.21.15.3:0/3172496124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 msgr2=0x7f7c0407b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:15.114 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.297+0000 7f7c0a200700 1 --2- 172.21.15.3:0/3172496124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0407b2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009e80 tx=0x7f7bf400b330 comp rx=0 tx=0).stop 2024-03-21T08:18:15.114 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.297+0000 7f7c0a200700 1 -- 172.21.15.3:0/3172496124 shutdown_connections 2024-03-21T08:18:15.114 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.297+0000 7f7c0a200700 1 --2- 172.21.15.3:0/3172496124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0407b2a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:15.114 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.297+0000 7f7c0a200700 1 -- 172.21.15.3:0/3172496124 >> 172.21.15.3:0/3172496124 conn(0x7f7c04102f50 msgr2=0x7f7c04105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:15.115 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.297+0000 7f7c0a200700 1 -- 172.21.15.3:0/3172496124 shutdown_connections 2024-03-21T08:18:15.115 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.297+0000 7f7c0a200700 1 -- 172.21.15.3:0/3172496124 wait complete. 2024-03-21T08:18:15.115 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.298+0000 7f7c0a200700 1 Processor -- start 2024-03-21T08:18:15.115 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.298+0000 7f7c0a200700 1 -- start start 2024-03-21T08:18:15.115 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.299+0000 7f7c0a200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:15.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.299+0000 7f7c0a200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0410de40 con 0x7f7c0407cdf0 2024-03-21T08:18:15.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.299+0000 7f7c03400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:15.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.299+0000 7f7c03400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0419d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35294/0 (socket says 172.21.15.3:35294) 2024-03-21T08:18:15.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.299+0000 7f7c03400700 1 -- 172.21.15.3:0/1295865191 learned_addr learned my addr 172.21.15.3:0/1295865191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:15.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.299+0000 7f7c03400700 1 -- 172.21.15.3:0/1295865191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0419df00 con 0x7f7c0407cdf0 2024-03-21T08:18:15.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.299+0000 7f7c03400700 1 --2- 172.21.15.3:0/1295865191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0419d9c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40042b0 tx=0x7f7bf4004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:15.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.300+0000 7f7c00c00700 1 -- 172.21.15.3:0/1295865191 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4020020 con 0x7f7c0407cdf0 2024-03-21T08:18:15.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.300+0000 7f7c00c00700 1 -- 172.21.15.3:0/1295865191 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f7bf400bba0 con 0x7f7c0407cdf0 2024-03-21T08:18:15.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.300+0000 7f7c00c00700 1 -- 172.21.15.3:0/1295865191 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4017040 con 0x7f7c0407cdf0 2024-03-21T08:18:15.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.300+0000 7f7c0a200700 1 -- 172.21.15.3:0/1295865191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0419e190 con 0x7f7c0407cdf0 2024-03-21T08:18:15.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.300+0000 7f7c0a200700 1 -- 172.21.15.3:0/1295865191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c041a0e80 con 0x7f7c0407cdf0 2024-03-21T08:18:15.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.300+0000 7f7c00c00700 1 -- 172.21.15.3:0/1295865191 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7bf4023020 con 0x7f7c0407cdf0 2024-03-21T08:18:15.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.300+0000 7f7c00c00700 1 -- 172.21.15.3:0/1295865191 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7bf400f050 con 0x7f7c0407cdf0 2024-03-21T08:18:15.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.301+0000 7f7bf9600700 1 -- 172.21.15.3:0/1295865191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc005330 con 0x7f7c0407cdf0 2024-03-21T08:18:15.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.304+0000 7f7c00c00700 1 -- 172.21.15.3:0/1295865191 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7bf402a020 con 0x7f7c0407cdf0 2024-03-21T08:18:15.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.384+0000 7f7bf9600700 1 -- 172.21.15.3:0/1295865191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7bdc0058f0 con 0x7f7c0407cdf0 2024-03-21T08:18:15.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.385+0000 7f7c00c00700 1 -- 172.21.15.3:0/1295865191 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f7bf400b860 con 0x7f7c0407cdf0 2024-03-21T08:18:15.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.386+0000 7f7bf9600700 1 -- 172.21.15.3:0/1295865191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 msgr2=0x7f7c0419d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:15.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.386+0000 7f7bf9600700 1 --2- 172.21.15.3:0/1295865191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0419d9c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40042b0 tx=0x7f7bf4004670 comp rx=0 tx=0).stop 2024-03-21T08:18:15.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.386+0000 7f7bf9600700 1 -- 172.21.15.3:0/1295865191 shutdown_connections 2024-03-21T08:18:15.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.386+0000 7f7bf9600700 1 --2- 172.21.15.3:0/1295865191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c0407cdf0 0x7f7c0419d9c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:15.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.386+0000 7f7bf9600700 1 -- 172.21.15.3:0/1295865191 >> 172.21.15.3:0/1295865191 conn(0x7f7c04102f50 msgr2=0x7f7c04103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:15.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.386+0000 7f7bf9600700 1 -- 172.21.15.3:0/1295865191 shutdown_connections 2024-03-21T08:18:15.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:14.386+0000 7f7bf9600700 1 -- 172.21.15.3:0/1295865191 wait complete. 2024-03-21T08:18:15.120 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (3/15)... 2024-03-21T08:18:15.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:15.175+0000 7fe62a5b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:18:15.927 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:15.465+0000 7fe62a5b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:18:15.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: Activating manager daemon x 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: mgrmap e2: x(active, starting, since 0.00620185s) 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: Manager daemon x is now available 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' 2024-03-21T08:18:15.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:15 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' 2024-03-21T08:18:16.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:16 smithi003 ceph-mon[31675]: from='mgr.14100 172.21.15.3:0/185201058' entity='mgr.x' 2024-03-21T08:18:16.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:16 smithi003 ceph-mon[31675]: mgrmap e3: x(active, since 1.01975s) 2024-03-21T08:18:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:18 smithi003 ceph-mon[31675]: mgrmap e4: x(active, since 2s) 2024-03-21T08:18:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/294271455' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:18:18.737 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 21, 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:18.738 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:18:18.739 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:18:18.739 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:18:18.739 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:18:18.739 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:18:18.740 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:18:18.741 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:18:18.741 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:18:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:18:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:17:50.597390+0000", 2024-03-21T08:18:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:18:18.753 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.752+0000 7fb083a00700 1 Processor -- start 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.752+0000 7fb083a00700 1 -- start start 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.753+0000 7fb083a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.753+0000 7fb083a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c10a570 con 0x7fb07c109b80 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.753+0000 7fb081000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.753+0000 7fb081000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c109fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35364/0 (socket says 172.21.15.3:35364) 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.753+0000 7fb081000700 1 -- 172.21.15.3:0/4171766510 learned_addr learned my addr 172.21.15.3:0/4171766510 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.754+0000 7fb081000700 1 -- 172.21.15.3:0/4171766510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c10ace0 con 0x7fb07c109b80 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.754+0000 7fb081000700 1 --2- 172.21.15.3:0/4171766510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c109fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb0700098a0 tx=0x7fb070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2665aa13220f87 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:18.754 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.754+0000 7fb07b400700 1 -- 172.21.15.3:0/4171766510 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb070004160 con 0x7fb07c109b80 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.754+0000 7fb07b400700 1 -- 172.21.15.3:0/4171766510 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fb070004300 con 0x7fb07c109b80 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.754+0000 7fb07b400700 1 -- 172.21.15.3:0/4171766510 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb070011a80 con 0x7fb07c109b80 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.755+0000 7fb083a00700 1 -- 172.21.15.3:0/4171766510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 msgr2=0x7fb07c109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.755+0000 7fb083a00700 1 --2- 172.21.15.3:0/4171766510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c109fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb0700098a0 tx=0x7fb070009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.755+0000 7fb083a00700 1 -- 172.21.15.3:0/4171766510 shutdown_connections 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.755+0000 7fb083a00700 1 --2- 172.21.15.3:0/4171766510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c109fa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.755+0000 7fb083a00700 1 -- 172.21.15.3:0/4171766510 >> 172.21.15.3:0/4171766510 conn(0x7fb07c1054a0 msgr2=0x7fb07c107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.755+0000 7fb083a00700 1 -- 172.21.15.3:0/4171766510 shutdown_connections 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.755+0000 7fb083a00700 1 -- 172.21.15.3:0/4171766510 wait complete. 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.756+0000 7fb083a00700 1 Processor -- start 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.756+0000 7fb083a00700 1 -- start start 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.757+0000 7fb083a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:18.755 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.757+0000 7fb083a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c10e090 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.757+0000 7fb081000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.757+0000 7fb081000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35374/0 (socket says 172.21.15.3:35374) 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.757+0000 7fb081000700 1 -- 172.21.15.3:0/294271455 learned_addr learned my addr 172.21.15.3:0/294271455 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.757+0000 7fb081000700 1 -- 172.21.15.3:0/294271455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c19df40 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.758+0000 7fb081000700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c19da00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb070000c00 tx=0x7fb070004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.758+0000 7fb079600700 1 -- 172.21.15.3:0/294271455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0700039c0 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.758+0000 7fb079600700 1 -- 172.21.15.3:0/294271455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fb070003b60 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.758+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c19e1d0 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.758+0000 7fb079600700 1 -- 172.21.15.3:0/294271455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb07001a6f0 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.758+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c1a0ec0 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.759+0000 7fb079600700 1 -- 172.21.15.3:0/294271455 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7fb070024a40 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.759+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c0637c0 con 0x7fb07c109b80 2024-03-21T08:18:18.756 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.760+0000 7fb079600700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7fb064039240 0x7fb06403b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.760+0000 7fb079600700 1 -- 172.21.15.3:0/294271455 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb07004d410 con 0x7fb07c109b80 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.760+0000 7fb07be00700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7fb064039240 0x7fb06403b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.761+0000 7fb07be00700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7fb064039240 0x7fb06403b700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb068006fd0 tx=0x7fb068006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:17.767+0000 7fb079600700 1 -- 172.21.15.3:0/294271455 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb07000e260 con 0x7fb07c109b80 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.089+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb07c1a1460 con 0x7fb07c109b80 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.090+0000 7fb079600700 1 -- 172.21.15.3:0/294271455 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb070004300 con 0x7fb07c109b80 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7fb064039240 msgr2=0x7fb06403b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7fb064039240 0x7fb06403b700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb068006fd0 tx=0x7fb068006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 msgr2=0x7fb07c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c19da00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb070000c00 tx=0x7fb070004290 comp rx=0 tx=0).stop 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 shutdown_connections 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7fb064039240 0x7fb06403b700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 --2- 172.21.15.3:0/294271455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb07c109b80 0x7fb07c19da00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:18.757 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 >> 172.21.15.3:0/294271455 conn(0x7fb07c1054a0 msgr2=0x7fb07c106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:18.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.093+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 shutdown_connections 2024-03-21T08:18:18.758 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:18.094+0000 7fb083a00700 1 -- 172.21.15.3:0/294271455 wait complete. 2024-03-21T08:18:18.758 INFO:teuthology.orchestra.run.smithi003.stdout:mgr is available 2024-03-21T08:18:19.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:19 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1019179312' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T08:18:20.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:18:20.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T08:18:20.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:18:20.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T08:18:20.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T08:18:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.358+0000 7ff349200700 1 Processor -- start 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.358+0000 7ff349200700 1 -- start start 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.359+0000 7ff349200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff344109d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.359+0000 7ff349200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34410a340 con 0x7ff344109950 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.359+0000 7ff343400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff344109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.360+0000 7ff343400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff344109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35376/0 (socket says 172.21.15.3:35376) 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.360+0000 7ff343400700 1 -- 172.21.15.3:0/772449527 learned_addr learned my addr 172.21.15.3:0/772449527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.360+0000 7ff343400700 1 -- 172.21.15.3:0/772449527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34410ab00 con 0x7ff344109950 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.361+0000 7ff343400700 1 --2- 172.21.15.3:0/772449527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff344109d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098a0 tx=0x7ff334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4504a5e5546f307 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.361+0000 7ff342000700 1 -- 172.21.15.3:0/772449527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff334004160 con 0x7ff344109950 2024-03-21T08:18:20.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.361+0000 7ff342000700 1 -- 172.21.15.3:0/772449527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff334004300 con 0x7ff344109950 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.361+0000 7ff342000700 1 -- 172.21.15.3:0/772449527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff334011a80 con 0x7ff344109950 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.362+0000 7ff349200700 1 -- 172.21.15.3:0/772449527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 msgr2=0x7ff344109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.362+0000 7ff349200700 1 --2- 172.21.15.3:0/772449527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff344109d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098a0 tx=0x7ff334009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.362+0000 7ff349200700 1 -- 172.21.15.3:0/772449527 shutdown_connections 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.362+0000 7ff349200700 1 --2- 172.21.15.3:0/772449527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff344109d70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.362+0000 7ff349200700 1 -- 172.21.15.3:0/772449527 >> 172.21.15.3:0/772449527 conn(0x7ff34407cdc0 msgr2=0x7ff3441074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.362+0000 7ff349200700 1 -- 172.21.15.3:0/772449527 shutdown_connections 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.362+0000 7ff349200700 1 -- 172.21.15.3:0/772449527 wait complete. 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.363+0000 7ff349200700 1 Processor -- start 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.363+0000 7ff349200700 1 -- start start 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.364+0000 7ff349200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff34419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.364+0000 7ff349200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34410deb0 con 0x7ff344109950 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.364+0000 7ff343400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff34419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:20.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.364+0000 7ff343400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff34419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35392/0 (socket says 172.21.15.3:35392) 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.364+0000 7ff343400700 1 -- 172.21.15.3:0/1019179312 learned_addr learned my addr 172.21.15.3:0/1019179312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.364+0000 7ff343400700 1 -- 172.21.15.3:0/1019179312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34419dce0 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.365+0000 7ff343400700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff34419d7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff334000c00 tx=0x7ff334011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.365+0000 7ff33be00700 1 -- 172.21.15.3:0/1019179312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3340039f0 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.365+0000 7ff33be00700 1 -- 172.21.15.3:0/1019179312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff334003b90 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.365+0000 7ff33be00700 1 -- 172.21.15.3:0/1019179312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff33401a740 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.365+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34419df70 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.365+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3441a0c60 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.366+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3440637c0 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.366+0000 7ff33be00700 1 -- 172.21.15.3:0/1019179312 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7ff334024af0 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.367+0000 7ff33be00700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7ff328039240 0x7ff32803b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.367+0000 7ff33be00700 1 -- 172.21.15.3:0/1019179312 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff33404d420 con 0x7ff344109950 2024-03-21T08:18:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.367+0000 7ff342a00700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7ff328039240 0x7ff32803b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.368+0000 7ff342a00700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7ff328039240 0x7ff32803b700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.372+0000 7ff33be00700 1 -- 172.21.15.3:0/1019179312 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff33400e260 con 0x7ff344109950 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.597+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff3441a0e30 con 0x7ff344109950 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.599+0000 7ff33be00700 1 -- 172.21.15.3:0/1019179312 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff33404ce40 con 0x7ff344109950 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7ff328039240 msgr2=0x7ff32803b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7ff328039240 0x7ff32803b700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 msgr2=0x7ff34419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff34419d7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff334000c00 tx=0x7ff334011b00 comp rx=0 tx=0).stop 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 shutdown_connections 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7ff328039240 0x7ff32803b700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 --2- 172.21.15.3:0/1019179312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff344109950 0x7ff34419d7a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 >> 172.21.15.3:0/1019179312 conn(0x7ff34407cdc0 msgr2=0x7ff344106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:20.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 shutdown_connections 2024-03-21T08:18:20.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:19.603+0000 7ff349200700 1 -- 172.21.15.3:0/1019179312 wait complete. 2024-03-21T08:18:20.302 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling cephadm module... 2024-03-21T08:18:21.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:21 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/701946169' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T08:18:21.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:21 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: ignoring --setuser ceph since I am not root 2024-03-21T08:18:21.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:21 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: ignoring --setgroup ceph since I am not root 2024-03-21T08:18:22.044 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.962+0000 7f9ba0c00700 1 Processor -- start 2024-03-21T08:18:22.044 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.963+0000 7f9ba0c00700 1 -- start start 2024-03-21T08:18:22.044 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.963+0000 7f9ba0c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:22.044 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.963+0000 7f9ba0c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c108280 con 0x7f9b9c107890 2024-03-21T08:18:22.044 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.964+0000 7f9b99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:22.044 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.964+0000 7f9b99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35862/0 (socket says 172.21.15.3:35862) 2024-03-21T08:18:22.044 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.964+0000 7f9b99e00700 1 -- 172.21.15.3:0/1922899783 learned_addr learned my addr 172.21.15.3:0/1922899783 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.964+0000 7f9b99e00700 1 -- 172.21.15.3:0/1922899783 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c108a40 con 0x7f9b9c107890 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.964+0000 7f9b99e00700 1 --2- 172.21.15.3:0/1922899783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c107cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0098a0 tx=0x7f9b8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f19aa559fc7e39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.965+0000 7f9b99400700 1 -- 172.21.15.3:0/1922899783 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b8c004160 con 0x7f9b9c107890 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.965+0000 7f9b99400700 1 -- 172.21.15.3:0/1922899783 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9b8c004300 con 0x7f9b9c107890 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.965+0000 7f9b99400700 1 -- 172.21.15.3:0/1922899783 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b8c011a80 con 0x7f9b9c107890 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.965+0000 7f9ba0c00700 1 -- 172.21.15.3:0/1922899783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 msgr2=0x7f9b9c107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.965+0000 7f9ba0c00700 1 --2- 172.21.15.3:0/1922899783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c107cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0098a0 tx=0x7f9b8c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:22.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.966+0000 7f9ba0c00700 1 -- 172.21.15.3:0/1922899783 shutdown_connections 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.966+0000 7f9ba0c00700 1 --2- 172.21.15.3:0/1922899783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c107cb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.966+0000 7f9ba0c00700 1 -- 172.21.15.3:0/1922899783 >> 172.21.15.3:0/1922899783 conn(0x7f9b9c102f50 msgr2=0x7f9b9c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.966+0000 7f9ba0c00700 1 -- 172.21.15.3:0/1922899783 shutdown_connections 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.966+0000 7f9ba0c00700 1 -- 172.21.15.3:0/1922899783 wait complete. 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.967+0000 7f9ba0c00700 1 Processor -- start 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.967+0000 7f9ba0c00700 1 -- start start 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.967+0000 7f9ba0c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.967+0000 7f9ba0c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c10bdf0 con 0x7f9b9c107890 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.967+0000 7f9b99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.967+0000 7f9b99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35878/0 (socket says 172.21.15.3:35878) 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.967+0000 7f9b99e00700 1 -- 172.21.15.3:0/701946169 learned_addr learned my addr 172.21.15.3:0/701946169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.968+0000 7f9b99e00700 1 -- 172.21.15.3:0/701946169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c19dee0 con 0x7f9b9c107890 2024-03-21T08:18:22.051 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.968+0000 7f9b99e00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c19d9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000c00 tx=0x7f9b8c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.968+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b8c0039f0 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.968+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9b8c003b90 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.968+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b8c01a740 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.968+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c19e170 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.969+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c1a0e60 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.970+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7f9b8c024af0 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.970+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c0637c0 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.971+0000 7f9b92a00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f9b80039180 0x7f9b8003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.971+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9b8c04d420 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.971+0000 7f9b93e00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f9b80039180 0x7f9b8003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.971+0000 7f9b93e00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f9b80039180 0x7f9b8003b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b84006fd0 tx=0x7f9b84006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:20.976+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9b8c00e260 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.265+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f9b9c1a1030 con 0x7f9b9c107890 2024-03-21T08:18:22.052 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.318+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f9b9c1a1030 con 0x7f9b9c107890 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f9b80039180 msgr2=0x7f9b8003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f9b80039180 0x7f9b8003b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b84006fd0 tx=0x7f9b84006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 msgr2=0x7f9b9c19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9b92a00700 1 -- 172.21.15.3:0/701946169 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f9b8c02d3f0 con 0x7f9b9c107890 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c19d9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000c00 tx=0x7f9b8c011b00 comp rx=0 tx=0).stop 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 shutdown_connections 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f9b80039180 0x7f9b8003b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 --2- 172.21.15.3:0/701946169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b9c107890 0x7f9b9c19d9a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.322+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 >> 172.21.15.3:0/701946169 conn(0x7f9b9c102f50 msgr2=0x7f9b9c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.323+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 shutdown_connections 2024-03-21T08:18:22.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:21.324+0000 7f9ba0c00700 1 -- 172.21.15.3:0/701946169 wait complete. 2024-03-21T08:18:22.328 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:22 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:22.110+0000 7f6626699200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:18:22.328 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:22 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:22.235+0000 7f6626699200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:18:22.605 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:22 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/701946169' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T08:18:22.605 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:22 smithi003 ceph-mon[31675]: mgrmap e5: x(active, since 5s) 2024-03-21T08:18:23.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:23.072+0000 7f6626699200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:18:23.515 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:23 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1212125172' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T08:18:23.516 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:23.200+0000 7f6626699200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:18:23.516 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:23.384+0000 7f6626699200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:18:23.516 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:23.513+0000 7f6626699200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:18:23.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:18:23.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T08:18:23.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T08:18:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T08:18:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T08:18:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:18:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.661+0000 7f8d67e00700 1 Processor -- start 2024-03-21T08:18:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.661+0000 7f8d67e00700 1 -- start start 2024-03-21T08:18:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.662+0000 7f8d67e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d68107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.662+0000 7f8d67e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68108080 con 0x7f8d68107690 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.662+0000 7f8d66a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d68107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.662+0000 7f8d66a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d68107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35896/0 (socket says 172.21.15.3:35896) 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.662+0000 7f8d66a00700 1 -- 172.21.15.3:0/1208518379 learned_addr learned my addr 172.21.15.3:0/1208518379 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.662+0000 7f8d66a00700 1 -- 172.21.15.3:0/1208518379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68108840 con 0x7f8d68107690 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.663+0000 7f8d66a00700 1 --2- 172.21.15.3:0/1208518379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d68107ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d580098a0 tx=0x7f8d58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e930ddd969aa2eb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.663+0000 7f8d65600700 1 -- 172.21.15.3:0/1208518379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d58004160 con 0x7f8d68107690 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.663+0000 7f8d65600700 1 -- 172.21.15.3:0/1208518379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8d58004300 con 0x7f8d68107690 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.663+0000 7f8d65600700 1 -- 172.21.15.3:0/1208518379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d58011a80 con 0x7f8d68107690 2024-03-21T08:18:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.664+0000 7f8d67e00700 1 -- 172.21.15.3:0/1208518379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 msgr2=0x7f8d68107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:23.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.664+0000 7f8d67e00700 1 --2- 172.21.15.3:0/1208518379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d68107ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d580098a0 tx=0x7f8d58009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.664+0000 7f8d67e00700 1 -- 172.21.15.3:0/1208518379 shutdown_connections 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.664+0000 7f8d67e00700 1 --2- 172.21.15.3:0/1208518379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d68107ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.664+0000 7f8d67e00700 1 -- 172.21.15.3:0/1208518379 >> 172.21.15.3:0/1208518379 conn(0x7f8d68102cd0 msgr2=0x7f8d681050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.664+0000 7f8d67e00700 1 -- 172.21.15.3:0/1208518379 shutdown_connections 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.664+0000 7f8d67e00700 1 -- 172.21.15.3:0/1208518379 wait complete. 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.665+0000 7f8d67e00700 1 Processor -- start 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.665+0000 7f8d67e00700 1 -- start start 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.665+0000 7f8d67e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d6819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.665+0000 7f8d67e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6810bbf0 con 0x7f8d68107690 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.666+0000 7f8d66a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d6819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:23.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.666+0000 7f8d66a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d6819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35908/0 (socket says 172.21.15.3:35908) 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.666+0000 7f8d66a00700 1 -- 172.21.15.3:0/1212125172 learned_addr learned my addr 172.21.15.3:0/1212125172 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.666+0000 7f8d66a00700 1 -- 172.21.15.3:0/1212125172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6819bb30 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.666+0000 7f8d66a00700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d6819b5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d58000c00 tx=0x7f8d58011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.667+0000 7f8d5f400700 1 -- 172.21.15.3:0/1212125172 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d580039f0 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.667+0000 7f8d5f400700 1 -- 172.21.15.3:0/1212125172 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8d58003b90 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.667+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6819bdc0 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.667+0000 7f8d5f400700 1 -- 172.21.15.3:0/1212125172 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d5801a740 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.667+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6819eab0 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.668+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d680637c0 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.668+0000 7f8d5f400700 1 -- 172.21.15.3:0/1212125172 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f8d58024af0 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.669+0000 7f8d5f400700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 0x7f8d4c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.669+0000 7f8d5f400700 1 -- 172.21.15.3:0/1212125172 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8d5804d530 con 0x7f8d68107690 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.669+0000 7f8d66000700 1 -- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 msgr2=0x7f8d4c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1378601347 2024-03-21T08:18:23.685 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.669+0000 7f8d66000700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 0x7f8d4c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.674+0000 7f8d5f400700 1 -- 172.21.15.3:0/1212125172 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d5800e260 con 0x7f8d68107690 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.870+0000 7f8d66000700 1 -- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 msgr2=0x7f8d4c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1378601347 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.870+0000 7f8d66000700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 0x7f8d4c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.963+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8d6819ed20 con 0x7f8d68107690 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.964+0000 7f8d5f400700 1 -- 172.21.15.3:0/1212125172 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8d5800bca0 con 0x7f8d68107690 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 msgr2=0x7f8d4c03b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 0x7f8d4c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 msgr2=0x7f8d6819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d6819b5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d58000c00 tx=0x7f8d58011b00 comp rx=0 tx=0).stop 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 shutdown_connections 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f8d4c0391d0 0x7f8d4c03b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 --2- 172.21.15.3:0/1212125172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d68107690 0x7f8d6819b5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 >> 172.21.15.3:0/1212125172 conn(0x7f8d68102cd0 msgr2=0x7f8d68103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:23.686 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 shutdown_connections 2024-03-21T08:18:23.687 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:22.967+0000 7f8d67e00700 1 -- 172.21.15.3:0/1212125172 wait complete. 2024-03-21T08:18:23.687 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-03-21T08:18:23.687 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 5... 2024-03-21T08:18:23.836 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:23.628+0000 7f6626699200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:18:24.097 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:23.990+0000 7f6626699200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:18:24.390 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:24.137+0000 7f6626699200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:18:24.390 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:24.258+0000 7f6626699200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:18:24.390 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:24.388+0000 7f6626699200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:18:24.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:24.629+0000 7f6626699200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:18:26.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:25 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:25.953+0000 7f6626699200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:18:26.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:26.130+0000 7f6626699200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:18:26.927 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:26.558+0000 7f6626699200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:18:26.927 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:26.679+0000 7f6626699200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:18:27.198 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:26.943+0000 7f6626699200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:18:27.583 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:27.195+0000 7f6626699200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:18:27.584 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:27.581+0000 7f6626699200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:18:28.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:27.903+0000 7f6626699200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:18:28.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:28.218+0000 7f6626699200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:18:29.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:29.280+0000 7f6626699200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:18:30.424 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:29.998+0000 7f6626699200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:18:30.424 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:30.131+0000 7f6626699200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:18:30.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:30.422+0000 7f6626699200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:18:30.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: Active manager daemon x restarted 2024-03-21T08:18:30.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: Activating manager daemon x 2024-03-21T08:18:30.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T08:18:30.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: mgrmap e6: x(active, starting, since 0.00942554s) 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: Manager daemon x is now available 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:30.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:30 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:31.829 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:31 smithi003 ceph-mon[31675]: Found migration_current of "None". Setting to last migration. 2024-03-21T08:18:31.830 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:31 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:18:31.830 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:31 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:18:31.830 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:31 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:18:31.830 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:31 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:18:31.830 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:31 smithi003 ceph-mon[31675]: mgrmap e7: x(active, since 1.01616s) 2024-03-21T08:18:32.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:18:32.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T08:18:32.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T08:18:32.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:18:32.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.248+0000 7f20a5200700 1 Processor -- start 2024-03-21T08:18:32.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.248+0000 7f20a5200700 1 -- start start 2024-03-21T08:18:32.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.249+0000 7f20a5200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a0107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:32.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.249+0000 7f20a5200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01082a0 con 0x7f20a01078b0 2024-03-21T08:18:32.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.249+0000 7f209de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a0107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:32.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.249+0000 7f209de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a0107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35920/0 (socket says 172.21.15.3:35920) 2024-03-21T08:18:32.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.249+0000 7f209de00700 1 -- 172.21.15.3:0/2007681477 learned_addr learned my addr 172.21.15.3:0/2007681477 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.250+0000 7f209de00700 1 -- 172.21.15.3:0/2007681477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0108a60 con 0x7f20a01078b0 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.250+0000 7f209de00700 1 --2- 172.21.15.3:0/2007681477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a0107cd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f20900098a0 tx=0x7f2090009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0142f2c39fbe885 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.250+0000 7f209ca00700 1 -- 172.21.15.3:0/2007681477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2090004160 con 0x7f20a01078b0 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.250+0000 7f209ca00700 1 -- 172.21.15.3:0/2007681477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2090004300 con 0x7f20a01078b0 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.250+0000 7f209ca00700 1 -- 172.21.15.3:0/2007681477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2090011a80 con 0x7f20a01078b0 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.251+0000 7f20a5200700 1 -- 172.21.15.3:0/2007681477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 msgr2=0x7f20a0107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.251+0000 7f20a5200700 1 --2- 172.21.15.3:0/2007681477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a0107cd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f20900098a0 tx=0x7f2090009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.251+0000 7f20a5200700 1 -- 172.21.15.3:0/2007681477 shutdown_connections 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.251+0000 7f20a5200700 1 --2- 172.21.15.3:0/2007681477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a0107cd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.251+0000 7f20a5200700 1 -- 172.21.15.3:0/2007681477 >> 172.21.15.3:0/2007681477 conn(0x7f20a0102f50 msgr2=0x7f20a0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.251+0000 7f20a5200700 1 -- 172.21.15.3:0/2007681477 shutdown_connections 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.251+0000 7f20a5200700 1 -- 172.21.15.3:0/2007681477 wait complete. 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.252+0000 7f20a5200700 1 Processor -- start 2024-03-21T08:18:32.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.253+0000 7f20a5200700 1 -- start start 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.253+0000 7f20a5200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.253+0000 7f20a5200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a010be10 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.253+0000 7f209de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.253+0000 7f209de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35922/0 (socket says 172.21.15.3:35922) 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.253+0000 7f209de00700 1 -- 172.21.15.3:0/2204970135 learned_addr learned my addr 172.21.15.3:0/2204970135 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.253+0000 7f209de00700 1 -- 172.21.15.3:0/2204970135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a019bd50 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.254+0000 7f209de00700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a019b810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2090006af0 tx=0x7f2090011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.254+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2090003930 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.254+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2090003ad0 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.254+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a019bfe0 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.254+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20900195e0 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.254+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a019ecd0 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.255+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f20900198e0 con 0x7f20a01078b0 2024-03-21T08:18:32.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.256+0000 7f2096a00700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.256+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 --> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f208403bd00 con 0x7f2084039130 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.256+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f209004d680 con 0x7f20a01078b0 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.256+0000 7f209d400700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 msgr2=0x7f208403b5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1378601347 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.256+0000 7f209d400700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.457+0000 7f209d400700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 msgr2=0x7f208403b5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1378601347 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.457+0000 7f209d400700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.858+0000 7f209d400700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 msgr2=0x7f208403b5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1378601347 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:24.858+0000 7f209d400700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T08:18:32.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:25.659+0000 7f209d400700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 msgr2=0x7f208403b5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1378601347 2024-03-21T08:18:32.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:25.659+0000 7f209d400700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T08:18:32.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:27.261+0000 7f209d400700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 msgr2=0x7f208403b5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1378601347 2024-03-21T08:18:32.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:27.261+0000 7f209d400700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T08:18:32.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:30.431+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 6) v1 ==== 46061+0+0 (secure 0 0 0) 0x7f209001a430 con 0x7f20a01078b0 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:30.431+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 msgr2=0x7f208403b5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:30.431+0000 7f2096a00700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.435+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) v1 ==== 46188+0+0 (secure 0 0 0) 0x7f209004f000 con 0x7f20a01078b0 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.435+0000 7f2096a00700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f208403ca20 0x7f208403ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.435+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2090001d30 con 0x7f208403ca20 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.437+0000 7f209d400700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f208403ca20 0x7f208403ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.441+0000 7f209d400700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f208403ca20 0x7f208403ee10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f208c003a10 tx=0x7f208c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.442+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f2090001d30 con 0x7f208403ca20 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.454+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f20a00637c0 con 0x7f208403ca20 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.454+0000 7f2096a00700 1 -- 172.21.15.3:0/2204970135 <== mgr.14120 v2:172.21.15.3:6800/2168061035 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f20a00637c0 con 0x7f208403ca20 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f208403ca20 msgr2=0x7f208403ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f208403ca20 0x7f208403ee10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f208c003a10 tx=0x7f208c009290 comp rx=0 tx=0).stop 2024-03-21T08:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 msgr2=0x7f20a019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a019b810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2090006af0 tx=0x7f2090011b70 comp rx=0 tx=0).stop 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 shutdown_connections 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f208403ca20 0x7f208403ee10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20a01078b0 0x7f20a019b810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 --2- 172.21.15.3:0/2204970135 >> [v2:172.21.15.3:6800/1378601347,v1:172.21.15.3:6801/1378601347] conn(0x7f2084039130 0x7f208403b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 >> 172.21.15.3:0/2204970135 conn(0x7f20a0102f50 msgr2=0x7f20a0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.455+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 shutdown_connections 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:31.456+0000 7f20a5200700 1 -- 172.21.15.3:0/2204970135 wait complete. 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 5 is available 2024-03-21T08:18:32.159 INFO:teuthology.orchestra.run.smithi003.stdout:Setting orchestrator backend to cephadm... 2024-03-21T08:18:33.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:33 smithi003 ceph-mon[31675]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T08:18:33.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:33 smithi003 ceph-mon[31675]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T08:18:33.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:33 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:33.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:33 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:33.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:33 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:33.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:33 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.717+0000 7fbc19c00700 1 Processor -- start 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.717+0000 7fbc19c00700 1 -- start start 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.718+0000 7fbc19c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc14109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.718+0000 7fbc19c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1410a590 con 0x7fbc14109ba0 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.718+0000 7fbc12a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc14109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.718+0000 7fbc12a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc14109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38310/0 (socket says 172.21.15.3:38310) 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.718+0000 7fbc12a00700 1 -- 172.21.15.3:0/1224512933 learned_addr learned my addr 172.21.15.3:0/1224512933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.719+0000 7fbc12a00700 1 -- 172.21.15.3:0/1224512933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1410ad00 con 0x7fbc14109ba0 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.719+0000 7fbc12a00700 1 --2- 172.21.15.3:0/1224512933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc14109fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc040098a0 tx=0x7fbc04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19715ea45e356ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:33.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.720+0000 7fbc11600700 1 -- 172.21.15.3:0/1224512933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc04004160 con 0x7fbc14109ba0 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.720+0000 7fbc11600700 1 -- 172.21.15.3:0/1224512933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fbc04004300 con 0x7fbc14109ba0 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.720+0000 7fbc11600700 1 -- 172.21.15.3:0/1224512933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc04011a80 con 0x7fbc14109ba0 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.720+0000 7fbc19c00700 1 -- 172.21.15.3:0/1224512933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 msgr2=0x7fbc14109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.720+0000 7fbc19c00700 1 --2- 172.21.15.3:0/1224512933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc14109fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc040098a0 tx=0x7fbc04009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.721+0000 7fbc19c00700 1 -- 172.21.15.3:0/1224512933 shutdown_connections 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.721+0000 7fbc19c00700 1 --2- 172.21.15.3:0/1224512933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc14109fc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.721+0000 7fbc19c00700 1 -- 172.21.15.3:0/1224512933 >> 172.21.15.3:0/1224512933 conn(0x7fbc141054a0 msgr2=0x7fbc14107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.721+0000 7fbc19c00700 1 -- 172.21.15.3:0/1224512933 shutdown_connections 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.721+0000 7fbc19c00700 1 -- 172.21.15.3:0/1224512933 wait complete. 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.722+0000 7fbc19c00700 1 Processor -- start 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.722+0000 7fbc19c00700 1 -- start start 2024-03-21T08:18:33.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.722+0000 7fbc19c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc1419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.722+0000 7fbc19c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1410e0b0 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.723+0000 7fbc12a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc1419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.723+0000 7fbc12a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc1419d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38324/0 (socket says 172.21.15.3:38324) 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.723+0000 7fbc12a00700 1 -- 172.21.15.3:0/4062139160 learned_addr learned my addr 172.21.15.3:0/4062139160 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.723+0000 7fbc12a00700 1 -- 172.21.15.3:0/4062139160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1419de90 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.723+0000 7fbc12a00700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc1419d950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc04011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.723+0000 7fbc0b400700 1 -- 172.21.15.3:0/4062139160 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc040039f0 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.724+0000 7fbc0b400700 1 -- 172.21.15.3:0/4062139160 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fbc04003b90 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.724+0000 7fbc0b400700 1 -- 172.21.15.3:0/4062139160 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc0401a740 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.724+0000 7fbc19c00700 1 -- 172.21.15.3:0/4062139160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1419e120 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.724+0000 7fbc19c00700 1 -- 172.21.15.3:0/4062139160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc141a0e10 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.725+0000 7fbc0b400700 1 -- 172.21.15.3:0/4062139160 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) v1 ==== 46188+0+0 (secure 0 0 0) 0x7fbc04024af0 con 0x7fbc14109ba0 2024-03-21T08:18:33.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.725+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec005330 con 0x7fbc14109ba0 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.725+0000 7fbc0b400700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fbbf8039170 0x7fbbf803b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.725+0000 7fbc0b400700 1 -- 172.21.15.3:0/4062139160 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbc0404d3d0 con 0x7fbc14109ba0 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.726+0000 7fbc12000700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fbbf8039170 0x7fbbf803b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.726+0000 7fbc12000700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fbbf8039170 0x7fbbf803b630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006fd0 tx=0x7fbc00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.731+0000 7fbc0b400700 1 -- 172.21.15.3:0/4062139160 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbc0402d910 con 0x7fbc14109ba0 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.970+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbec000bc0 con 0x7fbbf8039170 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.983+0000 7fbc0b400700 1 -- 172.21.15.3:0/4062139160 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbbec000bc0 con 0x7fbbf8039170 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fbbf8039170 msgr2=0x7fbbf803b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fbbf8039170 0x7fbbf803b630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006fd0 tx=0x7fbc00006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 msgr2=0x7fbc1419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc1419d950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc04011b00 comp rx=0 tx=0).stop 2024-03-21T08:18:33.654 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 shutdown_connections 2024-03-21T08:18:33.654 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fbbf8039170 0x7fbbf803b630 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:33.654 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 --2- 172.21.15.3:0/4062139160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc14109ba0 0x7fbc1419d950 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:33.654 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 >> 172.21.15.3:0/4062139160 conn(0x7fbc141054a0 msgr2=0x7fbc14106ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:33.654 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 shutdown_connections 2024-03-21T08:18:33.654 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:32.986+0000 7fbc08c00700 1 -- 172.21.15.3:0/4062139160 wait complete. 2024-03-21T08:18:34.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:34 smithi003 ceph-mon[31675]: [21/Mar/2024:08:18:32] ENGINE Bus STARTING 2024-03-21T08:18:34.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:34 smithi003 ceph-mon[31675]: [21/Mar/2024:08:18:32] ENGINE Serving on http://172.21.15.3:8765 2024-03-21T08:18:34.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:34 smithi003 ceph-mon[31675]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:18:34.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:34 smithi003 ceph-mon[31675]: [21/Mar/2024:08:18:33] ENGINE Serving on https://172.21.15.3:7150 2024-03-21T08:18:34.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:34 smithi003 ceph-mon[31675]: [21/Mar/2024:08:18:33] ENGINE Bus STARTED 2024-03-21T08:18:34.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:34 smithi003 ceph-mon[31675]: mgrmap e8: x(active, since 2s) 2024-03-21T08:18:35.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T08:18:35.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.232+0000 7f3da0a00700 1 Processor -- start 2024-03-21T08:18:35.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.232+0000 7f3da0a00700 1 -- start start 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.232+0000 7f3da0a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c10a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.232+0000 7f3da0a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c07ae90 con 0x7f3d9c1082f0 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.233+0000 7f3d99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c10a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.233+0000 7f3d99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c10a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38330/0 (socket says 172.21.15.3:38330) 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.233+0000 7f3d99e00700 1 -- 172.21.15.3:0/1826088245 learned_addr learned my addr 172.21.15.3:0/1826088245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.233+0000 7f3d99e00700 1 -- 172.21.15.3:0/1826088245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c10ac20 con 0x7f3d9c1082f0 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.234+0000 7f3d99e00700 1 --2- 172.21.15.3:0/1826088245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c10a6e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0098a0 tx=0x7f3d8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102761acd25e153b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.234+0000 7f3d98a00700 1 -- 172.21.15.3:0/1826088245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c004160 con 0x7f3d9c1082f0 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.234+0000 7f3d98a00700 1 -- 172.21.15.3:0/1826088245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3d8c004300 con 0x7f3d9c1082f0 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.234+0000 7f3d98a00700 1 -- 172.21.15.3:0/1826088245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c011a80 con 0x7f3d9c1082f0 2024-03-21T08:18:35.113 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.234+0000 7f3da0a00700 1 -- 172.21.15.3:0/1826088245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 msgr2=0x7f3d9c10a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:35.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.234+0000 7f3da0a00700 1 --2- 172.21.15.3:0/1826088245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c10a6e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0098a0 tx=0x7f3d8c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:35.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.235+0000 7f3da0a00700 1 -- 172.21.15.3:0/1826088245 shutdown_connections 2024-03-21T08:18:35.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.235+0000 7f3da0a00700 1 --2- 172.21.15.3:0/1826088245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c10a6e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:35.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.235+0000 7f3da0a00700 1 -- 172.21.15.3:0/1826088245 >> 172.21.15.3:0/1826088245 conn(0x7f3d9c101de0 msgr2=0x7f3d9c104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:35.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.235+0000 7f3da0a00700 1 -- 172.21.15.3:0/1826088245 shutdown_connections 2024-03-21T08:18:35.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.235+0000 7f3da0a00700 1 -- 172.21.15.3:0/1826088245 wait complete. 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.236+0000 7f3da0a00700 1 Processor -- start 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.236+0000 7f3da0a00700 1 -- start start 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.236+0000 7f3da0a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.236+0000 7f3da0a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c10bbf0 con 0x7f3d9c1082f0 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.237+0000 7f3d99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.237+0000 7f3d99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38340/0 (socket says 172.21.15.3:38340) 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.237+0000 7f3d99e00700 1 -- 172.21.15.3:0/2975785146 learned_addr learned my addr 172.21.15.3:0/2975785146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.237+0000 7f3d99e00700 1 -- 172.21.15.3:0/2975785146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c19df40 con 0x7f3d9c1082f0 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.237+0000 7f3d99e00700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c19da00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c011a50 tx=0x7f3d8c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.238+0000 7f3d92a00700 1 -- 172.21.15.3:0/2975785146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c0039f0 con 0x7f3d9c1082f0 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.238+0000 7f3d92a00700 1 -- 172.21.15.3:0/2975785146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f3d8c003b90 con 0x7f3d9c1082f0 2024-03-21T08:18:35.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.238+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c19e1d0 con 0x7f3d9c1082f0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.238+0000 7f3d92a00700 1 -- 172.21.15.3:0/2975785146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c01a600 con 0x7f3d9c1082f0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.238+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c1a0ec0 con 0x7f3d9c1082f0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.239+0000 7f3d92a00700 1 -- 172.21.15.3:0/2975785146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f3d8c024a40 con 0x7f3d9c1082f0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.239+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c0637c0 con 0x7f3d9c1082f0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.240+0000 7f3d92a00700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f3d800391d0 0x7f3d8003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.240+0000 7f3d92a00700 1 -- 172.21.15.3:0/2975785146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3d8c04d650 con 0x7f3d9c1082f0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.240+0000 7f3d99400700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f3d800391d0 0x7f3d8003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.241+0000 7f3d99400700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f3d800391d0 0x7f3d8003b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d88006fd0 tx=0x7f3d88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.248+0000 7f3d92a00700 1 -- 172.21.15.3:0/2975785146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3d8c00e260 con 0x7f3d9c1082f0 2024-03-21T08:18:35.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.475+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d9c102fa0 con 0x7f3d800391d0 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.477+0000 7f3d92a00700 1 -- 172.21.15.3:0/2975785146 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f3d9c102fa0 con 0x7f3d800391d0 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f3d800391d0 msgr2=0x7f3d8003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f3d800391d0 0x7f3d8003b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d88006fd0 tx=0x7f3d88006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 msgr2=0x7f3d9c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c19da00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c011a50 tx=0x7f3d8c004290 comp rx=0 tx=0).stop 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 shutdown_connections 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f3d800391d0 0x7f3d8003b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 --2- 172.21.15.3:0/2975785146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d9c1082f0 0x7f3d9c19da00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 >> 172.21.15.3:0/2975785146 conn(0x7f3d9c101de0 msgr2=0x7f3d9c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 shutdown_connections 2024-03-21T08:18:35.124 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:34.480+0000 7f3da0a00700 1 -- 172.21.15.3:0/2975785146 wait complete. 2024-03-21T08:18:35.125 INFO:teuthology.orchestra.run.smithi003.stdout:Generating ssh key... 2024-03-21T08:18:35.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:35 smithi003 ceph-mon[31675]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:18:36.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: Generating public/private rsa key pair. 2024-03-21T08:18:36.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: Your identification has been saved in /tmp/tmphsls010x/key. 2024-03-21T08:18:36.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: Your public key has been saved in /tmp/tmphsls010x/key.pub. 2024-03-21T08:18:36.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: The key fingerprint is: 2024-03-21T08:18:36.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: SHA256:ZWJs9BKv7ZiiC5iwxck/RO9lFC7ZM08IJaq2GQ5kxLM ceph-544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: The key's randomart image is: 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: +---[RSA 3072]----+ 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: |.. o.= | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: |.o . O * | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: | oo o o & * | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: |oE + . = % | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: |o O . . S o | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: |.O * . o + | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: |+ = o o o . | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: | . o . | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: | o. | 2024-03-21T08:18:36.428 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: +----[SHA256]-----+ 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.893+0000 7f6bf3e00700 1 Processor -- start 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.893+0000 7f6bf3e00700 1 -- start start 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.894+0000 7f6bf3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf4107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.894+0000 7f6bf3e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4108080 con 0x7f6bf4107690 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.894+0000 7f6bf2a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf4107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.894+0000 7f6bf2a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf4107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38354/0 (socket says 172.21.15.3:38354) 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.894+0000 7f6bf2a00700 1 -- 172.21.15.3:0/1216587207 learned_addr learned my addr 172.21.15.3:0/1216587207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:37.053 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.895+0000 7f6bf2a00700 1 -- 172.21.15.3:0/1216587207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf4108840 con 0x7f6bf4107690 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.895+0000 7f6bf2a00700 1 --2- 172.21.15.3:0/1216587207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf4107ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6be40098a0 tx=0x7f6be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62ac924ed4fd1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.895+0000 7f6bf1600700 1 -- 172.21.15.3:0/1216587207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be4004160 con 0x7f6bf4107690 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.895+0000 7f6bf1600700 1 -- 172.21.15.3:0/1216587207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f6be4004300 con 0x7f6bf4107690 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.895+0000 7f6bf1600700 1 -- 172.21.15.3:0/1216587207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be4011a80 con 0x7f6bf4107690 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.896+0000 7f6bf3e00700 1 -- 172.21.15.3:0/1216587207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 msgr2=0x7f6bf4107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.896+0000 7f6bf3e00700 1 --2- 172.21.15.3:0/1216587207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf4107ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6be40098a0 tx=0x7f6be4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.896+0000 7f6bf3e00700 1 -- 172.21.15.3:0/1216587207 shutdown_connections 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.896+0000 7f6bf3e00700 1 --2- 172.21.15.3:0/1216587207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf4107ab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.896+0000 7f6bf3e00700 1 -- 172.21.15.3:0/1216587207 >> 172.21.15.3:0/1216587207 conn(0x7f6bf4102cd0 msgr2=0x7f6bf41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.896+0000 7f6bf3e00700 1 -- 172.21.15.3:0/1216587207 shutdown_connections 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.896+0000 7f6bf3e00700 1 -- 172.21.15.3:0/1216587207 wait complete. 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.897+0000 7f6bf3e00700 1 Processor -- start 2024-03-21T08:18:37.054 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.898+0000 7f6bf3e00700 1 -- start start 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.898+0000 7f6bf3e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.898+0000 7f6bf3e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf410bbf0 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.898+0000 7f6bf2a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.898+0000 7f6bf2a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38364/0 (socket says 172.21.15.3:38364) 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.898+0000 7f6bf2a00700 1 -- 172.21.15.3:0/2029621316 learned_addr learned my addr 172.21.15.3:0/2029621316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.899+0000 7f6bf2a00700 1 -- 172.21.15.3:0/2029621316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf419dd30 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.899+0000 7f6bf2a00700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf419d7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.899+0000 7f6bf8a00700 1 -- 172.21.15.3:0/2029621316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be40039f0 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.899+0000 7f6bf8a00700 1 -- 172.21.15.3:0/2029621316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f6be4003b90 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.899+0000 7f6bf8a00700 1 -- 172.21.15.3:0/2029621316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be401a740 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.899+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf419dfc0 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.899+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf41a0cb0 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.901+0000 7f6bf8a00700 1 -- 172.21.15.3:0/2029621316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f6be4024af0 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.901+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf40637c0 con 0x7f6bf4107690 2024-03-21T08:18:37.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.901+0000 7f6bf8a00700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f6bd80391d0 0x7f6bd803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:37.056 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.901+0000 7f6bf8a00700 1 -- 172.21.15.3:0/2029621316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6be404d5a0 con 0x7f6bf4107690 2024-03-21T08:18:37.056 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.901+0000 7f6bf2000700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f6bd80391d0 0x7f6bd803b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:37.057 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.902+0000 7f6bf2000700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f6bd80391d0 0x7f6bd803b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:37.057 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:35.906+0000 7f6bf8a00700 1 -- 172.21.15.3:0/2029621316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6be404cef0 con 0x7f6bf4107690 2024-03-21T08:18:37.057 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.134+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bf4103f50 con 0x7f6bd80391d0 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.362+0000 7f6bf8a00700 1 -- 172.21.15.3:0/2029621316 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6bf4103f50 con 0x7f6bd80391d0 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.365+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f6bd80391d0 msgr2=0x7f6bd803b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.365+0000 7f6bf3e00700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f6bd80391d0 0x7f6bd803b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.365+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 msgr2=0x7f6bf419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.365+0000 7f6bf3e00700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf419d7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be4011b00 comp rx=0 tx=0).stop 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.366+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 shutdown_connections 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.366+0000 7f6bf3e00700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f6bd80391d0 0x7f6bd803b690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.366+0000 7f6bf3e00700 1 --2- 172.21.15.3:0/2029621316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bf4107690 0x7f6bf419d7f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.366+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 >> 172.21.15.3:0/2029621316 conn(0x7f6bf4102cd0 msgr2=0x7f6bf4103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.366+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 shutdown_connections 2024-03-21T08:18:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:36.366+0000 7f6bf3e00700 1 -- 172.21.15.3:0/2029621316 wait complete. 2024-03-21T08:18:37.675 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:37 smithi003 ceph-mon[31675]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:18:37.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:37 smithi003 ceph-mon[31675]: Generating ssh key... 2024-03-21T08:18:37.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:37 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:37.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:37 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:38.557 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnRrEwr8KBU6tmaST4rZPAWoIukpVtovy3cNIB7or967TzDpTHJNJ4zw0jISpRkLuFSoSy0tkw7l77IW3/7DkiPIwRkxh1iH8Q1V1qdbEfKnVnSui81TIdUKCLKfbYlnoP+FRQfg7LfXCoMW/SOUg4KxPibVeL9HbUbEWVjbgGuehA8p5CFtWDT54+uyzJmPFaNE+Ztio4cErXJhkzHw4NtJRCNYRNcdRml6cc6he8LOa83hJRf6iQuoOucZZspFvm9pdxEwxUHJHx524mDVnyAuOguwO/IkFq/Ndc1u7EweUgKenYdy1IjzVG1hQ80acJozJpzBB3o+uWj+tYRmWYN+NjXEc0C0cXxA+8QQ78RHqjUuLBwfl6PNMDscvGCZNg9E7QCYwEPxI5eqq65C4KYuAp3+4vmHjvQb1Au0EtefPKTuTpjOkPkBvbwFht3ZH/xt80D7Odvv/Vue3AoXRjMReQA4tJLaXYknPTG6YW2cqOcWH9CoQh51TP2q5jJjs= ceph-544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.672+0000 7fc16a000700 1 Processor -- start 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.673+0000 7fc16a000700 1 -- start start 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.673+0000 7fc16a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc16410e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.673+0000 7fc16a000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16410e9a0 con 0x7fc16410e040 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.673+0000 7fc163400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc16410e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.674+0000 7fc163400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc16410e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38368/0 (socket says 172.21.15.3:38368) 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.674+0000 7fc163400700 1 -- 172.21.15.3:0/726095754 learned_addr learned my addr 172.21.15.3:0/726095754 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.674+0000 7fc163400700 1 -- 172.21.15.3:0/726095754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16410f060 con 0x7fc16410e040 2024-03-21T08:18:38.558 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.674+0000 7fc163400700 1 --2- 172.21.15.3:0/726095754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc16410e460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc1540098a0 tx=0x7fc154009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8046368382335ba2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:38.559 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc162000700 1 -- 172.21.15.3:0/726095754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc154004160 con 0x7fc16410e040 2024-03-21T08:18:38.559 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc162000700 1 -- 172.21.15.3:0/726095754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc154004300 con 0x7fc16410e040 2024-03-21T08:18:38.559 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc162000700 1 -- 172.21.15.3:0/726095754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc154011a80 con 0x7fc16410e040 2024-03-21T08:18:38.559 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc16a000700 1 -- 172.21.15.3:0/726095754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 msgr2=0x7fc16410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:38.559 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc16a000700 1 --2- 172.21.15.3:0/726095754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc16410e460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc1540098a0 tx=0x7fc154009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:38.559 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc16a000700 1 -- 172.21.15.3:0/726095754 shutdown_connections 2024-03-21T08:18:38.560 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc16a000700 1 --2- 172.21.15.3:0/726095754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc16410e460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:38.560 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.675+0000 7fc16a000700 1 -- 172.21.15.3:0/726095754 >> 172.21.15.3:0/726095754 conn(0x7fc16407d090 msgr2=0x7fc16407f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:38.560 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.676+0000 7fc16a000700 1 -- 172.21.15.3:0/726095754 shutdown_connections 2024-03-21T08:18:38.560 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.676+0000 7fc16a000700 1 -- 172.21.15.3:0/726095754 wait complete. 2024-03-21T08:18:38.560 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.677+0000 7fc16a000700 1 Processor -- start 2024-03-21T08:18:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.677+0000 7fc16a000700 1 -- start start 2024-03-21T08:18:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.677+0000 7fc16a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc1641a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.677+0000 7fc16a000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc164112320 con 0x7fc16410e040 2024-03-21T08:18:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.677+0000 7fc163400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc1641a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.677+0000 7fc163400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc1641a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38382/0 (socket says 172.21.15.3:38382) 2024-03-21T08:18:38.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.677+0000 7fc163400700 1 -- 172.21.15.3:0/1362374252 learned_addr learned my addr 172.21.15.3:0/1362374252 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.678+0000 7fc163400700 1 -- 172.21.15.3:0/1362374252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1641a22a0 con 0x7fc16410e040 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.678+0000 7fc163400700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc1641a1d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc154000c00 tx=0x7fc154011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.678+0000 7fc15be00700 1 -- 172.21.15.3:0/1362374252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1540039f0 con 0x7fc16410e040 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.678+0000 7fc15be00700 1 -- 172.21.15.3:0/1362374252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc154003b90 con 0x7fc16410e040 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.678+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1641a2530 con 0x7fc16410e040 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.678+0000 7fc15be00700 1 -- 172.21.15.3:0/1362374252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc15401a740 con 0x7fc16410e040 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.679+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1641a5220 con 0x7fc16410e040 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.680+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1641124a0 con 0x7fc16410e040 2024-03-21T08:18:38.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.680+0000 7fc15be00700 1 -- 172.21.15.3:0/1362374252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fc154024af0 con 0x7fc16410e040 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.680+0000 7fc15be00700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fc148039290 0x7fc14803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.681+0000 7fc15be00700 1 -- 172.21.15.3:0/1362374252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc15404d5a0 con 0x7fc16410e040 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.681+0000 7fc162a00700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fc148039290 0x7fc14803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.682+0000 7fc162a00700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fc148039290 0x7fc14803b750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc150006fd0 tx=0x7fc150006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.686+0000 7fc15be00700 1 -- 172.21.15.3:0/1362374252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc15404cef0 con 0x7fc16410e040 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.914+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc16407e200 con 0x7fc148039290 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.915+0000 7fc15be00700 1 -- 172.21.15.3:0/1362374252 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fc16407e200 con 0x7fc148039290 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fc148039290 msgr2=0x7fc14803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fc148039290 0x7fc14803b750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc150006fd0 tx=0x7fc150006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 msgr2=0x7fc1641a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc1641a1d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc154000c00 tx=0x7fc154011b00 comp rx=0 tx=0).stop 2024-03-21T08:18:38.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 shutdown_connections 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fc148039290 0x7fc14803b750 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 --2- 172.21.15.3:0/1362374252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc16410e040 0x7fc1641a1d60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.918+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 >> 172.21.15.3:0/1362374252 conn(0x7fc16407d090 msgr2=0x7fc16407daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.919+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 shutdown_connections 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:37.919+0000 7fc16a000700 1 -- 172.21.15.3:0/1362374252 wait complete. 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T08:18:38.564 INFO:teuthology.orchestra.run.smithi003.stdout:Adding host smithi003... 2024-03-21T08:18:38.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:38 smithi003 ceph-mon[31675]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:18:39.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:39 smithi003 ceph-mon[31675]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:18:41.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:41 smithi003 ceph-mon[31675]: Deploying cephadm binary to smithi003 2024-03-21T08:18:46.868 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Added host 'smithi003' with addr '172.21.15.3' 2024-03-21T08:18:46.868 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.128+0000 7ff1e9000700 1 Processor -- start 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.128+0000 7ff1e9000700 1 -- start start 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.129+0000 7ff1e9000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e4109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.129+0000 7ff1e9000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e410a590 con 0x7ff1e4109ba0 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.129+0000 7ff1e1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e4109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.130+0000 7ff1e1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e4109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38394/0 (socket says 172.21.15.3:38394) 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.130+0000 7ff1e1e00700 1 -- 172.21.15.3:0/1399026500 learned_addr learned my addr 172.21.15.3:0/1399026500 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.130+0000 7ff1e1e00700 1 -- 172.21.15.3:0/1399026500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e410ad00 con 0x7ff1e4109ba0 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.131+0000 7ff1e1e00700 1 --2- 172.21.15.3:0/1399026500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e4109fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098a0 tx=0x7ff1d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad5e53d56236b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.131+0000 7ff1e0a00700 1 -- 172.21.15.3:0/1399026500 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d4004160 con 0x7ff1e4109ba0 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.131+0000 7ff1e0a00700 1 -- 172.21.15.3:0/1399026500 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff1d4004300 con 0x7ff1e4109ba0 2024-03-21T08:18:46.869 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.131+0000 7ff1e0a00700 1 -- 172.21.15.3:0/1399026500 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d4011a80 con 0x7ff1e4109ba0 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.132+0000 7ff1e9000700 1 -- 172.21.15.3:0/1399026500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 msgr2=0x7ff1e4109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.132+0000 7ff1e9000700 1 --2- 172.21.15.3:0/1399026500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e4109fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098a0 tx=0x7ff1d4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.132+0000 7ff1e9000700 1 -- 172.21.15.3:0/1399026500 shutdown_connections 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.132+0000 7ff1e9000700 1 --2- 172.21.15.3:0/1399026500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e4109fc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.132+0000 7ff1e9000700 1 -- 172.21.15.3:0/1399026500 >> 172.21.15.3:0/1399026500 conn(0x7ff1e41054a0 msgr2=0x7ff1e4107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.132+0000 7ff1e9000700 1 -- 172.21.15.3:0/1399026500 shutdown_connections 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.132+0000 7ff1e9000700 1 -- 172.21.15.3:0/1399026500 wait complete. 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.133+0000 7ff1e9000700 1 Processor -- start 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.133+0000 7ff1e9000700 1 -- start start 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.133+0000 7ff1e9000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.134+0000 7ff1e9000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e410e0b0 con 0x7ff1e4109ba0 2024-03-21T08:18:46.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.134+0000 7ff1e1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.134+0000 7ff1e1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e419d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38404/0 (socket says 172.21.15.3:38404) 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.134+0000 7ff1e1e00700 1 -- 172.21.15.3:0/3727680365 learned_addr learned my addr 172.21.15.3:0/3727680365 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.134+0000 7ff1e1e00700 1 -- 172.21.15.3:0/3727680365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e419dee0 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.134+0000 7ff1e1e00700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e419d9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006af0 tx=0x7ff1d4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.135+0000 7ff1daa00700 1 -- 172.21.15.3:0/3727680365 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d4003930 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.135+0000 7ff1daa00700 1 -- 172.21.15.3:0/3727680365 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff1d4003ad0 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.135+0000 7ff1daa00700 1 -- 172.21.15.3:0/3727680365 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1d40195e0 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.135+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e419e170 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.135+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e41a0e60 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.136+0000 7ff1daa00700 1 -- 172.21.15.3:0/3727680365 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7ff1d401a430 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.136+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e40637c0 con 0x7ff1e4109ba0 2024-03-21T08:18:46.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.137+0000 7ff1daa00700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7ff1c8039240 0x7ff1c803b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.137+0000 7ff1daa00700 1 -- 172.21.15.3:0/3727680365 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff1d404d6e0 con 0x7ff1e4109ba0 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.137+0000 7ff1e1400700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7ff1c8039240 0x7ff1c803b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.138+0000 7ff1e1400700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7ff1c8039240 0x7ff1c803b700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0006fd0 tx=0x7ff1d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.142+0000 7ff1daa00700 1 -- 172.21.15.3:0/3727680365 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff1d40172b0 con 0x7ff1e4109ba0 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:39.373+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1e4107640 con 0x7ff1c8039240 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.108+0000 7ff1daa00700 1 -- 172.21.15.3:0/3727680365 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ff1e4107640 con 0x7ff1c8039240 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7ff1c8039240 msgr2=0x7ff1c803b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7ff1c8039240 0x7ff1c803b700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0006fd0 tx=0x7ff1d0006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 msgr2=0x7ff1e419d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e419d9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006af0 tx=0x7ff1d4011b70 comp rx=0 tx=0).stop 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 shutdown_connections 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7ff1c8039240 0x7ff1c803b700 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:46.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 --2- 172.21.15.3:0/3727680365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1e4109ba0 0x7ff1e419d9a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:46.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 >> 172.21.15.3:0/3727680365 conn(0x7ff1e41054a0 msgr2=0x7ff1e4106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:46.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.112+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 shutdown_connections 2024-03-21T08:18:46.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:46.113+0000 7ff1e9000700 1 -- 172.21.15.3:0/3727680365 wait complete. 2024-03-21T08:18:46.873 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mon service... 2024-03-21T08:18:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:47 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:47 smithi003 ceph-mon[31675]: Added host smithi003 2024-03-21T08:18:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:47 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:18:48.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T08:18:48.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.600+0000 7fce1b600700 1 Processor -- start 2024-03-21T08:18:48.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.601+0000 7fce1b600700 1 -- start start 2024-03-21T08:18:48.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.601+0000 7fce1b600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:48.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.601+0000 7fce1b600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1407b800 con 0x7fce1407ce10 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.601+0000 7fce18c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.602+0000 7fce18c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1407b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42346/0 (socket says 172.21.15.3:42346) 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.602+0000 7fce18c00700 1 -- 172.21.15.3:0/2675467610 learned_addr learned my addr 172.21.15.3:0/2675467610 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.602+0000 7fce18c00700 1 -- 172.21.15.3:0/2675467610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1407b980 con 0x7fce1407ce10 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.602+0000 7fce18c00700 1 --2- 172.21.15.3:0/2675467610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1407b2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fce080098a0 tx=0x7fce08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d835d46a0e139e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.603+0000 7fce13400700 1 -- 172.21.15.3:0/2675467610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce08004160 con 0x7fce1407ce10 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.603+0000 7fce13400700 1 -- 172.21.15.3:0/2675467610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fce08004300 con 0x7fce1407ce10 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.603+0000 7fce13400700 1 -- 172.21.15.3:0/2675467610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce08011a80 con 0x7fce1407ce10 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.604+0000 7fce1b600700 1 -- 172.21.15.3:0/2675467610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 msgr2=0x7fce1407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:48.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.604+0000 7fce1b600700 1 --2- 172.21.15.3:0/2675467610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1407b2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fce080098a0 tx=0x7fce08009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.604+0000 7fce1b600700 1 -- 172.21.15.3:0/2675467610 shutdown_connections 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.604+0000 7fce1b600700 1 --2- 172.21.15.3:0/2675467610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1407b2c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.604+0000 7fce1b600700 1 -- 172.21.15.3:0/2675467610 >> 172.21.15.3:0/2675467610 conn(0x7fce14102f50 msgr2=0x7fce14105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.604+0000 7fce1b600700 1 -- 172.21.15.3:0/2675467610 shutdown_connections 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.604+0000 7fce1b600700 1 -- 172.21.15.3:0/2675467610 wait complete. 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.605+0000 7fce1b600700 1 Processor -- start 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.606+0000 7fce1b600700 1 -- start start 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.606+0000 7fce1b600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.606+0000 7fce1b600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1410de00 con 0x7fce1407ce10 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.606+0000 7fce18c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.606+0000 7fce18c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42352/0 (socket says 172.21.15.3:42352) 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.606+0000 7fce18c00700 1 -- 172.21.15.3:0/644040012 learned_addr learned my addr 172.21.15.3:0/644040012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:48.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.607+0000 7fce18c00700 1 -- 172.21.15.3:0/644040012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1419deb0 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.607+0000 7fce18c00700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1419d970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fce08011a50 tx=0x7fce08011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.607+0000 7fce11600700 1 -- 172.21.15.3:0/644040012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce08003a80 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.607+0000 7fce11600700 1 -- 172.21.15.3:0/644040012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fce08003c20 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.607+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1419e140 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.607+0000 7fce11600700 1 -- 172.21.15.3:0/644040012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce0801a650 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.607+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce141a0e30 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.609+0000 7fce11600700 1 -- 172.21.15.3:0/644040012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fce0801a950 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.609+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce140637c0 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.609+0000 7fce11600700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fcdfc039180 0x7fcdfc03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.609+0000 7fce11600700 1 -- 172.21.15.3:0/644040012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fce0804d7c0 con 0x7fce1407ce10 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.609+0000 7fce13e00700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fcdfc039180 0x7fcdfc03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.610+0000 7fce13e00700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fcdfc039180 0x7fcdfc03b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce00006fd0 tx=0x7fce00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:48.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.616+0000 7fce11600700 1 -- 172.21.15.3:0/644040012 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce0800bc40 con 0x7fce1407ce10 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.862+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fce141041a0 con 0x7fcdfc039180 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.872+0000 7fce11600700 1 -- 172.21.15.3:0/644040012 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fce141041a0 con 0x7fcdfc039180 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fcdfc039180 msgr2=0x7fcdfc03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fcdfc039180 0x7fcdfc03b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce00006fd0 tx=0x7fce00006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 msgr2=0x7fce1419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1419d970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fce08011a50 tx=0x7fce08011b30 comp rx=0 tx=0).stop 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 shutdown_connections 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fcdfc039180 0x7fcdfc03b640 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 --2- 172.21.15.3:0/644040012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce1407ce10 0x7fce1419d970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 >> 172.21.15.3:0/644040012 conn(0x7fce14102f50 msgr2=0x7fce14103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 shutdown_connections 2024-03-21T08:18:48.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:47.875+0000 7fce1b600700 1 -- 172.21.15.3:0/644040012 wait complete. 2024-03-21T08:18:48.988 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mgr service... 2024-03-21T08:18:49.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:48 smithi003 ceph-mon[31675]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:18:49.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:48 smithi003 ceph-mon[31675]: Saving service mon spec with placement count:5 2024-03-21T08:18:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:48 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:50.625 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:50 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:50.625 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:50 smithi003 ceph-mon[31675]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:18:50.625 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:50 smithi003 ceph-mon[31675]: Saving service mgr spec with placement count:2 2024-03-21T08:18:50.625 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:50 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:50.626 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:50 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:50.933 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T08:18:50.933 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.021+0000 7f5a86200700 1 Processor -- start 2024-03-21T08:18:50.933 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.021+0000 7f5a86200700 1 -- start start 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.022+0000 7f5a86200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8007b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.022+0000 7f5a86200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8007b8b0 con 0x7f5a8007ce30 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.022+0000 7f5a84e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8007b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.022+0000 7f5a84e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8007b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53440/0 (socket says 172.21.15.3:53440) 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.022+0000 7f5a84e00700 1 -- 172.21.15.3:0/1374051963 learned_addr learned my addr 172.21.15.3:0/1374051963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.023+0000 7f5a84e00700 1 -- 172.21.15.3:0/1374051963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8007ba30 con 0x7f5a8007ce30 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.023+0000 7f5a84e00700 1 --2- 172.21.15.3:0/1374051963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8007b2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a740098a0 tx=0x7f5a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b54869fa7542e88 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.023+0000 7f5a7f400700 1 -- 172.21.15.3:0/1374051963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a74004160 con 0x7f5a8007ce30 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.023+0000 7f5a7f400700 1 -- 172.21.15.3:0/1374051963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a74004300 con 0x7f5a8007ce30 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.023+0000 7f5a7f400700 1 -- 172.21.15.3:0/1374051963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a74011a50 con 0x7f5a8007ce30 2024-03-21T08:18:50.934 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.024+0000 7f5a86200700 1 -- 172.21.15.3:0/1374051963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 msgr2=0x7f5a8007b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.024+0000 7f5a86200700 1 --2- 172.21.15.3:0/1374051963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8007b2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a740098a0 tx=0x7f5a74009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.024+0000 7f5a86200700 1 -- 172.21.15.3:0/1374051963 shutdown_connections 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.024+0000 7f5a86200700 1 --2- 172.21.15.3:0/1374051963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8007b2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.024+0000 7f5a86200700 1 -- 172.21.15.3:0/1374051963 >> 172.21.15.3:0/1374051963 conn(0x7f5a80102cd0 msgr2=0x7f5a801050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.024+0000 7f5a86200700 1 -- 172.21.15.3:0/1374051963 shutdown_connections 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.024+0000 7f5a86200700 1 -- 172.21.15.3:0/1374051963 wait complete. 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.025+0000 7f5a86200700 1 Processor -- start 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.025+0000 7f5a86200700 1 -- start start 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.026+0000 7f5a86200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.026+0000 7f5a86200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8010dc30 con 0x7f5a8007ce30 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.026+0000 7f5a84e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.026+0000 7f5a84e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53448/0 (socket says 172.21.15.3:53448) 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.026+0000 7f5a84e00700 1 -- 172.21.15.3:0/1512090986 learned_addr learned my addr 172.21.15.3:0/1512090986 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:50.935 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.026+0000 7f5a84e00700 1 -- 172.21.15.3:0/1512090986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8019dc80 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.027+0000 7f5a84e00700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8019d740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a74006b20 tx=0x7f5a74011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.027+0000 7f5a7d600700 1 -- 172.21.15.3:0/1512090986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a74003b70 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.027+0000 7f5a7d600700 1 -- 172.21.15.3:0/1512090986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a7401a8e0 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.027+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8019df10 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.027+0000 7f5a7d600700 1 -- 172.21.15.3:0/1512090986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a740249b0 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.027+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a801a0c00 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.028+0000 7f5a7d600700 1 -- 172.21.15.3:0/1512090986 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f5a74022020 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.028+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a800637c0 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.029+0000 7f5a7d600700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5a680391d0 0x7f5a6803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.029+0000 7f5a7d600700 1 -- 172.21.15.3:0/1512090986 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5a7404d9d0 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.030+0000 7f5a7fe00700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5a680391d0 0x7f5a6803b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.030+0000 7f5a7fe00700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5a680391d0 0x7f5a6803b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c006fd0 tx=0x7f5a6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.034+0000 7f5a7d600700 1 -- 172.21.15.3:0/1512090986 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a7400b3e0 con 0x7f5a8007ce30 2024-03-21T08:18:50.936 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.278+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5a80103f10 con 0x7f5a680391d0 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.286+0000 7f5a7d600700 1 -- 172.21.15.3:0/1512090986 <== mgr.14120 v2:172.21.15.3:6800/2168061035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5a80103f10 con 0x7f5a680391d0 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.288+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5a680391d0 msgr2=0x7f5a6803b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.288+0000 7f5a86200700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5a680391d0 0x7f5a6803b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c006fd0 tx=0x7f5a6c006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.288+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 msgr2=0x7f5a8019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.288+0000 7f5a86200700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8019d740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a74006b20 tx=0x7f5a74011ad0 comp rx=0 tx=0).stop 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.289+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 shutdown_connections 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.289+0000 7f5a86200700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5a680391d0 0x7f5a6803b690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.289+0000 7f5a86200700 1 --2- 172.21.15.3:0/1512090986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a8007ce30 0x7f5a8019d740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.289+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 >> 172.21.15.3:0/1512090986 conn(0x7f5a80102cd0 msgr2=0x7f5a80103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.289+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 shutdown_connections 2024-03-21T08:18:50.937 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:50.289+0000 7f5a86200700 1 -- 172.21.15.3:0/1512090986 wait complete. 2024-03-21T08:18:51.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:51 smithi003 ceph-mon[31675]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:18:53.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:52 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/444889260' entity='client.admin' 2024-03-21T08:18:53.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:52 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:53.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.536+0000 7fd861e00700 1 Processor -- start 2024-03-21T08:18:53.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.536+0000 7fd861e00700 1 -- start start 2024-03-21T08:18:53.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.536+0000 7fd861e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c07b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:53.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.536+0000 7fd861e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c07b8d0 con 0x7fd85c07ce50 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.537+0000 7fd860a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c07b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.537+0000 7fd860a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c07b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53450/0 (socket says 172.21.15.3:53450) 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.537+0000 7fd860a00700 1 -- 172.21.15.3:0/911964934 learned_addr learned my addr 172.21.15.3:0/911964934 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.537+0000 7fd860a00700 1 -- 172.21.15.3:0/911964934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c07ba50 con 0x7fd85c07ce50 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.538+0000 7fd860a00700 1 --2- 172.21.15.3:0/911964934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c07b300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8500098a0 tx=0x7fd850009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee943404cd8da19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.538+0000 7fd85b400700 1 -- 172.21.15.3:0/911964934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd850004160 con 0x7fd85c07ce50 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.538+0000 7fd85b400700 1 -- 172.21.15.3:0/911964934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd850004300 con 0x7fd85c07ce50 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.538+0000 7fd85b400700 1 -- 172.21.15.3:0/911964934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd850011a50 con 0x7fd85c07ce50 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.539+0000 7fd861e00700 1 -- 172.21.15.3:0/911964934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 msgr2=0x7fd85c07b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.539+0000 7fd861e00700 1 --2- 172.21.15.3:0/911964934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c07b300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8500098a0 tx=0x7fd850009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.539+0000 7fd861e00700 1 -- 172.21.15.3:0/911964934 shutdown_connections 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.539+0000 7fd861e00700 1 --2- 172.21.15.3:0/911964934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c07b300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:53.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.539+0000 7fd861e00700 1 -- 172.21.15.3:0/911964934 >> 172.21.15.3:0/911964934 conn(0x7fd85c102dc0 msgr2=0x7fd85c1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.539+0000 7fd861e00700 1 -- 172.21.15.3:0/911964934 shutdown_connections 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.539+0000 7fd861e00700 1 -- 172.21.15.3:0/911964934 wait complete. 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.540+0000 7fd861e00700 1 Processor -- start 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.541+0000 7fd861e00700 1 -- start start 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.541+0000 7fd861e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c10a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.541+0000 7fd861e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c10dce0 con 0x7fd85c07ce50 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.541+0000 7fd860a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c10a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.541+0000 7fd860a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c10a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53458/0 (socket says 172.21.15.3:53458) 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.541+0000 7fd860a00700 1 -- 172.21.15.3:0/444889260 learned_addr learned my addr 172.21.15.3:0/444889260 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.542+0000 7fd860a00700 1 -- 172.21.15.3:0/444889260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c10af10 con 0x7fd85c07ce50 2024-03-21T08:18:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.542+0000 7fd860a00700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c10a9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd850011a00 tx=0x7fd850011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.542+0000 7fd859600700 1 -- 172.21.15.3:0/444889260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd850003af0 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.542+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c10b1a0 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.542+0000 7fd859600700 1 -- 172.21.15.3:0/444889260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd85001a8e0 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.542+0000 7fd859600700 1 -- 172.21.15.3:0/444889260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd850024840 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.543+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c107760 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.544+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c10b370 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.544+0000 7fd859600700 1 -- 172.21.15.3:0/444889260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fd850022020 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.545+0000 7fd859600700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fd844039290 0x7fd84403b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.545+0000 7fd859600700 1 -- 172.21.15.3:0/444889260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd85004d540 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.545+0000 7fd85be00700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fd844039290 0x7fd84403b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.546+0000 7fd85be00700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fd844039290 0x7fd84403b750 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd848006fd0 tx=0x7fd848006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.550+0000 7fd859600700 1 -- 172.21.15.3:0/444889260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd85c10b370 con 0x7fd85c07ce50 2024-03-21T08:18:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.787+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd85c107d00 con 0x7fd85c07ce50 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.790+0000 7fd859600700 1 -- 172.21.15.3:0/444889260 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd85002ddb0 con 0x7fd85c07ce50 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fd844039290 msgr2=0x7fd84403b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fd844039290 0x7fd84403b750 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd848006fd0 tx=0x7fd848006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 msgr2=0x7fd85c10a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c10a9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd850011a00 tx=0x7fd850011ae0 comp rx=0 tx=0).stop 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 shutdown_connections 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fd844039290 0x7fd84403b750 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 --2- 172.21.15.3:0/444889260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c07ce50 0x7fd85c10a9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 >> 172.21.15.3:0/444889260 conn(0x7fd85c102dc0 msgr2=0x7fd85c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 shutdown_connections 2024-03-21T08:18:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:51.794+0000 7fd861e00700 1 -- 172.21.15.3:0/444889260 wait complete. 2024-03-21T08:18:53.887 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:53 smithi003 ceph-mon[31675]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.641+0000 7f3029a00700 1 Processor -- start 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.641+0000 7f3029a00700 1 -- start start 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.641+0000 7f3029a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f302410a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.641+0000 7f3029a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302407ae50 con 0x7f3024108320 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.642+0000 7f3022a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f302410a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.642+0000 7f3022a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f302410a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53472/0 (socket says 172.21.15.3:53472) 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.642+0000 7f3022a00700 1 -- 172.21.15.3:0/549907453 learned_addr learned my addr 172.21.15.3:0/549907453 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.642+0000 7f3022a00700 1 -- 172.21.15.3:0/549907453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302410ac50 con 0x7f3024108320 2024-03-21T08:18:54.561 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.643+0000 7f3022a00700 1 --2- 172.21.15.3:0/549907453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f302410a710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30140098a0 tx=0x7f3014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c80db94ad9f2426 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.643+0000 7f3021600700 1 -- 172.21.15.3:0/549907453 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3014004160 con 0x7f3024108320 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.643+0000 7f3021600700 1 -- 172.21.15.3:0/549907453 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3014004300 con 0x7f3024108320 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.643+0000 7f3021600700 1 -- 172.21.15.3:0/549907453 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3014011a50 con 0x7f3024108320 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.644+0000 7f3029a00700 1 -- 172.21.15.3:0/549907453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 msgr2=0x7f302410a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.644+0000 7f3029a00700 1 --2- 172.21.15.3:0/549907453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f302410a710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30140098a0 tx=0x7f3014009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.644+0000 7f3029a00700 1 -- 172.21.15.3:0/549907453 shutdown_connections 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.644+0000 7f3029a00700 1 --2- 172.21.15.3:0/549907453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f302410a710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.644+0000 7f3029a00700 1 -- 172.21.15.3:0/549907453 >> 172.21.15.3:0/549907453 conn(0x7f3024101e30 msgr2=0x7f3024104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.644+0000 7f3029a00700 1 -- 172.21.15.3:0/549907453 shutdown_connections 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.644+0000 7f3029a00700 1 -- 172.21.15.3:0/549907453 wait complete. 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.645+0000 7f3029a00700 1 Processor -- start 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.645+0000 7f3029a00700 1 -- start start 2024-03-21T08:18:54.562 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.646+0000 7f3029a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f3024107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.646+0000 7f3029a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302410bc50 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.646+0000 7f3022a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f3024107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.646+0000 7f3022a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f3024107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53488/0 (socket says 172.21.15.3:53488) 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.646+0000 7f3022a00700 1 -- 172.21.15.3:0/1434268162 learned_addr learned my addr 172.21.15.3:0/1434268162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.646+0000 7f3022a00700 1 -- 172.21.15.3:0/1434268162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30241080e0 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.647+0000 7f3022a00700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f3024107ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3014011a00 tx=0x7f3014011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.647+0000 7f301b400700 1 -- 172.21.15.3:0/1434268162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3014003af0 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.647+0000 7f301b400700 1 -- 172.21.15.3:0/1434268162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f301401a8e0 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.647+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30241062d0 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.647+0000 7f301b400700 1 -- 172.21.15.3:0/1434268162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3014024840 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.647+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3024106810 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.648+0000 7f301b400700 1 -- 172.21.15.3:0/1434268162 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f3014022020 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.649+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30240637c0 con 0x7f3024108320 2024-03-21T08:18:54.563 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.649+0000 7f301b400700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f30080391d0 0x7f300803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.649+0000 7f301b400700 1 -- 172.21.15.3:0/1434268162 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f301404d540 con 0x7f3024108320 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.649+0000 7f3022000700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f30080391d0 0x7f300803b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.650+0000 7f3022000700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f30080391d0 0x7f300803b690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3010006fd0 tx=0x7f3010006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.655+0000 7f301b400700 1 -- 172.21.15.3:0/1434268162 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f301402ddb0 con 0x7f3024108320 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.883+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f3024106b50 con 0x7f3024108320 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.894+0000 7f301b400700 1 -- 172.21.15.3:0/1434268162 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f3024106b50 con 0x7f3024108320 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.897+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f30080391d0 msgr2=0x7f300803b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f30080391d0 0x7f300803b690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3010006fd0 tx=0x7f3010006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 msgr2=0x7f3024107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f3024107ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3014011a00 tx=0x7f3014011ae0 comp rx=0 tx=0).stop 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 shutdown_connections 2024-03-21T08:18:54.564 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f30080391d0 0x7f300803b690 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:54.565 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 --2- 172.21.15.3:0/1434268162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3024108320 0x7f3024107ba0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:54.565 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 >> 172.21.15.3:0/1434268162 conn(0x7f3024101e30 msgr2=0x7f3024102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:54.565 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 shutdown_connections 2024-03-21T08:18:54.565 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:53.898+0000 7f3029a00700 1 -- 172.21.15.3:0/1434268162 wait complete. 2024-03-21T08:18:54.565 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling the dashboard module... 2024-03-21T08:18:55.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:54 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1434268162' entity='client.admin' 2024-03-21T08:18:56.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:55 smithi003 ceph-mon[31675]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:18:56.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:55 smithi003 ceph-mon[31675]: from='mgr.14120 172.21.15.3:0/3280818683' entity='mgr.x' 2024-03-21T08:18:56.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:55 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4112219890' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T08:18:56.230 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: ignoring --setuser ceph since I am not root 2024-03-21T08:18:56.230 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: ignoring --setgroup ceph since I am not root 2024-03-21T08:18:57.110 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:56.791+0000 7f471e972200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:18:57.110 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:56.918+0000 7f471e972200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:18:57.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:56 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4112219890' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T08:18:57.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:56 smithi003 ceph-mon[31675]: mgrmap e9: x(active, since 25s) 2024-03-21T08:18:57.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.218+0000 7f7772c00700 1 Processor -- start 2024-03-21T08:18:57.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.218+0000 7f7772c00700 1 -- start start 2024-03-21T08:18:57.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.219+0000 7f7772c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c107d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.219+0000 7f7772c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c108310 con 0x7f776c107920 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.219+0000 7f776be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c107d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.219+0000 7f776be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c107d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53492/0 (socket says 172.21.15.3:53492) 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.219+0000 7f776be00700 1 -- 172.21.15.3:0/2045014368 learned_addr learned my addr 172.21.15.3:0/2045014368 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.220+0000 7f776be00700 1 -- 172.21.15.3:0/2045014368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c108b20 con 0x7f776c107920 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.220+0000 7f776be00700 1 --2- 172.21.15.3:0/2045014368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c107d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098a0 tx=0x7f775c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1448d88524085fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.220+0000 7f776b400700 1 -- 172.21.15.3:0/2045014368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f775c004160 con 0x7f776c107920 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.220+0000 7f776b400700 1 -- 172.21.15.3:0/2045014368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f775c004300 con 0x7f776c107920 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.220+0000 7f776b400700 1 -- 172.21.15.3:0/2045014368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f775c011a50 con 0x7f776c107920 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.221+0000 7f7772c00700 1 -- 172.21.15.3:0/2045014368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 msgr2=0x7f776c107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.221+0000 7f7772c00700 1 --2- 172.21.15.3:0/2045014368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c107d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098a0 tx=0x7f775c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:57.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.222+0000 7f7772c00700 1 -- 172.21.15.3:0/2045014368 shutdown_connections 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.222+0000 7f7772c00700 1 --2- 172.21.15.3:0/2045014368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c107d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.222+0000 7f7772c00700 1 -- 172.21.15.3:0/2045014368 >> 172.21.15.3:0/2045014368 conn(0x7f776c102fa0 msgr2=0x7f776c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.222+0000 7f7772c00700 1 -- 172.21.15.3:0/2045014368 shutdown_connections 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.222+0000 7f7772c00700 1 -- 172.21.15.3:0/2045014368 wait complete. 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.224+0000 7f7772c00700 1 Processor -- start 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.224+0000 7f7772c00700 1 -- start start 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.224+0000 7f7772c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c07cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.224+0000 7f7772c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c10bed0 con 0x7f776c107920 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.224+0000 7f776be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c07cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.224+0000 7f776be00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c07cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53494/0 (socket says 172.21.15.3:53494) 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.224+0000 7f776be00700 1 -- 172.21.15.3:0/4112219890 learned_addr learned my addr 172.21.15.3:0/4112219890 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.225+0000 7f776be00700 1 -- 172.21.15.3:0/4112219890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c07d090 con 0x7f776c107920 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.225+0000 7f776be00700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c07cb50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f775c011710 tx=0x7f775c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:57.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.225+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f775c0039c0 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.225+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f775c003b60 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.225+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c07b280 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.226+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f775c01a690 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.226+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c07b7c0 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.227+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f775c01a940 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.227+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c0637c0 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.227+0000 7f7769600700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f7750039240 0x7f775003b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.227+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f775c04d730 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.228+0000 7f7763400700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f7750039240 0x7f775003b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.228+0000 7f7763400700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f7750039240 0x7f775003b700 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7754006fd0 tx=0x7f7754006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.237+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f775c019c70 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.535+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f776c07ba40 con 0x7f776c107920 2024-03-21T08:18:57.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.961+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f775c016df0 con 0x7f776c107920 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.964+0000 7f7769600700 1 -- 172.21.15.3:0/4112219890 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f775c02d8c0 con 0x7f776c107920 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f7750039240 msgr2=0x7f775003b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f7750039240 0x7f775003b700 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7754006fd0 tx=0x7f7754006e40 comp rx=0 tx=0).stop 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 msgr2=0x7f776c07cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c07cb50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f775c011710 tx=0x7f775c011ad0 comp rx=0 tx=0).stop 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 shutdown_connections 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f7750039240 0x7f775003b700 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 --2- 172.21.15.3:0/4112219890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f776c107920 0x7f776c07cb50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.967+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 >> 172.21.15.3:0/4112219890 conn(0x7f776c102fa0 msgr2=0x7f776c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.968+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 shutdown_connections 2024-03-21T08:18:57.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:55.968+0000 7f7772c00700 1 -- 172.21.15.3:0/4112219890 wait complete. 2024-03-21T08:18:57.885 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:57 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:57.759+0000 7f471e972200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:18:58.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:57 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:57.888+0000 7f471e972200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:18:58.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:58.075+0000 7f471e972200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:18:58.498 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:58.213+0000 7f471e972200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:18:58.498 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:58.328+0000 7f471e972200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:18:58.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:18:58 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/632597470' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T08:18:58.793 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:58.679+0000 7f471e972200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:18:59.075 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:58.827+0000 7f471e972200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:18:59.076 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:58.943+0000 7f471e972200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:18:59.076 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:59.072+0000 7f471e972200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:18:59.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:18:59.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.190+0000 7f5e1a000700 1 Processor -- start 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.190+0000 7f5e1a000700 1 -- start start 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.190+0000 7f5e1a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e14106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.191+0000 7f5e1a000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1407ae10 con 0x7f5e14106160 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.191+0000 7f5e13400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e14106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:59.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.191+0000 7f5e13400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e14106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53528/0 (socket says 172.21.15.3:53528) 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.191+0000 7f5e13400700 1 -- 172.21.15.3:0/1340998402 learned_addr learned my addr 172.21.15.3:0/1340998402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.192+0000 7f5e13400700 1 -- 172.21.15.3:0/1340998402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e14106ac0 con 0x7f5e14106160 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.192+0000 7f5e13400700 1 --2- 172.21.15.3:0/1340998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e14106580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e040098a0 tx=0x7f5e04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e06083b8421b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.192+0000 7f5e12000700 1 -- 172.21.15.3:0/1340998402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e04004160 con 0x7f5e14106160 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.192+0000 7f5e12000700 1 -- 172.21.15.3:0/1340998402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e04004300 con 0x7f5e14106160 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.192+0000 7f5e12000700 1 -- 172.21.15.3:0/1340998402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e04011a50 con 0x7f5e14106160 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.193+0000 7f5e1a000700 1 -- 172.21.15.3:0/1340998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 msgr2=0x7f5e14106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.193+0000 7f5e1a000700 1 --2- 172.21.15.3:0/1340998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e14106580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e040098a0 tx=0x7f5e04009bb0 comp rx=0 tx=0).stop 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.193+0000 7f5e1a000700 1 -- 172.21.15.3:0/1340998402 shutdown_connections 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.193+0000 7f5e1a000700 1 --2- 172.21.15.3:0/1340998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e14106580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.193+0000 7f5e1a000700 1 -- 172.21.15.3:0/1340998402 >> 172.21.15.3:0/1340998402 conn(0x7f5e14101de0 msgr2=0x7f5e14104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.193+0000 7f5e1a000700 1 -- 172.21.15.3:0/1340998402 shutdown_connections 2024-03-21T08:18:59.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.193+0000 7f5e1a000700 1 -- 172.21.15.3:0/1340998402 wait complete. 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.194+0000 7f5e1a000700 1 Processor -- start 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.195+0000 7f5e1a000700 1 -- start start 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.195+0000 7f5e1a000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e1419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.195+0000 7f5e1a000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14107b30 con 0x7f5e14106160 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.195+0000 7f5e13400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e1419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.195+0000 7f5e13400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e1419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53530/0 (socket says 172.21.15.3:53530) 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.195+0000 7f5e13400700 1 -- 172.21.15.3:0/632597470 learned_addr learned my addr 172.21.15.3:0/632597470 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.195+0000 7f5e13400700 1 -- 172.21.15.3:0/632597470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1419ded0 con 0x7f5e14106160 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.196+0000 7f5e13400700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e1419d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e04011a00 tx=0x7f5e04011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.196+0000 7f5e0be00700 1 -- 172.21.15.3:0/632597470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e04003af0 con 0x7f5e14106160 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.196+0000 7f5e0be00700 1 -- 172.21.15.3:0/632597470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e0401a8e0 con 0x7f5e14106160 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.196+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1419e160 con 0x7f5e14106160 2024-03-21T08:18:59.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.196+0000 7f5e0be00700 1 -- 172.21.15.3:0/632597470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e04024840 con 0x7f5e14106160 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.196+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e141a0e50 con 0x7f5e14106160 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.197+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e140637c0 con 0x7f5e14106160 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.197+0000 7f5e0be00700 1 -- 172.21.15.3:0/632597470 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f5e04022020 con 0x7f5e14106160 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.198+0000 7f5e0be00700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 0x7f5df803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.198+0000 7f5e0be00700 1 -- 172.21.15.3:0/632597470 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5e0404d5f0 con 0x7f5e14106160 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.198+0000 7f5e12a00700 1 -- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 msgr2=0x7f5df803b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2168061035 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.198+0000 7f5e12a00700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 0x7f5df803b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.208+0000 7f5e0be00700 1 -- 172.21.15.3:0/632597470 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e0400b3e0 con 0x7f5e14106160 2024-03-21T08:18:59.121 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.399+0000 7f5e12a00700 1 -- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 msgr2=0x7f5df803b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2168061035 2024-03-21T08:18:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.399+0000 7f5e12a00700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 0x7f5df803b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:18:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.495+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5e1402dc80 con 0x7f5e14106160 2024-03-21T08:18:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.496+0000 7f5e0be00700 1 -- 172.21.15.3:0/632597470 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5e04016260 con 0x7f5e14106160 2024-03-21T08:18:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 msgr2=0x7f5df803b6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:18:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 0x7f5df803b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 msgr2=0x7f5e1419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:18:59.122 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e1419d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e04011a00 tx=0x7f5e04011ae0 comp rx=0 tx=0).stop 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 shutdown_connections 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7f5df8039230 0x7f5df803b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 --2- 172.21.15.3:0/632597470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e14106160 0x7f5e1419d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.499+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 >> 172.21.15.3:0/632597470 conn(0x7f5e14101de0 msgr2=0x7f5e14194570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.500+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 shutdown_connections 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:58.500+0000 7f5e1a000700 1 -- 172.21.15.3:0/632597470 wait complete. 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-03-21T08:18:59.123 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 9... 2024-03-21T08:18:59.418 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:18:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:18:59.302+0000 7f471e972200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:19:00.927 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:00.637+0000 7f471e972200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:19:00.927 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:00.815+0000 7f471e972200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:19:01.373 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:01.247+0000 7f471e972200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:19:01.638 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:01.369+0000 7f471e972200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:19:01.893 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:01.635+0000 7f471e972200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:19:02.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:01.890+0000 7f471e972200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:19:02.608 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:02.280+0000 7f471e972200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:19:02.608 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:02.605+0000 7f471e972200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:19:03.426 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:02.925+0000 7f471e972200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:19:04.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:03.999+0000 7f471e972200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:19:05.156 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:04.723+0000 7f471e972200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:19:05.157 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:04.859+0000 7f471e972200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:19:05.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:19:05.153+0000 7f471e972200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:19:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: Active manager daemon x restarted 2024-03-21T08:19:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: Activating manager daemon x 2024-03-21T08:19:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T08:19:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: mgrmap e10: x(active, starting, since 0.00723613s) 2024-03-21T08:19:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:19:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:19:05.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:19:05.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:19:05.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:19:05.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:05 smithi003 ceph-mon[31675]: Manager daemon x is now available 2024-03-21T08:19:06.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:19:06.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:19:06.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:19:06.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:06 smithi003 ceph-mon[31675]: mgrmap e11: x(active, since 1.00889s) 2024-03-21T08:19:06.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:19:06.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T08:19:06.849 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T08:19:06.849 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:19:06.849 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.688+0000 7fef20a00700 1 Processor -- start 2024-03-21T08:19:06.849 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.688+0000 7fef20a00700 1 -- start start 2024-03-21T08:19:06.850 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.689+0000 7fef20a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:06.850 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.689+0000 7fef20a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c10a570 con 0x7fef1c109b80 2024-03-21T08:19:06.850 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.689+0000 7fef19e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:06.850 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.689+0000 7fef19e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c109fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53534/0 (socket says 172.21.15.3:53534) 2024-03-21T08:19:06.851 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.689+0000 7fef19e00700 1 -- 172.21.15.3:0/2795053250 learned_addr learned my addr 172.21.15.3:0/2795053250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:06.851 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.690+0000 7fef19e00700 1 -- 172.21.15.3:0/2795053250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c10ace0 con 0x7fef1c109b80 2024-03-21T08:19:06.851 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.690+0000 7fef19e00700 1 --2- 172.21.15.3:0/2795053250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c109fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0098a0 tx=0x7fef0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1daa9982e29ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:06.851 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.690+0000 7fef18a00700 1 -- 172.21.15.3:0/2795053250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef0c004160 con 0x7fef1c109b80 2024-03-21T08:19:06.851 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.690+0000 7fef18a00700 1 -- 172.21.15.3:0/2795053250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef0c004300 con 0x7fef1c109b80 2024-03-21T08:19:06.852 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.690+0000 7fef18a00700 1 -- 172.21.15.3:0/2795053250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef0c011a50 con 0x7fef1c109b80 2024-03-21T08:19:06.852 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.691+0000 7fef20a00700 1 -- 172.21.15.3:0/2795053250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 msgr2=0x7fef1c109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:06.853 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.691+0000 7fef20a00700 1 --2- 172.21.15.3:0/2795053250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c109fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0098a0 tx=0x7fef0c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:19:06.853 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.691+0000 7fef20a00700 1 -- 172.21.15.3:0/2795053250 shutdown_connections 2024-03-21T08:19:06.853 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.691+0000 7fef20a00700 1 --2- 172.21.15.3:0/2795053250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c109fa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:06.853 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.691+0000 7fef20a00700 1 -- 172.21.15.3:0/2795053250 >> 172.21.15.3:0/2795053250 conn(0x7fef1c1054a0 msgr2=0x7fef1c107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:06.854 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.691+0000 7fef20a00700 1 -- 172.21.15.3:0/2795053250 shutdown_connections 2024-03-21T08:19:06.854 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.691+0000 7fef20a00700 1 -- 172.21.15.3:0/2795053250 wait complete. 2024-03-21T08:19:06.854 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.692+0000 7fef20a00700 1 Processor -- start 2024-03-21T08:19:06.854 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.692+0000 7fef20a00700 1 -- start start 2024-03-21T08:19:06.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.693+0000 7fef20a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.693+0000 7fef20a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c10e090 con 0x7fef1c109b80 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.693+0000 7fef19e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.693+0000 7fef19e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53540/0 (socket says 172.21.15.3:53540) 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.693+0000 7fef19e00700 1 -- 172.21.15.3:0/4166226182 learned_addr learned my addr 172.21.15.3:0/4166226182 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.693+0000 7fef19e00700 1 -- 172.21.15.3:0/4166226182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c19de90 con 0x7fef1c109b80 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.694+0000 7fef19e00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c19d950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef0c011710 tx=0x7fef0c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.694+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef0c0039c0 con 0x7fef1c109b80 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.694+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef0c003b60 con 0x7fef1c109b80 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.694+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c19e120 con 0x7fef1c109b80 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.694+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef0c01a690 con 0x7fef1c109b80 2024-03-21T08:19:06.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.694+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c1a0e10 con 0x7fef1c109b80 2024-03-21T08:19:06.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.695+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fef0c024430 con 0x7fef1c109b80 2024-03-21T08:19:06.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.696+0000 7fef12a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:06.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.696+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 --> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef0003be70 con 0x7fef000392a0 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.696+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fef0c04d8e0 con 0x7fef1c109b80 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.696+0000 7fef19400700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 msgr2=0x7fef0003b760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2168061035 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.696+0000 7fef19400700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.897+0000 7fef19400700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 msgr2=0x7fef0003b760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2168061035 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:18:59.897+0000 7fef19400700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:00.298+0000 7fef19400700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 msgr2=0x7fef0003b760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2168061035 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:00.298+0000 7fef19400700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:01.099+0000 7fef19400700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 msgr2=0x7fef0003b760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2168061035 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:01.099+0000 7fef19400700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:02.701+0000 7fef19400700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 msgr2=0x7fef0003b760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2168061035 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:02.701+0000 7fef19400700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T08:19:06.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:05.161+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 10) v1 ==== 46074+0+0 (secure 0 0 0) 0x7fef0c02d770 con 0x7fef1c109b80 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:05.161+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 msgr2=0x7fef0003b760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:05.161+0000 7fef12a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.162+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 11) v1 ==== 46201+0+0 (secure 0 0 0) 0x7fef0c04f340 con 0x7fef1c109b80 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.162+0000 7fef12a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef0003ce70 0x7fef0003f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.162+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef0c02ee50 con 0x7fef0003ce70 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.164+0000 7fef19400700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef0003ce70 0x7fef0003f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.165+0000 7fef19400700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef0003ce70 0x7fef0003f260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fef08003a10 tx=0x7fef08009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.166+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fef0c02ee50 con 0x7fef0003ce70 2024-03-21T08:19:06.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.173+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fef1c0637c0 con 0x7fef0003ce70 2024-03-21T08:19:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef12a00700 1 -- 172.21.15.3:0/4166226182 <== mgr.14152 v2:172.21.15.3:6800/1520535032 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fef1c0637c0 con 0x7fef0003ce70 2024-03-21T08:19:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef0003ce70 msgr2=0x7fef0003f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef0003ce70 0x7fef0003f260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fef08003a10 tx=0x7fef08009270 comp rx=0 tx=0).stop 2024-03-21T08:19:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 msgr2=0x7fef1c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c19d950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef0c011710 tx=0x7fef0c011ad0 comp rx=0 tx=0).stop 2024-03-21T08:19:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 shutdown_connections 2024-03-21T08:19:06.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef0003ce70 0x7fef0003f260 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef1c109b80 0x7fef1c19d950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 --2- 172.21.15.3:0/4166226182 >> [v2:172.21.15.3:6800/2168061035,v1:172.21.15.3:6801/2168061035] conn(0x7fef000392a0 0x7fef0003b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.174+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 >> 172.21.15.3:0/4166226182 conn(0x7fef1c1054a0 msgr2=0x7fef1c106ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.175+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 shutdown_connections 2024-03-21T08:19:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:06.175+0000 7fef20a00700 1 -- 172.21.15.3:0/4166226182 wait complete. 2024-03-21T08:19:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 9 is available 2024-03-21T08:19:06.864 INFO:teuthology.orchestra.run.smithi003.stdout:Generating a dashboard self-signed certificate... 2024-03-21T08:19:07.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:07 smithi003 ceph-mon[31675]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T08:19:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:07 smithi003 ceph-mon[31675]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T08:19:08.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:08 smithi003 ceph-mon[31675]: [21/Mar/2024:08:19:07] ENGINE Bus STARTING 2024-03-21T08:19:08.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:08 smithi003 ceph-mon[31675]: mgrmap e12: x(active, since 2s) 2024-03-21T08:19:08.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:08 smithi003 ceph-mon[31675]: [21/Mar/2024:08:19:07] ENGINE Serving on http://172.21.15.3:8765 2024-03-21T08:19:08.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:08 smithi003 ceph-mon[31675]: [21/Mar/2024:08:19:07] ENGINE Serving on https://172.21.15.3:7150 2024-03-21T08:19:08.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:08 smithi003 ceph-mon[31675]: [21/Mar/2024:08:19:07] ENGINE Bus STARTED 2024-03-21T08:19:08.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:08 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:09.281 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T08:19:09.282 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.207+0000 7fdae3800700 1 Processor -- start 2024-03-21T08:19:09.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.207+0000 7fdae3800700 1 -- start start 2024-03-21T08:19:09.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.208+0000 7fdae3800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:09.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.208+0000 7fdae3800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc07b800 con 0x7fdadc07ce10 2024-03-21T08:19:09.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.208+0000 7fdae0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:09.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.208+0000 7fdae0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc07b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34972/0 (socket says 172.21.15.3:34972) 2024-03-21T08:19:09.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.208+0000 7fdae0e00700 1 -- 172.21.15.3:0/88207746 learned_addr learned my addr 172.21.15.3:0/88207746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:09.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.209+0000 7fdae0e00700 1 -- 172.21.15.3:0/88207746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc07b980 con 0x7fdadc07ce10 2024-03-21T08:19:09.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.209+0000 7fdae0e00700 1 --2- 172.21.15.3:0/88207746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc07b2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdad00098a0 tx=0x7fdad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3559ae08511bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:09.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.210+0000 7fdadb400700 1 -- 172.21.15.3:0/88207746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad0004160 con 0x7fdadc07ce10 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.210+0000 7fdadb400700 1 -- 172.21.15.3:0/88207746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdad0004300 con 0x7fdadc07ce10 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.210+0000 7fdadb400700 1 -- 172.21.15.3:0/88207746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad0011a50 con 0x7fdadc07ce10 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.210+0000 7fdae3800700 1 -- 172.21.15.3:0/88207746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 msgr2=0x7fdadc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.210+0000 7fdae3800700 1 --2- 172.21.15.3:0/88207746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc07b2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdad00098a0 tx=0x7fdad0009bb0 comp rx=0 tx=0).stop 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.211+0000 7fdae3800700 1 -- 172.21.15.3:0/88207746 shutdown_connections 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.211+0000 7fdae3800700 1 --2- 172.21.15.3:0/88207746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc07b2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.211+0000 7fdae3800700 1 -- 172.21.15.3:0/88207746 >> 172.21.15.3:0/88207746 conn(0x7fdadc102f50 msgr2=0x7fdadc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.211+0000 7fdae3800700 1 -- 172.21.15.3:0/88207746 shutdown_connections 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.211+0000 7fdae3800700 1 -- 172.21.15.3:0/88207746 wait complete. 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.212+0000 7fdae3800700 1 Processor -- start 2024-03-21T08:19:09.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.212+0000 7fdae3800700 1 -- start start 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.212+0000 7fdae3800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.212+0000 7fdae3800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc10de10 con 0x7fdadc07ce10 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.213+0000 7fdae0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.213+0000 7fdae0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34978/0 (socket says 172.21.15.3:34978) 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.213+0000 7fdae0e00700 1 -- 172.21.15.3:0/101340618 learned_addr learned my addr 172.21.15.3:0/101340618 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.213+0000 7fdae0e00700 1 -- 172.21.15.3:0/101340618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc19dee0 con 0x7fdadc07ce10 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.214+0000 7fdae0e00700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc19d9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009130 tx=0x7fdad0004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.214+0000 7fdad9600700 1 -- 172.21.15.3:0/101340618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad0003af0 con 0x7fdadc07ce10 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.214+0000 7fdad9600700 1 -- 172.21.15.3:0/101340618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdad001a8e0 con 0x7fdadc07ce10 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.214+0000 7fdad9600700 1 -- 172.21.15.3:0/101340618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad00248b0 con 0x7fdadc07ce10 2024-03-21T08:19:09.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.214+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc19e170 con 0x7fdadc07ce10 2024-03-21T08:19:09.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.215+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc1a0e60 con 0x7fdadc07ce10 2024-03-21T08:19:09.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.216+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc0637c0 con 0x7fdadc07ce10 2024-03-21T08:19:09.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.216+0000 7fdad9600700 1 -- 172.21.15.3:0/101340618 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fdad0022020 con 0x7fdadc07ce10 2024-03-21T08:19:09.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.217+0000 7fdad9600700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdac4039230 0x7fdac403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:09.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.217+0000 7fdad9600700 1 -- 172.21.15.3:0/101340618 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdad004d650 con 0x7fdadc07ce10 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.217+0000 7fdadbe00700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdac4039230 0x7fdac403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.218+0000 7fdadbe00700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdac4039230 0x7fdac403b6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006fd0 tx=0x7fdac8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.223+0000 7fdad9600700 1 -- 172.21.15.3:0/101340618 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdad0016260 con 0x7fdadc07ce10 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.459+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fdadc1040f0 con 0x7fdac4039230 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.525+0000 7fdad9600700 1 -- 172.21.15.3:0/101340618 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fdadc1040f0 con 0x7fdac4039230 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.528+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdac4039230 msgr2=0x7fdac403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.528+0000 7fdae3800700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdac4039230 0x7fdac403b6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006fd0 tx=0x7fdac8006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.528+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 msgr2=0x7fdadc19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:09.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.528+0000 7fdae3800700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc19d9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009130 tx=0x7fdad0004620 comp rx=0 tx=0).stop 2024-03-21T08:19:09.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.529+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 shutdown_connections 2024-03-21T08:19:09.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.529+0000 7fdae3800700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdac4039230 0x7fdac403b6f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:09.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.529+0000 7fdae3800700 1 --2- 172.21.15.3:0/101340618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdadc07ce10 0x7fdadc19d9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:09.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.529+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 >> 172.21.15.3:0/101340618 conn(0x7fdadc102f50 msgr2=0x7fdadc1039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:09.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.529+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 shutdown_connections 2024-03-21T08:19:09.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:08.529+0000 7fdae3800700 1 -- 172.21.15.3:0/101340618 wait complete. 2024-03-21T08:19:09.292 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial admin user... 2024-03-21T08:19:09.831 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:09 smithi003 ceph-mon[31675]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:19:09.831 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:09.831 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:11.126 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:11 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:11.126 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:11 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:11.140 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$eM.pOyiKAB.vCfN0qch6pOcJcC0soir54OsA.yN4gYCN1.IO.MJrG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711009150, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T08:19:11.140 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.885+0000 7f57f4800700 1 Processor -- start 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.886+0000 7f57f4800700 1 -- start start 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.886+0000 7f57f4800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.886+0000 7f57f4800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec1082a0 con 0x7f57ec1078b0 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.887+0000 7f57f1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.887+0000 7f57f1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54306/0 (socket says 172.21.15.3:54306) 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.887+0000 7f57f1e00700 1 -- 172.21.15.3:0/1507569270 learned_addr learned my addr 172.21.15.3:0/1507569270 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.887+0000 7f57f1e00700 1 -- 172.21.15.3:0/1507569270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec108a60 con 0x7f57ec1078b0 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.888+0000 7f57f1e00700 1 --2- 172.21.15.3:0/1507569270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec107cd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f57e00098a0 tx=0x7f57e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19763864db66f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.888+0000 7f57f0a00700 1 -- 172.21.15.3:0/1507569270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57e0004160 con 0x7f57ec1078b0 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.888+0000 7f57f0a00700 1 -- 172.21.15.3:0/1507569270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57e0004300 con 0x7f57ec1078b0 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.888+0000 7f57f0a00700 1 -- 172.21.15.3:0/1507569270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57e0011a50 con 0x7f57ec1078b0 2024-03-21T08:19:11.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.889+0000 7f57f4800700 1 -- 172.21.15.3:0/1507569270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 msgr2=0x7f57ec107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.889+0000 7f57f4800700 1 --2- 172.21.15.3:0/1507569270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec107cd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f57e00098a0 tx=0x7f57e0009bb0 comp rx=0 tx=0).stop 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.889+0000 7f57f4800700 1 -- 172.21.15.3:0/1507569270 shutdown_connections 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.889+0000 7f57f4800700 1 --2- 172.21.15.3:0/1507569270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec107cd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.889+0000 7f57f4800700 1 -- 172.21.15.3:0/1507569270 >> 172.21.15.3:0/1507569270 conn(0x7f57ec102f50 msgr2=0x7f57ec105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.889+0000 7f57f4800700 1 -- 172.21.15.3:0/1507569270 shutdown_connections 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.889+0000 7f57f4800700 1 -- 172.21.15.3:0/1507569270 wait complete. 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.890+0000 7f57f4800700 1 Processor -- start 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.890+0000 7f57f4800700 1 -- start start 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.890+0000 7f57f4800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.890+0000 7f57f4800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec10be10 con 0x7f57ec1078b0 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.890+0000 7f57f1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:11.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.891+0000 7f57f1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54310/0 (socket says 172.21.15.3:54310) 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.891+0000 7f57f1e00700 1 -- 172.21.15.3:0/3078692307 learned_addr learned my addr 172.21.15.3:0/3078692307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.891+0000 7f57f1e00700 1 -- 172.21.15.3:0/3078692307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec19def0 con 0x7f57ec1078b0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.891+0000 7f57f1e00700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec19d9b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57e0011a20 tx=0x7f57e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.891+0000 7f57e6a00700 1 -- 172.21.15.3:0/3078692307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57e0003af0 con 0x7f57ec1078b0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.891+0000 7f57e6a00700 1 -- 172.21.15.3:0/3078692307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57e001a8e0 con 0x7f57ec1078b0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.891+0000 7f57e6a00700 1 -- 172.21.15.3:0/3078692307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57e0024860 con 0x7f57ec1078b0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.892+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec19e180 con 0x7f57ec1078b0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.892+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec1a0e70 con 0x7f57ec1078b0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.893+0000 7f57e6a00700 1 -- 172.21.15.3:0/3078692307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f57e0022020 con 0x7f57ec1078b0 2024-03-21T08:19:11.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.893+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec0637c0 con 0x7f57ec1078b0 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.893+0000 7f57e6a00700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f57d40391d0 0x7f57d403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.893+0000 7f57e6a00700 1 -- 172.21.15.3:0/3078692307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f57e004d580 con 0x7f57ec1078b0 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.894+0000 7f57f1400700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f57d40391d0 0x7f57d403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.895+0000 7f57f1400700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f57d40391d0 0x7f57d403b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57dc006fd0 tx=0x7f57dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:09.899+0000 7f57e6a00700 1 -- 172.21.15.3:0/3078692307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f57e002ad60 con 0x7f57ec1078b0 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.160+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f57ec19e660 con 0x7f57d40391d0 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.436+0000 7f57e6a00700 1 -- 172.21.15.3:0/3078692307 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f57ec19e660 con 0x7f57d40391d0 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f57d40391d0 msgr2=0x7f57d403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f57d40391d0 0x7f57d403b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57dc006fd0 tx=0x7f57dc006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 msgr2=0x7f57ec19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec19d9b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57e0011a20 tx=0x7f57e0011b00 comp rx=0 tx=0).stop 2024-03-21T08:19:11.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 shutdown_connections 2024-03-21T08:19:11.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f57d40391d0 0x7f57d403b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:11.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 --2- 172.21.15.3:0/3078692307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57ec1078b0 0x7f57ec19d9b0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57e0011a20 tx=0x7f57e0011b00 comp rx=0 tx=0).stop 2024-03-21T08:19:11.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.439+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 >> 172.21.15.3:0/3078692307 conn(0x7f57ec102f50 msgr2=0x7f57ec103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:11.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.440+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 shutdown_connections 2024-03-21T08:19:11.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:10.440+0000 7f57f4800700 1 -- 172.21.15.3:0/3078692307 wait complete. 2024-03-21T08:19:11.145 INFO:teuthology.orchestra.run.smithi003.stdout:Fetching dashboard port number... 2024-03-21T08:19:12.311 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:12 smithi003 ceph-mon[31675]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:19:12.311 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:12 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1858804665' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.703+0000 7fea57200700 1 Processor -- start 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.704+0000 7fea57200700 1 -- start start 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.704+0000 7fea57200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea5010a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.704+0000 7fea57200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5007ae50 con 0x7fea50108320 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.705+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea5010a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.705+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea5010a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54312/0 (socket says 172.21.15.3:54312) 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.705+0000 7fea4fe00700 1 -- 172.21.15.3:0/3338500588 learned_addr learned my addr 172.21.15.3:0/3338500588 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.705+0000 7fea4fe00700 1 -- 172.21.15.3:0/3338500588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5010ac50 con 0x7fea50108320 2024-03-21T08:19:12.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.706+0000 7fea4fe00700 1 --2- 172.21.15.3:0/3338500588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea5010a710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea400098a0 tx=0x7fea40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dab6bf1ec05b74 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.706+0000 7fea4ea00700 1 -- 172.21.15.3:0/3338500588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea40004160 con 0x7fea50108320 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.706+0000 7fea4ea00700 1 -- 172.21.15.3:0/3338500588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea40004300 con 0x7fea50108320 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.706+0000 7fea4ea00700 1 -- 172.21.15.3:0/3338500588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea40011a50 con 0x7fea50108320 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.707+0000 7fea57200700 1 -- 172.21.15.3:0/3338500588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 msgr2=0x7fea5010a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.707+0000 7fea57200700 1 --2- 172.21.15.3:0/3338500588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea5010a710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea400098a0 tx=0x7fea40009bb0 comp rx=0 tx=0).stop 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.707+0000 7fea57200700 1 -- 172.21.15.3:0/3338500588 shutdown_connections 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.707+0000 7fea57200700 1 --2- 172.21.15.3:0/3338500588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea5010a710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.707+0000 7fea57200700 1 -- 172.21.15.3:0/3338500588 >> 172.21.15.3:0/3338500588 conn(0x7fea50101e30 msgr2=0x7fea50104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.707+0000 7fea57200700 1 -- 172.21.15.3:0/3338500588 shutdown_connections 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.707+0000 7fea57200700 1 -- 172.21.15.3:0/3338500588 wait complete. 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.709+0000 7fea57200700 1 Processor -- start 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.709+0000 7fea57200700 1 -- start start 2024-03-21T08:19:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.709+0000 7fea57200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea50107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.709+0000 7fea57200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5010bc50 con 0x7fea50108320 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.710+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea50107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.710+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea50107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54328/0 (socket says 172.21.15.3:54328) 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.710+0000 7fea4fe00700 1 -- 172.21.15.3:0/1858804665 learned_addr learned my addr 172.21.15.3:0/1858804665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.710+0000 7fea4fe00700 1 -- 172.21.15.3:0/1858804665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea501080e0 con 0x7fea50108320 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.711+0000 7fea4fe00700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea50107ba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea40011a20 tx=0x7fea40011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.711+0000 7fea4cc00700 1 -- 172.21.15.3:0/1858804665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea400039c0 con 0x7fea50108320 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.711+0000 7fea4cc00700 1 -- 172.21.15.3:0/1858804665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea40003b60 con 0x7fea50108320 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.711+0000 7fea4cc00700 1 -- 172.21.15.3:0/1858804665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea4001a6b0 con 0x7fea50108320 2024-03-21T08:19:12.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.711+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea501062d0 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.711+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea50106810 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.712+0000 7fea4cc00700 1 -- 172.21.15.3:0/1858804665 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fea4001a960 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.713+0000 7fea4cc00700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea340391e0 0x7fea3403b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.713+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea500637c0 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.713+0000 7fea4cc00700 1 -- 172.21.15.3:0/1858804665 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fea4004dee0 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.713+0000 7fea4f400700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea340391e0 0x7fea3403b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.714+0000 7fea4f400700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea340391e0 0x7fea3403b6a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea3c006fd0 tx=0x7fea3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.719+0000 7fea4cc00700 1 -- 172.21.15.3:0/1858804665 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fea40018350 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.951+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fea50106b50 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.952+0000 7fea4cc00700 1 -- 172.21.15.3:0/1858804665 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fea50106b50 con 0x7fea50108320 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.954+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea340391e0 msgr2=0x7fea3403b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.954+0000 7fea57200700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea340391e0 0x7fea3403b6a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea3c006fd0 tx=0x7fea3c006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:12.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 msgr2=0x7fea50107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:12.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea50107ba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea40011a20 tx=0x7fea40011b00 comp rx=0 tx=0).stop 2024-03-21T08:19:12.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 shutdown_connections 2024-03-21T08:19:12.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea340391e0 0x7fea3403b6a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:12.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 --2- 172.21.15.3:0/1858804665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50108320 0x7fea50107ba0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:12.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 >> 172.21.15.3:0/1858804665 conn(0x7fea50101e30 msgr2=0x7fea50102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:12.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 shutdown_connections 2024-03-21T08:19:12.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:11.955+0000 7fea57200700 1 -- 172.21.15.3:0/1858804665 wait complete. 2024-03-21T08:19:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:19:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-03-21T08:19:12.836 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:19:12.836 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-03-21T08:19:12.837 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-03-21T08:19:12.837 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T08:19:12.845 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph Dashboard is now available at: 2024-03-21T08:19:12.845 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:12.845 INFO:teuthology.orchestra.run.smithi003.stdout: URL: https://smithi003.front.sepia.ceph.com:8443/ 2024-03-21T08:19:12.845 INFO:teuthology.orchestra.run.smithi003.stdout: User: admin 2024-03-21T08:19:12.845 INFO:teuthology.orchestra.run.smithi003.stdout: Password: cqc113imai 2024-03-21T08:19:12.845 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:12.846 INFO:teuthology.orchestra.run.smithi003.stdout:Saving cluster configuration to /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config directory 2024-03-21T08:19:12.848 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling autotune for osd_memory_target 2024-03-21T08:19:13.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:13 smithi003 ceph-mon[31675]: mgrmap e13: x(active, since 6s) 2024-03-21T08:19:14.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.956+0000 7fb2f4400700 1 Processor -- start 2024-03-21T08:19:14.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.956+0000 7fb2f4400700 1 -- start start 2024-03-21T08:19:14.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.956+0000 7fb2f4400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec07b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:14.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.956+0000 7fb2f4400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec07b8d0 con 0x7fb2ec07cea0 2024-03-21T08:19:14.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.957+0000 7fb2f1a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec07b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:14.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.957+0000 7fb2f1a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec07b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54330/0 (socket says 172.21.15.3:54330) 2024-03-21T08:19:14.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.957+0000 7fb2f1a00700 1 -- 172.21.15.3:0/2457582463 learned_addr learned my addr 172.21.15.3:0/2457582463 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:14.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.957+0000 7fb2f1a00700 1 -- 172.21.15.3:0/2457582463 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec07ba50 con 0x7fb2ec07cea0 2024-03-21T08:19:14.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.957+0000 7fb2f1a00700 1 --2- 172.21.15.3:0/2457582463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec07b300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00098a0 tx=0x7fb2e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668c6bc07fbc5894 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:14.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.958+0000 7fb2ebe00700 1 -- 172.21.15.3:0/2457582463 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2e0004160 con 0x7fb2ec07cea0 2024-03-21T08:19:14.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.958+0000 7fb2ebe00700 1 -- 172.21.15.3:0/2457582463 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb2e0004300 con 0x7fb2ec07cea0 2024-03-21T08:19:14.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.958+0000 7fb2ebe00700 1 -- 172.21.15.3:0/2457582463 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2e0011a50 con 0x7fb2ec07cea0 2024-03-21T08:19:14.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.958+0000 7fb2f4400700 1 -- 172.21.15.3:0/2457582463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 msgr2=0x7fb2ec07b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:14.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.958+0000 7fb2f4400700 1 --2- 172.21.15.3:0/2457582463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec07b300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00098a0 tx=0x7fb2e0009bb0 comp rx=0 tx=0).stop 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.959+0000 7fb2f4400700 1 -- 172.21.15.3:0/2457582463 shutdown_connections 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.959+0000 7fb2f4400700 1 --2- 172.21.15.3:0/2457582463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec07b300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.959+0000 7fb2f4400700 1 -- 172.21.15.3:0/2457582463 >> 172.21.15.3:0/2457582463 conn(0x7fb2ec102fa0 msgr2=0x7fb2ec1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.959+0000 7fb2f4400700 1 -- 172.21.15.3:0/2457582463 shutdown_connections 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.959+0000 7fb2f4400700 1 -- 172.21.15.3:0/2457582463 wait complete. 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.960+0000 7fb2f4400700 1 Processor -- start 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.960+0000 7fb2f4400700 1 -- start start 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.960+0000 7fb2f4400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.960+0000 7fb2f4400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec10df00 con 0x7fb2ec07cea0 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.961+0000 7fb2f1a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.961+0000 7fb2f1a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54344/0 (socket says 172.21.15.3:54344) 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.961+0000 7fb2f1a00700 1 -- 172.21.15.3:0/1708825578 learned_addr learned my addr 172.21.15.3:0/1708825578 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:14.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.961+0000 7fb2f1a00700 1 -- 172.21.15.3:0/1708825578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec19ded0 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.961+0000 7fb2f1a00700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec19d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0011a20 tx=0x7fb2e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.962+0000 7fb2ea000700 1 -- 172.21.15.3:0/1708825578 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2e0003af0 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.962+0000 7fb2ea000700 1 -- 172.21.15.3:0/1708825578 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb2e001a8e0 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.962+0000 7fb2ea000700 1 -- 172.21.15.3:0/1708825578 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2e0024860 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.962+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec19e160 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.962+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec1a0e50 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.963+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec0637c0 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.963+0000 7fb2ea000700 1 -- 172.21.15.3:0/1708825578 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7fb2e0022020 con 0x7fb2ec07cea0 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.963+0000 7fb2ea000700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb2d4039390 0x7fb2d403b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.964+0000 7fb2ea000700 1 -- 172.21.15.3:0/1708825578 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb2e004d6e0 con 0x7fb2ec07cea0 2024-03-21T08:19:14.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.964+0000 7fb2f1000700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb2d4039390 0x7fb2d403b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:14.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.965+0000 7fb2f1000700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb2d4039390 0x7fb2d403b850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8006fd0 tx=0x7fb2d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:14.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:13.969+0000 7fb2ea000700 1 -- 172.21.15.3:0/1708825578 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2e0016260 con 0x7fb2ec07cea0 2024-03-21T08:19:14.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.196+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fb2ec1a1020 con 0x7fb2ec07cea0 2024-03-21T08:19:14.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.197+0000 7fb2ea000700 1 -- 172.21.15.3:0/1708825578 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fb2e00094f0 con 0x7fb2ec07cea0 2024-03-21T08:19:14.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb2d4039390 msgr2=0x7fb2d403b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb2d4039390 0x7fb2d403b850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8006fd0 tx=0x7fb2d8006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 msgr2=0x7fb2ec19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec19d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0011a20 tx=0x7fb2e0011b00 comp rx=0 tx=0).stop 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 shutdown_connections 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb2d4039390 0x7fb2d403b850 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 --2- 172.21.15.3:0/1708825578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2ec07cea0 0x7fb2ec19d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 >> 172.21.15.3:0/1708825578 conn(0x7fb2ec102fa0 msgr2=0x7fb2ec103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 shutdown_connections 2024-03-21T08:19:14.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:14.200+0000 7fb2f4400700 1 -- 172.21.15.3:0/1708825578 wait complete. 2024-03-21T08:19:16.536 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.494+0000 7f3dfa200700 1 Processor -- start 2024-03-21T08:19:16.536 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.494+0000 7f3dfa200700 1 -- start start 2024-03-21T08:19:16.536 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.494+0000 7f3dfa200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df4109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:16.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.494+0000 7f3dfa200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df410a590 con 0x7f3df4109ba0 2024-03-21T08:19:16.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.495+0000 7f3df3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df4109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:16.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.495+0000 7f3df3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df4109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54360/0 (socket says 172.21.15.3:54360) 2024-03-21T08:19:16.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.495+0000 7f3df3400700 1 -- 172.21.15.3:0/3235058462 learned_addr learned my addr 172.21.15.3:0/3235058462 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:16.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.495+0000 7f3df3400700 1 -- 172.21.15.3:0/3235058462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df410ad00 con 0x7f3df4109ba0 2024-03-21T08:19:16.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.495+0000 7f3df3400700 1 --2- 172.21.15.3:0/3235058462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df4109fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3de40098a0 tx=0x7f3de4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28854055e6ae501 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:16.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.496+0000 7f3df2a00700 1 -- 172.21.15.3:0/3235058462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de4004160 con 0x7f3df4109ba0 2024-03-21T08:19:16.541 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.496+0000 7f3df2a00700 1 -- 172.21.15.3:0/3235058462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3de4004300 con 0x7f3df4109ba0 2024-03-21T08:19:16.541 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.496+0000 7f3df2a00700 1 -- 172.21.15.3:0/3235058462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de4011a50 con 0x7f3df4109ba0 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.497+0000 7f3dfa200700 1 -- 172.21.15.3:0/3235058462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 msgr2=0x7f3df4109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.497+0000 7f3dfa200700 1 --2- 172.21.15.3:0/3235058462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df4109fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3de40098a0 tx=0x7f3de4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.497+0000 7f3dfa200700 1 -- 172.21.15.3:0/3235058462 shutdown_connections 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.497+0000 7f3dfa200700 1 --2- 172.21.15.3:0/3235058462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df4109fc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.497+0000 7f3dfa200700 1 -- 172.21.15.3:0/3235058462 >> 172.21.15.3:0/3235058462 conn(0x7f3df41054a0 msgr2=0x7f3df4107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.497+0000 7f3dfa200700 1 -- 172.21.15.3:0/3235058462 shutdown_connections 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.497+0000 7f3dfa200700 1 -- 172.21.15.3:0/3235058462 wait complete. 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.498+0000 7f3dfa200700 1 Processor -- start 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.498+0000 7f3dfa200700 1 -- start start 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.499+0000 7f3dfa200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.499+0000 7f3dfa200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df410e0b0 con 0x7f3df4109ba0 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.499+0000 7f3df3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:16.542 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.499+0000 7f3df3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54362/0 (socket says 172.21.15.3:54362) 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.499+0000 7f3df3400700 1 -- 172.21.15.3:0/2012686875 learned_addr learned my addr 172.21.15.3:0/2012686875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.500+0000 7f3df3400700 1 -- 172.21.15.3:0/2012686875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df419deb0 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.500+0000 7f3df3400700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df419d970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3de4011710 tx=0x7f3de4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.500+0000 7f3df0c00700 1 -- 172.21.15.3:0/2012686875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de40039c0 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.500+0000 7f3df0c00700 1 -- 172.21.15.3:0/2012686875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3de4003b60 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.500+0000 7f3df0c00700 1 -- 172.21.15.3:0/2012686875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de401a690 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.500+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df419e140 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.501+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41a0e30 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.502+0000 7f3df0c00700 1 -- 172.21.15.3:0/2012686875 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f3de4024430 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.502+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df40637c0 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.502+0000 7f3df0c00700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f3dd8039340 0x7f3dd803b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.502+0000 7f3df0c00700 1 -- 172.21.15.3:0/2012686875 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3de404da10 con 0x7f3df4109ba0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.503+0000 7f3deaa00700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f3dd8039340 0x7f3dd803b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:16.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.503+0000 7f3deaa00700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f3dd8039340 0x7f3dd803b800 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00ad30 tx=0x7f3ddc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.508+0000 7f3df0c00700 1 -- 172.21.15.3:0/2012686875 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3de4016df0 con 0x7f3df4109ba0 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.854+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3df41a1000 con 0x7f3df4109ba0 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.859+0000 7f3df0c00700 1 -- 172.21.15.3:0/2012686875 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3de4024a70 con 0x7f3df4109ba0 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f3dd8039340 msgr2=0x7f3dd803b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f3dd8039340 0x7f3dd803b800 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00ad30 tx=0x7f3ddc0093f0 comp rx=0 tx=0).stop 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 msgr2=0x7f3df419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df419d970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3de4011710 tx=0x7f3de4011ad0 comp rx=0 tx=0).stop 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 shutdown_connections 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f3dd8039340 0x7f3dd803b800 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 --2- 172.21.15.3:0/2012686875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3df4109ba0 0x7f3df419d970 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.862+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 >> 172.21.15.3:0/2012686875 conn(0x7f3df41054a0 msgr2=0x7f3df4106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.863+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 shutdown_connections 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-03-21T08:19:15.863+0000 7f3dfa200700 1 -- 172.21.15.3:0/2012686875 wait complete. 2024-03-21T08:19:16.544 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T08:19:16.545 INFO:teuthology.orchestra.run.smithi003.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T08:19:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:19:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.545 INFO:teuthology.orchestra.run.smithi003.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T08:19:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: ceph telemetry on 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout:For more information see: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:16.546 INFO:teuthology.orchestra.run.smithi003.stdout:Bootstrap complete. 2024-03-21T08:19:16.648 INFO:tasks.cephadm:Fetching config... 2024-03-21T08:19:16.648 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:19:16.648 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T08:19:16.694 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T08:19:16.694 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:19:16.694 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T08:19:16.781 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T08:19:16.782 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:19:16.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-21T08:19:16.921 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T08:19:16.921 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:19:16.922 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T08:19:16.974 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T08:19:16.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnRrEwr8KBU6tmaST4rZPAWoIukpVtovy3cNIB7or967TzDpTHJNJ4zw0jISpRkLuFSoSy0tkw7l77IW3/7DkiPIwRkxh1iH8Q1V1qdbEfKnVnSui81TIdUKCLKfbYlnoP+FRQfg7LfXCoMW/SOUg4KxPibVeL9HbUbEWVjbgGuehA8p5CFtWDT54+uyzJmPFaNE+Ztio4cErXJhkzHw4NtJRCNYRNcdRml6cc6he8LOa83hJRf6iQuoOucZZspFvm9pdxEwxUHJHx524mDVnyAuOguwO/IkFq/Ndc1u7EweUgKenYdy1IjzVG1hQ80acJozJpzBB3o+uWj+tYRmWYN+NjXEc0C0cXxA+8QQ78RHqjUuLBwfl6PNMDscvGCZNg9E7QCYwEPxI5eqq65C4KYuAp3+4vmHjvQb1Au0EtefPKTuTpjOkPkBvbwFht3ZH/xt80D7Odvv/Vue3AoXRjMReQA4tJLaXYknPTG6YW2cqOcWH9CoQh51TP2q5jJjs= ceph-544fc91e-e75b-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T08:19:17.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:16 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2012686875' entity='client.admin' 2024-03-21T08:19:17.099 INFO:teuthology.orchestra.run.smithi003.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnRrEwr8KBU6tmaST4rZPAWoIukpVtovy3cNIB7or967TzDpTHJNJ4zw0jISpRkLuFSoSy0tkw7l77IW3/7DkiPIwRkxh1iH8Q1V1qdbEfKnVnSui81TIdUKCLKfbYlnoP+FRQfg7LfXCoMW/SOUg4KxPibVeL9HbUbEWVjbgGuehA8p5CFtWDT54+uyzJmPFaNE+Ztio4cErXJhkzHw4NtJRCNYRNcdRml6cc6he8LOa83hJRf6iQuoOucZZspFvm9pdxEwxUHJHx524mDVnyAuOguwO/IkFq/Ndc1u7EweUgKenYdy1IjzVG1hQ80acJozJpzBB3o+uWj+tYRmWYN+NjXEc0C0cXxA+8QQ78RHqjUuLBwfl6PNMDscvGCZNg9E7QCYwEPxI5eqq65C4KYuAp3+4vmHjvQb1Au0EtefPKTuTpjOkPkBvbwFht3ZH/xt80D7Odvv/Vue3AoXRjMReQA4tJLaXYknPTG6YW2cqOcWH9CoQh51TP2q5jJjs= ceph-544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:19:17.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnRrEwr8KBU6tmaST4rZPAWoIukpVtovy3cNIB7or967TzDpTHJNJ4zw0jISpRkLuFSoSy0tkw7l77IW3/7DkiPIwRkxh1iH8Q1V1qdbEfKnVnSui81TIdUKCLKfbYlnoP+FRQfg7LfXCoMW/SOUg4KxPibVeL9HbUbEWVjbgGuehA8p5CFtWDT54+uyzJmPFaNE+Ztio4cErXJhkzHw4NtJRCNYRNcdRml6cc6he8LOa83hJRf6iQuoOucZZspFvm9pdxEwxUHJHx524mDVnyAuOguwO/IkFq/Ndc1u7EweUgKenYdy1IjzVG1hQ80acJozJpzBB3o+uWj+tYRmWYN+NjXEc0C0cXxA+8QQ78RHqjUuLBwfl6PNMDscvGCZNg9E7QCYwEPxI5eqq65C4KYuAp3+4vmHjvQb1Au0EtefPKTuTpjOkPkBvbwFht3ZH/xt80D7Odvv/Vue3AoXRjMReQA4tJLaXYknPTG6YW2cqOcWH9CoQh51TP2q5jJjs= ceph-544fc91e-e75b-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T08:19:17.300 INFO:teuthology.orchestra.run.smithi084.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnRrEwr8KBU6tmaST4rZPAWoIukpVtovy3cNIB7or967TzDpTHJNJ4zw0jISpRkLuFSoSy0tkw7l77IW3/7DkiPIwRkxh1iH8Q1V1qdbEfKnVnSui81TIdUKCLKfbYlnoP+FRQfg7LfXCoMW/SOUg4KxPibVeL9HbUbEWVjbgGuehA8p5CFtWDT54+uyzJmPFaNE+Ztio4cErXJhkzHw4NtJRCNYRNcdRml6cc6he8LOa83hJRf6iQuoOucZZspFvm9pdxEwxUHJHx524mDVnyAuOguwO/IkFq/Ndc1u7EweUgKenYdy1IjzVG1hQ80acJozJpzBB3o+uWj+tYRmWYN+NjXEc0C0cXxA+8QQ78RHqjUuLBwfl6PNMDscvGCZNg9E7QCYwEPxI5eqq65C4KYuAp3+4vmHjvQb1Au0EtefPKTuTpjOkPkBvbwFht3ZH/xt80D7Odvv/Vue3AoXRjMReQA4tJLaXYknPTG6YW2cqOcWH9CoQh51TP2q5jJjs= ceph-544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:19:17.357 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnRrEwr8KBU6tmaST4rZPAWoIukpVtovy3cNIB7or967TzDpTHJNJ4zw0jISpRkLuFSoSy0tkw7l77IW3/7DkiPIwRkxh1iH8Q1V1qdbEfKnVnSui81TIdUKCLKfbYlnoP+FRQfg7LfXCoMW/SOUg4KxPibVeL9HbUbEWVjbgGuehA8p5CFtWDT54+uyzJmPFaNE+Ztio4cErXJhkzHw4NtJRCNYRNcdRml6cc6he8LOa83hJRf6iQuoOucZZspFvm9pdxEwxUHJHx524mDVnyAuOguwO/IkFq/Ndc1u7EweUgKenYdy1IjzVG1hQ80acJozJpzBB3o+uWj+tYRmWYN+NjXEc0C0cXxA+8QQ78RHqjUuLBwfl6PNMDscvGCZNg9E7QCYwEPxI5eqq65C4KYuAp3+4vmHjvQb1Au0EtefPKTuTpjOkPkBvbwFht3ZH/xt80D7Odvv/Vue3AoXRjMReQA4tJLaXYknPTG6YW2cqOcWH9CoQh51TP2q5jJjs= ceph-544fc91e-e75b-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T08:19:17.486 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnRrEwr8KBU6tmaST4rZPAWoIukpVtovy3cNIB7or967TzDpTHJNJ4zw0jISpRkLuFSoSy0tkw7l77IW3/7DkiPIwRkxh1iH8Q1V1qdbEfKnVnSui81TIdUKCLKfbYlnoP+FRQfg7LfXCoMW/SOUg4KxPibVeL9HbUbEWVjbgGuehA8p5CFtWDT54+uyzJmPFaNE+Ztio4cErXJhkzHw4NtJRCNYRNcdRml6cc6he8LOa83hJRf6iQuoOucZZspFvm9pdxEwxUHJHx524mDVnyAuOguwO/IkFq/Ndc1u7EweUgKenYdy1IjzVG1hQ80acJozJpzBB3o+uWj+tYRmWYN+NjXEc0C0cXxA+8QQ78RHqjUuLBwfl6PNMDscvGCZNg9E7QCYwEPxI5eqq65C4KYuAp3+4vmHjvQb1Au0EtefPKTuTpjOkPkBvbwFht3ZH/xt80D7Odvv/Vue3AoXRjMReQA4tJLaXYknPTG6YW2cqOcWH9CoQh51TP2q5jJjs= ceph-544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:19:17.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T08:19:17.983 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:19:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.690+0000 7f4753e00700 1 -- 172.21.15.3:0/4039832749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 msgr2=0x7f47541026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.690+0000 7f4753e00700 1 --2- 172.21.15.3:0/4039832749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f47541026c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4744009910 tx=0x7f4744009c20 comp rx=0 tx=0).stop 2024-03-21T08:19:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.691+0000 7f4753e00700 1 -- 172.21.15.3:0/4039832749 shutdown_connections 2024-03-21T08:19:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.691+0000 7f4753e00700 1 --2- 172.21.15.3:0/4039832749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f47541026c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.691+0000 7f4753e00700 1 -- 172.21.15.3:0/4039832749 >> 172.21.15.3:0/4039832749 conn(0x7f47540fbeb0 msgr2=0x7f47540fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.691+0000 7f4753e00700 1 -- 172.21.15.3:0/4039832749 shutdown_connections 2024-03-21T08:19:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.691+0000 7f4753e00700 1 -- 172.21.15.3:0/4039832749 wait complete. 2024-03-21T08:19:20.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.692+0000 7f4753e00700 1 Processor -- start 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.692+0000 7f4753e00700 1 -- start start 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.692+0000 7f4753e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f475419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.692+0000 7f4753e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754105800 con 0x7f4754100290 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.693+0000 7f4752a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f475419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.693+0000 7f4752a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f475419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49810/0 (socket says 172.21.15.3:49810) 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.693+0000 7f4752a00700 1 -- 172.21.15.3:0/834425954 learned_addr learned my addr 172.21.15.3:0/834425954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.693+0000 7f4752a00700 1 -- 172.21.15.3:0/834425954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475419db50 con 0x7f4754100290 2024-03-21T08:19:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.693+0000 7f4752a00700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f475419d610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4744005190 tx=0x7f4744004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:20.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.694+0000 7f474b400700 1 -- 172.21.15.3:0/834425954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4744027070 con 0x7f4754100290 2024-03-21T08:19:20.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.694+0000 7f474b400700 1 -- 172.21.15.3:0/834425954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4744022470 con 0x7f4754100290 2024-03-21T08:19:20.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.694+0000 7f474b400700 1 -- 172.21.15.3:0/834425954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f474400f5f0 con 0x7f4754100290 2024-03-21T08:19:20.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.694+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475419dde0 con 0x7f4754100290 2024-03-21T08:19:20.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.694+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47541a0ad0 con 0x7f4754100290 2024-03-21T08:19:20.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.695+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4754105f80 con 0x7f4754100290 2024-03-21T08:19:20.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.695+0000 7f474b400700 1 -- 172.21.15.3:0/834425954 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f474402a070 con 0x7f4754100290 2024-03-21T08:19:20.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.696+0000 7f474b400700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4738039280 0x7f473803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:20.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.696+0000 7f474b400700 1 -- 172.21.15.3:0/834425954 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f474401e030 con 0x7f4754100290 2024-03-21T08:19:20.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.696+0000 7f4752000700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4738039280 0x7f473803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:20.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.697+0000 7f4752000700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4738039280 0x7f473803b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4740006fd0 tx=0x7f4740006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:20.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.701+0000 7f474b400700 1 -- 172.21.15.3:0/834425954 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4744013070 con 0x7f4754100290 2024-03-21T08:19:20.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.925+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f475402db40 con 0x7f4754100290 2024-03-21T08:19:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.930+0000 7f474b400700 1 -- 172.21.15.3:0/834425954 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f47440310c0 con 0x7f4754100290 2024-03-21T08:19:20.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.933+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4738039280 msgr2=0x7f473803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:20.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.933+0000 7f4753e00700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4738039280 0x7f473803b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4740006fd0 tx=0x7f4740006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:20.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.933+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 msgr2=0x7f475419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:20.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.933+0000 7f4753e00700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f475419d610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4744005190 tx=0x7f4744004f80 comp rx=0 tx=0).stop 2024-03-21T08:19:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.934+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 shutdown_connections 2024-03-21T08:19:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.934+0000 7f4753e00700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4738039280 0x7f473803b740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.934+0000 7f4753e00700 1 --2- 172.21.15.3:0/834425954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4754100290 0x7f475419d610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.934+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 >> 172.21.15.3:0/834425954 conn(0x7f47540fbeb0 msgr2=0x7f47540fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.935+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 shutdown_connections 2024-03-21T08:19:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:20.935+0000 7f4753e00700 1 -- 172.21.15.3:0/834425954 wait complete. 2024-03-21T08:19:21.701 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T08:19:21.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T08:19:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:21 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/834425954' entity='client.admin' 2024-03-21T08:19:22.821 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:19:23.916 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:23.917 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:23.917 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:19:23.917 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:23.917 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:19:23.917 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:24.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.716+0000 7fea9aa00700 1 -- 172.21.15.3:0/331515917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 msgr2=0x7fea941006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:24.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.716+0000 7fea9aa00700 1 --2- 172.21.15.3:0/331515917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea941006b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea88009910 tx=0x7fea88009c20 comp rx=0 tx=0).stop 2024-03-21T08:19:24.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.716+0000 7fea9aa00700 1 -- 172.21.15.3:0/331515917 shutdown_connections 2024-03-21T08:19:24.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.716+0000 7fea9aa00700 1 --2- 172.21.15.3:0/331515917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea941006b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:24.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.716+0000 7fea9aa00700 1 -- 172.21.15.3:0/331515917 >> 172.21.15.3:0/331515917 conn(0x7fea940fbeb0 msgr2=0x7fea940fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:24.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.716+0000 7fea9aa00700 1 -- 172.21.15.3:0/331515917 shutdown_connections 2024-03-21T08:19:24.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.716+0000 7fea9aa00700 1 -- 172.21.15.3:0/331515917 wait complete. 2024-03-21T08:19:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.717+0000 7fea9aa00700 1 Processor -- start 2024-03-21T08:19:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.718+0000 7fea9aa00700 1 -- start start 2024-03-21T08:19:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.718+0000 7fea9aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea94199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.718+0000 7fea9aa00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea94109b40 con 0x7fea94100290 2024-03-21T08:19:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.718+0000 7fea99600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea94199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.719+0000 7fea99600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea94199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49832/0 (socket says 172.21.15.3:49832) 2024-03-21T08:19:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.719+0000 7fea99600700 1 -- 172.21.15.3:0/1024605246 learned_addr learned my addr 172.21.15.3:0/1024605246 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.719+0000 7fea99600700 1 -- 172.21.15.3:0/1024605246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94199790 con 0x7fea94100290 2024-03-21T08:19:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.719+0000 7fea99600700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea94199250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea88005190 tx=0x7fea88004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.719+0000 7fea92000700 1 -- 172.21.15.3:0/1024605246 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea88027070 con 0x7fea94100290 2024-03-21T08:19:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.720+0000 7fea92000700 1 -- 172.21.15.3:0/1024605246 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea88022470 con 0x7fea94100290 2024-03-21T08:19:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.720+0000 7fea92000700 1 -- 172.21.15.3:0/1024605246 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea8800f5f0 con 0x7fea94100290 2024-03-21T08:19:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.720+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea94199a20 con 0x7fea94100290 2024-03-21T08:19:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.720+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9419c6c0 con 0x7fea94100290 2024-03-21T08:19:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.721+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9410a2c0 con 0x7fea94100290 2024-03-21T08:19:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.721+0000 7fea92000700 1 -- 172.21.15.3:0/1024605246 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7fea8802a070 con 0x7fea94100290 2024-03-21T08:19:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.721+0000 7fea92000700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea7c039280 0x7fea7c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.721+0000 7fea92000700 1 -- 172.21.15.3:0/1024605246 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fea8801e030 con 0x7fea94100290 2024-03-21T08:19:24.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.722+0000 7fea98c00700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea7c039280 0x7fea7c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:24.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.723+0000 7fea98c00700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea7c039280 0x7fea7c03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea84006fd0 tx=0x7fea84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:24.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.727+0000 7fea92000700 1 -- 172.21.15.3:0/1024605246 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fea88013070 con 0x7fea94100290 2024-03-21T08:19:24.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.959+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fea9402db40 con 0x7fea7c039280 2024-03-21T08:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.967+0000 7fea92000700 1 -- 172.21.15.3:0/1024605246 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fea9402db40 con 0x7fea7c039280 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea7c039280 msgr2=0x7fea7c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea7c039280 0x7fea7c03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea84006fd0 tx=0x7fea84006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 msgr2=0x7fea94199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea94199250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea88005190 tx=0x7fea88004f80 comp rx=0 tx=0).stop 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 shutdown_connections 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fea7c039280 0x7fea7c03b740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 --2- 172.21.15.3:0/1024605246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea94100290 0x7fea94199250 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 >> 172.21.15.3:0/1024605246 conn(0x7fea940fbeb0 msgr2=0x7fea940fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 shutdown_connections 2024-03-21T08:19:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:24.970+0000 7fea9aa00700 1 -- 172.21.15.3:0/1024605246 wait complete. 2024-03-21T08:19:25.807 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi084 2024-03-21T08:19:25.807 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:19:25.807 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.conf 2024-03-21T08:19:25.864 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:19:25.864 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:19:25.962 INFO:tasks.cephadm:Adding host smithi084 to orchestrator... 2024-03-21T08:19:25.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch host add smithi084 2024-03-21T08:19:26.034 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:25 smithi003 ceph-mon[31675]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:19:26.034 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:26.035 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:19:26.035 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:19:26.035 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:19:26.035 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:25 smithi003 ceph-mon[31675]: Updating smithi003:/etc/ceph/ceph.conf 2024-03-21T08:19:26.384 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:19:27.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:26 smithi003 ceph-mon[31675]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:27.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:26 smithi003 ceph-mon[31675]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:19:28.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:27 smithi003 ceph-mon[31675]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:19:28.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:27 smithi003 ceph-mon[31675]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:19:28.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:28.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:28.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:28.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.425+0000 7f8d4b400700 1 -- 172.21.15.3:0/54388154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 msgr2=0x7f8d441021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:28.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.425+0000 7f8d4b400700 1 --2- 172.21.15.3:0/54388154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d441021b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009910 tx=0x7f8d38009c20 comp rx=0 tx=0).stop 2024-03-21T08:19:28.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.425+0000 7f8d4b400700 1 -- 172.21.15.3:0/54388154 shutdown_connections 2024-03-21T08:19:28.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.425+0000 7f8d4b400700 1 --2- 172.21.15.3:0/54388154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d441021b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:28.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.425+0000 7f8d4b400700 1 -- 172.21.15.3:0/54388154 >> 172.21.15.3:0/54388154 conn(0x7f8d440fd6a0 msgr2=0x7f8d440ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.426+0000 7f8d4b400700 1 -- 172.21.15.3:0/54388154 shutdown_connections 2024-03-21T08:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.426+0000 7f8d4b400700 1 -- 172.21.15.3:0/54388154 wait complete. 2024-03-21T08:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.427+0000 7f8d4b400700 1 Processor -- start 2024-03-21T08:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.427+0000 7f8d4b400700 1 -- start start 2024-03-21T08:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.427+0000 7f8d4b400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d4419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.427+0000 7f8d4b400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4410c050 con 0x7f8d44101dd0 2024-03-21T08:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.428+0000 7f8d48a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d4419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.428+0000 7f8d48a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d4419f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49856/0 (socket says 172.21.15.3:49856) 2024-03-21T08:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.428+0000 7f8d48a00700 1 -- 172.21.15.3:0/1349808443 learned_addr learned my addr 172.21.15.3:0/1349808443 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.428+0000 7f8d48a00700 1 -- 172.21.15.3:0/1349808443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4419f970 con 0x7f8d44101dd0 2024-03-21T08:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.428+0000 7f8d48a00700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d4419f430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d38005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.429+0000 7f8d41600700 1 -- 172.21.15.3:0/1349808443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d38026070 con 0x7f8d44101dd0 2024-03-21T08:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.429+0000 7f8d41600700 1 -- 172.21.15.3:0/1349808443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d3800bba0 con 0x7f8d44101dd0 2024-03-21T08:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.429+0000 7f8d41600700 1 -- 172.21.15.3:0/1349808443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d380038a0 con 0x7f8d44101dd0 2024-03-21T08:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.429+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4419fc60 con 0x7f8d44101dd0 2024-03-21T08:19:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.429+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d441995f0 con 0x7f8d44101dd0 2024-03-21T08:19:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.430+0000 7f8d41600700 1 -- 172.21.15.3:0/1349808443 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f8d38029070 con 0x7f8d44101dd0 2024-03-21T08:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.430+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d441032b0 con 0x7f8d44101dd0 2024-03-21T08:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.430+0000 7f8d41600700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8d2c039280 0x7f8d2c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.431+0000 7f8d41600700 1 -- 172.21.15.3:0/1349808443 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d3801e070 con 0x7f8d44101dd0 2024-03-21T08:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.431+0000 7f8d43e00700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8d2c039280 0x7f8d2c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.431+0000 7f8d43e00700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8d2c039280 0x7f8d2c03b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006fd0 tx=0x7f8d30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.436+0000 7f8d41600700 1 -- 172.21.15.3:0/1349808443 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d38013070 con 0x7f8d44101dd0 2024-03-21T08:19:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:28.666+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d4419ab20 con 0x7f8d2c039280 2024-03-21T08:19:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:28 smithi003 ceph-mon[31675]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:30.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:29 smithi003 ceph-mon[31675]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:19:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:30 smithi003 ceph-mon[31675]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:30 smithi003 ceph-mon[31675]: Deploying cephadm binary to smithi084 2024-03-21T08:19:33.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:32 smithi003 ceph-mon[31675]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.355+0000 7f8d41600700 1 -- 172.21.15.3:0/1349808443 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f8d4419ab20 con 0x7f8d2c039280 2024-03-21T08:19:35.357 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi084' with addr '172.21.15.84' 2024-03-21T08:19:35.358 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:34 smithi003 ceph-mon[31675]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8d2c039280 msgr2=0x7f8d2c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8d2c039280 0x7f8d2c03b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006fd0 tx=0x7f8d30006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 msgr2=0x7f8d4419f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d4419f430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d38005780 comp rx=0 tx=0).stop 2024-03-21T08:19:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 shutdown_connections 2024-03-21T08:19:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8d2c039280 0x7f8d2c03b740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 --2- 172.21.15.3:0/1349808443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d44101dd0 0x7f8d4419f430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.359+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 >> 172.21.15.3:0/1349808443 conn(0x7f8d440fd6a0 msgr2=0x7f8d440743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.360+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 shutdown_connections 2024-03-21T08:19:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:35.360+0000 7f8d4b400700 1 -- 172.21.15.3:0/1349808443 wait complete. 2024-03-21T08:19:36.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T08:19:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:19:36.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:36 smithi003 ceph-mon[31675]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:36.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:36.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:36 smithi003 ceph-mon[31675]: Added host smithi084 2024-03-21T08:19:36.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:19:36.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:38.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.640+0000 7f437ca00700 1 -- 172.21.15.3:0/551667235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 msgr2=0x7f4378103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.640+0000 7f437ca00700 1 --2- 172.21.15.3:0/551667235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f4378103180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4368009910 tx=0x7f4368009c20 comp rx=0 tx=0).stop 2024-03-21T08:19:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.640+0000 7f437ca00700 1 -- 172.21.15.3:0/551667235 shutdown_connections 2024-03-21T08:19:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.640+0000 7f437ca00700 1 --2- 172.21.15.3:0/551667235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f4378103180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.640+0000 7f437ca00700 1 -- 172.21.15.3:0/551667235 >> 172.21.15.3:0/551667235 conn(0x7f43780776d0 msgr2=0x7f43780fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.640+0000 7f437ca00700 1 -- 172.21.15.3:0/551667235 shutdown_connections 2024-03-21T08:19:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.640+0000 7f437ca00700 1 -- 172.21.15.3:0/551667235 wait complete. 2024-03-21T08:19:38.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.641+0000 7f437ca00700 1 Processor -- start 2024-03-21T08:19:38.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.641+0000 7f437ca00700 1 -- start start 2024-03-21T08:19:38.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.642+0000 7f437ca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f437819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:38.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.642+0000 7f437ca00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437810c0b0 con 0x7f4378102da0 2024-03-21T08:19:38.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.642+0000 7f4375e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f437819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:38.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.642+0000 7f4375e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f437819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58362/0 (socket says 172.21.15.3:58362) 2024-03-21T08:19:38.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.642+0000 7f4375e00700 1 -- 172.21.15.3:0/1148816924 learned_addr learned my addr 172.21.15.3:0/1148816924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:38.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.643+0000 7f4375e00700 1 -- 172.21.15.3:0/1148816924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437819fad0 con 0x7f4378102da0 2024-03-21T08:19:38.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.643+0000 7f4375e00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f437819f590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43680056d0 tx=0x7f43680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:38.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.643+0000 7f436ea00700 1 -- 172.21.15.3:0/1148816924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4368026070 con 0x7f4378102da0 2024-03-21T08:19:38.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.643+0000 7f436ea00700 1 -- 172.21.15.3:0/1148816924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4368004370 con 0x7f4378102da0 2024-03-21T08:19:38.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.643+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437819fd60 con 0x7f4378102da0 2024-03-21T08:19:38.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.643+0000 7f436ea00700 1 -- 172.21.15.3:0/1148816924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43680039b0 con 0x7f4378102da0 2024-03-21T08:19:38.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.643+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4378199660 con 0x7f4378102da0 2024-03-21T08:19:38.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.644+0000 7f436ea00700 1 -- 172.21.15.3:0/1148816924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f4368029070 con 0x7f4378102da0 2024-03-21T08:19:38.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.644+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378069de0 con 0x7f4378102da0 2024-03-21T08:19:38.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.645+0000 7f436ea00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f435c039280 0x7f435c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:38.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.645+0000 7f436ea00700 1 -- 172.21.15.3:0/1148816924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f436801e070 con 0x7f4378102da0 2024-03-21T08:19:38.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.646+0000 7f436fe00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f435c039280 0x7f435c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:38.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.646+0000 7f436fe00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f435c039280 0x7f435c03b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4360006fd0 tx=0x7f4360006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:38.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.651+0000 7f436ea00700 1 -- 172.21.15.3:0/1148816924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4368013070 con 0x7f4378102da0 2024-03-21T08:19:38.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.882+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43780637c0 con 0x7f435c039280 2024-03-21T08:19:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.885+0000 7f436ea00700 1 -- 172.21.15.3:0/1148816924 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f43780637c0 con 0x7f435c039280 2024-03-21T08:19:38.887 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:38.887 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}] 2024-03-21T08:19:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.888+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f435c039280 msgr2=0x7f435c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.888+0000 7f437ca00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f435c039280 0x7f435c03b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4360006fd0 tx=0x7f4360006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.888+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 msgr2=0x7f437819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.888+0000 7f437ca00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f437819f590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43680056d0 tx=0x7f43680057b0 comp rx=0 tx=0).stop 2024-03-21T08:19:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.889+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 shutdown_connections 2024-03-21T08:19:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.889+0000 7f437ca00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f435c039280 0x7f435c03b740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.889+0000 7f437ca00700 1 --2- 172.21.15.3:0/1148816924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4378102da0 0x7f437819f590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.889+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 >> 172.21.15.3:0/1148816924 conn(0x7f43780776d0 msgr2=0x7f4378107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.889+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 shutdown_connections 2024-03-21T08:19:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:38.889+0000 7f437ca00700 1 -- 172.21.15.3:0/1148816924 wait complete. 2024-03-21T08:19:39.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:38 smithi003 ceph-mon[31675]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:39.572 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-03-21T08:19:39.572 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:19:39.573 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-03-21T08:19:39.629 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:19:39.629 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:19:39.725 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-03-21T08:19:39.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch host add smithi171 2024-03-21T08:19:40.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:39 smithi003 ceph-mon[31675]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:19:40.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:19:41.353 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:40 smithi003 ceph-mon[31675]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.223+0000 7fb031a00700 1 -- 172.21.15.3:0/3203080037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 msgr2=0x7fb02c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.223+0000 7fb031a00700 1 --2- 172.21.15.3:0/3203080037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c1021a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009910 tx=0x7fb01c009c20 comp rx=0 tx=0).stop 2024-03-21T08:19:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.223+0000 7fb031a00700 1 -- 172.21.15.3:0/3203080037 shutdown_connections 2024-03-21T08:19:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.223+0000 7fb031a00700 1 --2- 172.21.15.3:0/3203080037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c1021a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.223+0000 7fb031a00700 1 -- 172.21.15.3:0/3203080037 >> 172.21.15.3:0/3203080037 conn(0x7fb02c0fd670 msgr2=0x7fb02c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.223+0000 7fb031a00700 1 -- 172.21.15.3:0/3203080037 shutdown_connections 2024-03-21T08:19:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.223+0000 7fb031a00700 1 -- 172.21.15.3:0/3203080037 wait complete. 2024-03-21T08:19:42.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.224+0000 7fb031a00700 1 Processor -- start 2024-03-21T08:19:42.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.225+0000 7fb031a00700 1 -- start start 2024-03-21T08:19:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.225+0000 7fb031a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.225+0000 7fb031a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c10c040 con 0x7fb02c101dc0 2024-03-21T08:19:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.225+0000 7fb02aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.225+0000 7fb02aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57116/0 (socket says 172.21.15.3:57116) 2024-03-21T08:19:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.225+0000 7fb02aa00700 1 -- 172.21.15.3:0/85435743 learned_addr learned my addr 172.21.15.3:0/85435743 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.226+0000 7fb02aa00700 1 -- 172.21.15.3:0/85435743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c076380 con 0x7fb02c101dc0 2024-03-21T08:19:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.226+0000 7fb02aa00700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c074490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0056d0 tx=0x7fb01c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.226+0000 7fb023e00700 1 -- 172.21.15.3:0/85435743 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb01c026070 con 0x7fb02c101dc0 2024-03-21T08:19:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.226+0000 7fb023e00700 1 -- 172.21.15.3:0/85435743 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb01c00bc00 con 0x7fb02c101dc0 2024-03-21T08:19:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.226+0000 7fb023e00700 1 -- 172.21.15.3:0/85435743 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb01c0039b0 con 0x7fb02c101dc0 2024-03-21T08:19:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.226+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c074a30 con 0x7fb02c101dc0 2024-03-21T08:19:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.227+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c074ef0 con 0x7fb02c101dc0 2024-03-21T08:19:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.228+0000 7fb023e00700 1 -- 172.21.15.3:0/85435743 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7fb01c029070 con 0x7fb02c101dc0 2024-03-21T08:19:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.228+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c1032a0 con 0x7fb02c101dc0 2024-03-21T08:19:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.228+0000 7fb023e00700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb010039280 0x7fb01003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.228+0000 7fb023e00700 1 -- 172.21.15.3:0/85435743 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb01c01e070 con 0x7fb02c101dc0 2024-03-21T08:19:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.229+0000 7fb022000700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb010039280 0x7fb01003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.229+0000 7fb022000700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb010039280 0x7fb01003b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb014006fd0 tx=0x7fb014006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.234+0000 7fb023e00700 1 -- 172.21.15.3:0/85435743 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb01c013070 con 0x7fb02c101dc0 2024-03-21T08:19:42.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:42.465+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) v1 -- 0x7fb02c0759c0 con 0x7fb010039280 2024-03-21T08:19:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:42 smithi003 ceph-mon[31675]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:44.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:43 smithi003 ceph-mon[31675]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:19:45.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:45.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: Deploying cephadm binary to smithi171 2024-03-21T08:19:45.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:45.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:45.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:45.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:45.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:19:45.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:19:45.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:19:46.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:45 smithi003 ceph-mon[31675]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:19:46.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:45 smithi003 ceph-mon[31675]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:19:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:46 smithi003 ceph-mon[31675]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:49.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.169+0000 7fb023e00700 1 -- 172.21.15.3:0/85435743 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb02c0759c0 con 0x7fb010039280 2024-03-21T08:19:49.172 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-03-21T08:19:49.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb010039280 msgr2=0x7fb01003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:49.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb010039280 0x7fb01003b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb014006fd0 tx=0x7fb014006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:49.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 msgr2=0x7fb02c074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:49.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c074490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0056d0 tx=0x7fb01c0057b0 comp rx=0 tx=0).stop 2024-03-21T08:19:49.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 shutdown_connections 2024-03-21T08:19:49.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb010039280 0x7fb01003b740 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:49.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 --2- 172.21.15.3:0/85435743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02c101dc0 0x7fb02c074490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:49.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 >> 172.21.15.3:0/85435743 conn(0x7fb02c0fd670 msgr2=0x7fb02c103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:49.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 shutdown_connections 2024-03-21T08:19:49.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:49.173+0000 7fb031a00700 1 -- 172.21.15.3:0/85435743 wait complete. 2024-03-21T08:19:49.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:48 smithi003 ceph-mon[31675]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:49.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T08:19:50.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:49 smithi003 ceph-mon[31675]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:19:50.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:49 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:50.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:49 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:50.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:49 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:50.408 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:19:51.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:50 smithi003 ceph-mon[31675]: Added host smithi171 2024-03-21T08:19:51.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:50 smithi003 ceph-mon[31675]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:51.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:50 smithi003 ceph-mon[31675]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:19:51.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:50 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:51.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:50 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:19:52.321 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:52 smithi003 ceph-mon[31675]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:52.321 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:52.322 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:52.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.491+0000 7f9747a00700 1 -- 172.21.15.3:0/4272335359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 msgr2=0x7f97401087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:52.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.491+0000 7f9747a00700 1 --2- 172.21.15.3:0/4272335359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401087d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9730009910 tx=0x7f9730009c20 comp rx=0 tx=0).stop 2024-03-21T08:19:52.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.491+0000 7f9747a00700 1 -- 172.21.15.3:0/4272335359 shutdown_connections 2024-03-21T08:19:52.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.491+0000 7f9747a00700 1 --2- 172.21.15.3:0/4272335359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401087d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:52.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.491+0000 7f9747a00700 1 -- 172.21.15.3:0/4272335359 >> 172.21.15.3:0/4272335359 conn(0x7f9740077930 msgr2=0x7f9740079d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:52.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.491+0000 7f9747a00700 1 -- 172.21.15.3:0/4272335359 shutdown_connections 2024-03-21T08:19:52.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.491+0000 7f9747a00700 1 -- 172.21.15.3:0/4272335359 wait complete. 2024-03-21T08:19:52.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.492+0000 7f9747a00700 1 Processor -- start 2024-03-21T08:19:52.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.493+0000 7f9747a00700 1 -- start start 2024-03-21T08:19:52.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.493+0000 7f9747a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:52.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.493+0000 7f9747a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740112670 con 0x7f97401083f0 2024-03-21T08:19:52.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.493+0000 7f9745000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:52.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.493+0000 7f9745000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39504/0 (socket says 172.21.15.3:39504) 2024-03-21T08:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.493+0000 7f9745000700 1 -- 172.21.15.3:0/371924188 learned_addr learned my addr 172.21.15.3:0/371924188 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.494+0000 7f9745000700 1 -- 172.21.15.3:0/371924188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97401a3d50 con 0x7f97401083f0 2024-03-21T08:19:52.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.494+0000 7f9745000700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401a3810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97300056d0 tx=0x7f97300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:52.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.494+0000 7f973e000700 1 -- 172.21.15.3:0/371924188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9730026070 con 0x7f97401083f0 2024-03-21T08:19:52.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.494+0000 7f973e000700 1 -- 172.21.15.3:0/371924188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f973000bba0 con 0x7f97401083f0 2024-03-21T08:19:52.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.494+0000 7f973e000700 1 -- 172.21.15.3:0/371924188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97300037f0 con 0x7f97401083f0 2024-03-21T08:19:52.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.495+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97401a4040 con 0x7f97401083f0 2024-03-21T08:19:52.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.495+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974019d9d0 con 0x7f97401083f0 2024-03-21T08:19:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.496+0000 7f973e000700 1 -- 172.21.15.3:0/371924188 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f9730029070 con 0x7f97401083f0 2024-03-21T08:19:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.496+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97401098d0 con 0x7f97401083f0 2024-03-21T08:19:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.496+0000 7f973e000700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9724039280 0x7f972403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.496+0000 7f973e000700 1 -- 172.21.15.3:0/371924188 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f973001e070 con 0x7f97401083f0 2024-03-21T08:19:52.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.497+0000 7f9737e00700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9724039280 0x7f972403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:52.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.497+0000 7f9737e00700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9724039280 0x7f972403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9728006fd0 tx=0x7f9728006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:52.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.502+0000 7f973e000700 1 -- 172.21.15.3:0/371924188 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9730013070 con 0x7f97401083f0 2024-03-21T08:19:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.737+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f974019ef70 con 0x7f9724039280 2024-03-21T08:19:52.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.740+0000 7f973e000700 1 -- 172.21.15.3:0/371924188 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f974019ef70 con 0x7f9724039280 2024-03-21T08:19:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:19:52.743 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9724039280 msgr2=0x7f972403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9724039280 0x7f972403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9728006fd0 tx=0x7f9728006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 msgr2=0x7f97401a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401a3810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97300056d0 tx=0x7f97300057b0 comp rx=0 tx=0).stop 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 shutdown_connections 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9724039280 0x7f972403b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 --2- 172.21.15.3:0/371924188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97401083f0 0x7f97401a3810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 >> 172.21.15.3:0/371924188 conn(0x7f9740077930 msgr2=0x7f9740109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.743+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 shutdown_connections 2024-03-21T08:19:52.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:52.744+0000 7f9747a00700 1 -- 172.21.15.3:0/371924188 wait complete. 2024-03-21T08:19:53.556 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:53 smithi003 ceph-mon[31675]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:19:53.614 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T08:19:53.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T08:19:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:19:54.518 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:54 smithi003 ceph-mon[31675]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:54.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:19:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.103+0000 7f8fa0e00700 1 -- 172.21.15.3:0/1401126129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 msgr2=0x7f8f9c1043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.103+0000 7f8fa0e00700 1 --2- 172.21.15.3:0/1401126129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c1043b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009910 tx=0x7f8f8c009c20 comp rx=0 tx=0).stop 2024-03-21T08:19:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.103+0000 7f8fa0e00700 1 -- 172.21.15.3:0/1401126129 shutdown_connections 2024-03-21T08:19:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.103+0000 7f8fa0e00700 1 --2- 172.21.15.3:0/1401126129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c1043b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.103+0000 7f8fa0e00700 1 -- 172.21.15.3:0/1401126129 >> 172.21.15.3:0/1401126129 conn(0x7f8f9c0ff860 msgr2=0x7f8f9c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.104+0000 7f8fa0e00700 1 -- 172.21.15.3:0/1401126129 shutdown_connections 2024-03-21T08:19:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.104+0000 7f8fa0e00700 1 -- 172.21.15.3:0/1401126129 wait complete. 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.105+0000 7f8fa0e00700 1 Processor -- start 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.105+0000 7f8fa0e00700 1 -- start start 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.105+0000 7f8fa0e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.105+0000 7f8fa0e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c10e250 con 0x7f8f9c103fd0 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.105+0000 7f8f99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.105+0000 7f8f99e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39524/0 (socket says 172.21.15.3:39524) 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.105+0000 7f8f99e00700 1 -- 172.21.15.3:0/3399190596 learned_addr learned my addr 172.21.15.3:0/3399190596 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:19:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.106+0000 7f8f99e00700 1 -- 172.21.15.3:0/3399190596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c19f960 con 0x7f8f9c103fd0 2024-03-21T08:19:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.106+0000 7f8f99e00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c19f420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006010 tx=0x7f8f8c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.106+0000 7f8f92a00700 1 -- 172.21.15.3:0/3399190596 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f8c026070 con 0x7f8f9c103fd0 2024-03-21T08:19:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.106+0000 7f8f92a00700 1 -- 172.21.15.3:0/3399190596 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8f8c00bba0 con 0x7f8f9c103fd0 2024-03-21T08:19:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.106+0000 7f8f92a00700 1 -- 172.21.15.3:0/3399190596 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f8c0038a0 con 0x7f8f9c103fd0 2024-03-21T08:19:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.106+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9c19fc50 con 0x7f8f9c103fd0 2024-03-21T08:19:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.107+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9c1995e0 con 0x7f8f9c103fd0 2024-03-21T08:19:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.108+0000 7f8f92a00700 1 -- 172.21.15.3:0/3399190596 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f8f8c029070 con 0x7f8f9c103fd0 2024-03-21T08:19:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.108+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9c1054b0 con 0x7f8f9c103fd0 2024-03-21T08:19:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.108+0000 7f8f92a00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8f80039280 0x7f8f8003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:19:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.108+0000 7f8f92a00700 1 -- 172.21.15.3:0/3399190596 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f8c01e070 con 0x7f8f9c103fd0 2024-03-21T08:19:56.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.109+0000 7f8f93e00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8f80039280 0x7f8f8003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:19:56.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.109+0000 7f8f93e00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8f80039280 0x7f8f8003b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8f84006fd0 tx=0x7f8f84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:19:56.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.114+0000 7f8f92a00700 1 -- 172.21.15.3:0/3399190596 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f8c013070 con 0x7f8f9c103fd0 2024-03-21T08:19:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:56.351+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8f9c0637c0 con 0x7f8f9c103fd0 2024-03-21T08:19:56.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:56 smithi003 ceph-mon[31675]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:57.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.228+0000 7f8f92a00700 1 -- 172.21.15.3:0/3399190596 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8f8c030090 con 0x7f8f9c103fd0 2024-03-21T08:19:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.233+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8f80039280 msgr2=0x7f8f8003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8f80039280 0x7f8f8003b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8f84006fd0 tx=0x7f8f84006e40 comp rx=0 tx=0).stop 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 msgr2=0x7f8f9c19f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c19f420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006010 tx=0x7f8f8c005780 comp rx=0 tx=0).stop 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 shutdown_connections 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8f80039280 0x7f8f8003b740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 --2- 172.21.15.3:0/3399190596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c103fd0 0x7f8f9c19f420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 >> 172.21.15.3:0/3399190596 conn(0x7f8f9c0ff860 msgr2=0x7f8f9c1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 shutdown_connections 2024-03-21T08:19:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:19:57.234+0000 7f8fa0e00700 1 -- 172.21.15.3:0/3399190596 wait complete. 2024-03-21T08:19:57.238 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-03-21T08:19:57.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:57 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3399190596' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T08:19:57.967 INFO:tasks.cephadm:Adding mon.a on smithi003 2024-03-21T08:19:57.968 INFO:tasks.cephadm:Adding mon.b on smithi084 2024-03-21T08:19:57.968 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-03-21T08:19:57.968 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c' 2024-03-21T08:19:58.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:58 smithi003 ceph-mon[31675]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:19:58.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:58 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3399190596' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T08:19:58.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:19:58 smithi003 ceph-mon[31675]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:19:59.011 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T08:19:59.106 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T08:20:00.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:00.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:00.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:00.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:00.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:20:00.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:00.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:00.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:00 smithi003 ceph-mon[31675]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:20:01.402 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.400+0000 7ff117600700 1 -- 172.21.15.171:0/3370941119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 msgr2=0x7ff1100745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:01.403 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.400+0000 7ff117600700 1 --2- 172.21.15.171:0/3370941119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff1100745c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff104009910 tx=0x7ff104009c20 comp rx=0 tx=0).stop 2024-03-21T08:20:01.403 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.400+0000 7ff117600700 1 -- 172.21.15.171:0/3370941119 shutdown_connections 2024-03-21T08:20:01.403 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.400+0000 7ff117600700 1 --2- 172.21.15.171:0/3370941119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff1100745c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:01.403 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.400+0000 7ff117600700 1 -- 172.21.15.171:0/3370941119 >> 172.21.15.171:0/3370941119 conn(0x7ff1100fd160 msgr2=0x7ff1100ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:01.403 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.400+0000 7ff117600700 1 -- 172.21.15.171:0/3370941119 shutdown_connections 2024-03-21T08:20:01.403 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.400+0000 7ff117600700 1 -- 172.21.15.171:0/3370941119 wait complete. 2024-03-21T08:20:01.404 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.402+0000 7ff117600700 1 Processor -- start 2024-03-21T08:20:01.404 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.402+0000 7ff117600700 1 -- start start 2024-03-21T08:20:01.404 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.402+0000 7ff117600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff110199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:01.404 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.402+0000 7ff117600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110109cf0 con 0x7ff110076160 2024-03-21T08:20:01.405 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.403+0000 7ff114c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff110199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:01.405 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.403+0000 7ff114c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff110199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.171:50448/0 (socket says 172.21.15.171:50448) 2024-03-21T08:20:01.405 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.403+0000 7ff114c00700 1 -- 172.21.15.171:0/4270058656 learned_addr learned my addr 172.21.15.171:0/4270058656 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:20:01.405 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.404+0000 7ff114c00700 1 -- 172.21.15.171:0/4270058656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1101998d0 con 0x7ff110076160 2024-03-21T08:20:01.406 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.404+0000 7ff114c00700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff110199390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff104005190 tx=0x7ff104004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:01.406 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.404+0000 7ff10d600700 1 -- 172.21.15.171:0/4270058656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff104027070 con 0x7ff110076160 2024-03-21T08:20:01.407 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.404+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff110199b60 con 0x7ff110076160 2024-03-21T08:20:01.407 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.405+0000 7ff10d600700 1 -- 172.21.15.171:0/4270058656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff104022470 con 0x7ff110076160 2024-03-21T08:20:01.407 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.405+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11019c850 con 0x7ff110076160 2024-03-21T08:20:01.407 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.405+0000 7ff10d600700 1 -- 172.21.15.171:0/4270058656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff10400f460 con 0x7ff110076160 2024-03-21T08:20:01.408 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.406+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11010a470 con 0x7ff110076160 2024-03-21T08:20:01.408 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.406+0000 7ff10d600700 1 -- 172.21.15.171:0/4270058656 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7ff10402a070 con 0x7ff110076160 2024-03-21T08:20:01.409 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.407+0000 7ff10d600700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0f8039280 0x7ff0f803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:01.409 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.407+0000 7ff10d600700 1 -- 172.21.15.171:0/4270058656 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff10401e030 con 0x7ff110076160 2024-03-21T08:20:01.409 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.407+0000 7ff10fe00700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0f8039280 0x7ff0f803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:01.410 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.409+0000 7ff10fe00700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0f8039280 0x7ff0f803b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc006fd0 tx=0x7ff0fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:01.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.423+0000 7ff10d600700 1 -- 172.21.15.171:0/4270058656 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff104013070 con 0x7ff110076160 2024-03-21T08:20:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:01 smithi003 ceph-mon[31675]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:01.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:01 smithi003 ceph-mon[31675]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:01.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.662+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}) v1 -- 0x7ff11002dc10 con 0x7ff0f8039280 2024-03-21T08:20:01.682 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.680+0000 7ff10d600700 1 -- 172.21.15.171:0/4270058656 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff11002dc10 con 0x7ff0f8039280 2024-03-21T08:20:01.682 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.682+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0f8039280 msgr2=0x7ff0f803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.682+0000 7ff117600700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0f8039280 0x7ff0f803b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc006fd0 tx=0x7ff0fc006e40 comp rx=0 tx=0).stop 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.682+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 msgr2=0x7ff110199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.682+0000 7ff117600700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff110199390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff104005190 tx=0x7ff104004f80 comp rx=0 tx=0).stop 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.683+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 shutdown_connections 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.683+0000 7ff117600700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0f8039280 0x7ff0f803b740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.683+0000 7ff117600700 1 --2- 172.21.15.171:0/4270058656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff110076160 0x7ff110199390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.683+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 >> 172.21.15.171:0/4270058656 conn(0x7ff1100fd160 msgr2=0x7ff1100fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:01.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.683+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 shutdown_connections 2024-03-21T08:20:01.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:01.683+0000 7ff117600700 1 -- 172.21.15.171:0/4270058656 wait complete. 2024-03-21T08:20:02.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:02 smithi003 ceph-mon[31675]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:02.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:02.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:02.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:02.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:02.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:02.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:02.449 DEBUG:teuthology.orchestra.run.smithi084:mon.b> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.b.service 2024-03-21T08:20:02.457 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.c.service 2024-03-21T08:20:02.462 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T08:20:02.462 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T08:20:03.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:03 smithi003 ceph-mon[31675]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:03.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:03 smithi003 ceph-mon[31675]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:20:03.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:03 smithi003 ceph-mon[31675]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:03.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:03 smithi003 ceph-mon[31675]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c;count:3 2024-03-21T08:20:03.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:03 smithi003 ceph-mon[31675]: Deploying daemon mon.c on smithi171 2024-03-21T08:20:03.925 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:05 smithi003 ceph-mon[31675]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:07.023 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.020+0000 7f5d51200700 1 -- 172.21.15.171:0/222980357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 msgr2=0x7f5d4c100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:07.023 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.020+0000 7f5d51200700 1 --2- 172.21.15.171:0/222980357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c100cd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009910 tx=0x7f5d3c009c20 comp rx=0 tx=0).stop 2024-03-21T08:20:07.023 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.020+0000 7f5d51200700 1 -- 172.21.15.171:0/222980357 shutdown_connections 2024-03-21T08:20:07.023 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.020+0000 7f5d51200700 1 --2- 172.21.15.171:0/222980357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c100cd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:07.023 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.020+0000 7f5d51200700 1 -- 172.21.15.171:0/222980357 >> 172.21.15.171:0/222980357 conn(0x7f5d4c0776d0 msgr2=0x7f5d4c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:07.024 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.020+0000 7f5d51200700 1 -- 172.21.15.171:0/222980357 shutdown_connections 2024-03-21T08:20:07.024 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.020+0000 7f5d51200700 1 -- 172.21.15.171:0/222980357 wait complete. 2024-03-21T08:20:07.024 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.021+0000 7f5d51200700 1 Processor -- start 2024-03-21T08:20:07.025 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.021+0000 7f5d51200700 1 -- start start 2024-03-21T08:20:07.025 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.022+0000 7f5d51200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:07.025 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.022+0000 7f5d51200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c10e2b0 con 0x7f5d4c1008f0 2024-03-21T08:20:07.025 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.022+0000 7f5d49e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:07.025 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.023+0000 7f5d49e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.171:50462/0 (socket says 172.21.15.171:50462) 2024-03-21T08:20:07.025 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.023+0000 7f5d49e00700 1 -- 172.21.15.171:0/3589045064 learned_addr learned my addr 172.21.15.171:0/3589045064 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:20:07.025 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.023+0000 7f5d49e00700 1 -- 172.21.15.171:0/3589045064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c106b30 con 0x7f5d4c1008f0 2024-03-21T08:20:07.026 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.024+0000 7f5d49e00700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c102bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c006010 tx=0x7f5d3c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:07.026 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.024+0000 7f5d42a00700 1 -- 172.21.15.171:0/3589045064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c026070 con 0x7f5d4c1008f0 2024-03-21T08:20:07.026 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.024+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c103110 con 0x7f5d4c1008f0 2024-03-21T08:20:07.026 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.024+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c103570 con 0x7f5d4c1008f0 2024-03-21T08:20:07.027 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.024+0000 7f5d42a00700 1 -- 172.21.15.171:0/3589045064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d3c00bc00 con 0x7f5d4c1008f0 2024-03-21T08:20:07.027 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.025+0000 7f5d42a00700 1 -- 172.21.15.171:0/3589045064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c0038a0 con 0x7f5d4c1008f0 2024-03-21T08:20:07.027 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.026+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c069e20 con 0x7f5d4c1008f0 2024-03-21T08:20:07.028 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.026+0000 7f5d42a00700 1 -- 172.21.15.171:0/3589045064 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f5d3c029070 con 0x7f5d4c1008f0 2024-03-21T08:20:07.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.027+0000 7f5d42a00700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5d30039220 0x7f5d3003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:07.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.027+0000 7f5d42a00700 1 -- 172.21.15.171:0/3589045064 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d3c01e070 con 0x7f5d4c1008f0 2024-03-21T08:20:07.029 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.027+0000 7f5d49400700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5d30039220 0x7f5d3003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:07.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.028+0000 7f5d49400700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5d30039220 0x7f5d3003b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:07.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.042+0000 7f5d42a00700 1 -- 172.21.15.171:0/3589045064 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d3c013070 con 0x7f5d4c1008f0 2024-03-21T08:20:07.385 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.383+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5d4c0637c0 con 0x7f5d4c1008f0 2024-03-21T08:20:07.386 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.384+0000 7f5d42a00700 1 -- 172.21.15.171:0/3589045064 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+741 (secure 0 0 0) 0x7f5d3c02a3b0 con 0x7f5d4c1008f0 2024-03-21T08:20:07.386 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:20:07.387 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","modified":"2024-03-21T08:17:46.599812Z","created":"2024-03-21T08:17:46.599812Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5d30039220 msgr2=0x7f5d3003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5d30039220 0x7f5d3003b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38006e40 comp rx=0 tx=0).stop 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 msgr2=0x7f5d4c102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c102bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c006010 tx=0x7f5d3c005780 comp rx=0 tx=0).stop 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 shutdown_connections 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5d30039220 0x7f5d3003b6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 --2- 172.21.15.171:0/3589045064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c1008f0 0x7f5d4c102bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:07.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.387+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 >> 172.21.15.171:0/3589045064 conn(0x7f5d4c0776d0 msgr2=0x7f5d4c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:07.391 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.388+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 shutdown_connections 2024-03-21T08:20:07.391 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:07.388+0000 7f5d51200700 1 -- 172.21.15.171:0/3589045064 wait complete. 2024-03-21T08:20:07.391 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2024-03-21T08:20:07.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:07 smithi003 ceph-mon[31675]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:07.726 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:07 smithi171 systemd[1]: Starting Ceph mon.c for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:20:08.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 podman[35369]: 2024-03-21 08:20:08.151081448 +0000 UTC m=+0.035773953 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:20:08.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 podman[35369]: 2024-03-21 08:20:08.252901787 +0000 UTC m=+0.137594335 container create 5ae9e19f644ea5c5932a2549f4d4cce050ed8060f70831d2dc14c23d95bf9c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:20:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:08 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/3589045064' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:08.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 podman[35369]: 2024-03-21 08:20:08.473302556 +0000 UTC m=+0.357995083 container init 5ae9e19f644ea5c5932a2549f4d4cce050ed8060f70831d2dc14c23d95bf9c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS) 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 podman[35369]: 2024-03-21 08:20:08.483963068 +0000 UTC m=+0.368655596 container start 5ae9e19f644ea5c5932a2549f4d4cce050ed8060f70831d2dc14c23d95bf9c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pidfile_write: ignore empty --pid-file 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: load: jerasure load: lrc 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: RocksDB version: 7.9.2 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Git sha 0 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: DB SUMMARY 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: DB Session ID: AXB0GS9M5Z2E6H1DXZIP 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: CURRENT file: CURRENT 2024-03-21T08:20:08.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.error_if_exists: 0 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.create_if_missing: 0 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.paranoid_checks: 1 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.env: 0x55ecfb5bcb20 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.info_log: 0x55ecfd5d9960 2024-03-21T08:20:08.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.statistics: (nil) 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.use_fsync: 0 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_log_file_size: 0 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.allow_fallocate: 1 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T08:20:08.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.use_direct_reads: 0 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.db_log_dir: 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.wal_dir: 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T08:20:08.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.write_buffer_manager: 0x55ecfd5ac1e0 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.unordered_write: 0 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T08:20:08.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T08:20:08.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.row_cache: None 2024-03-21T08:20:08.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.wal_filter: None 2024-03-21T08:20:08.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T08:20:08.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T08:20:08.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.two_write_queues: 0 2024-03-21T08:20:08.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T08:20:08.892 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.wal_compression: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.atomic_flush: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.log_readahead_size: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T08:20:08.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_background_jobs: 2 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_background_compactions: -1 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_subcompactions: 1 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_open_files: -1 2024-03-21T08:20:08.894 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_background_flushes: -1 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Compression algorithms supported: 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kZSTD supported: 0 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kXpressCompression supported: 0 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kZlibCompression supported: 1 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kSnappyCompression supported: 1 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kLZ4Compression supported: 1 2024-03-21T08:20:08.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: kBZip2Compression supported: 0 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.merge_operator: 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_filter: None 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ecfd5d9a60) 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T08:20:08.896 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_type: 0 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_index_type: 0 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_shortening: 1 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: checksum: 4 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: no_block_cache: 0 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache: 0x55ecfd5df090 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_options: 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: capacity : 536870912 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_shard_bits : 4 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: strict_capacity_limit : 0 2024-03-21T08:20:08.897 INFO:journalctl@ceph.mon.c.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_compressed: (nil) 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: persistent_cache: (nil) 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size: 4096 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size_deviation: 10 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_restart_interval: 16 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_block_restart_interval: 1 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: metadata_block_size: 4096 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: partition_filters: 0 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: use_delta_encoding: 1 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: filter_policy: bloomfilter 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: whole_key_filtering: 1 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: verify_compression: 0 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: format_version: 5 2024-03-21T08:20:08.898 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_index_compression: 1 2024-03-21T08:20:08.899 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_align: 0 2024-03-21T08:20:08.899 INFO:journalctl@ceph.mon.c.smithi171.stdout: max_auto_readahead_size: 262144 2024-03-21T08:20:08.899 INFO:journalctl@ceph.mon.c.smithi171.stdout: prepopulate_block_cache: 0 2024-03-21T08:20:08.899 INFO:journalctl@ceph.mon.c.smithi171.stdout: initial_auto_readahead_size: 8192 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression: NoCompression 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.num_levels: 7 2024-03-21T08:20:08.900 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T08:20:08.901 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T08:20:08.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T08:20:08.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.table_properties_collectors: 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.inplace_update_support: 0 2024-03-21T08:20:08.904 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.bloom_locality: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.max_successive_merges: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.ttl: 2592000 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.enable_blob_files: false 2024-03-21T08:20:08.905 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.min_blob_size: 0 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6a682693-6f72-438c-ab6e-42735d801184 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009208542401, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T08:20:08.906 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009208543265, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711009208, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6a682693-6f72-438c-ab6e-42735d801184", "db_session_id": "AXB0GS9M5Z2E6H1DXZIP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009208543519, "job": 1, "event": "recovery_finished"} 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ecfd6d2000 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: DB pointer 0x55ecfd6bc000 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** DB Stats ** 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:20:08.907 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:08.908 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache BinnedLRUCache@0x55ecfd5df090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T08:20:08.909 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: starting mon.c rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(???) e0 preinit fsid 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).mds e1 new map 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2024-03-21T08:20:08.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Deploying cephadm binary to smithi084 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Added host smithi084 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Deploying cephadm binary to smithi171 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.913 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Added host smithi171 2024-03-21T08:20:08.914 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.914 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:20:08.914 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.914 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:08.914 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3399190596' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3399190596' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.915 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:08.916 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c;count:3 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: Deploying daemon mon.c on smithi171 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/3589045064' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 ceph-mon[35388]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 bash[35369]: 5ae9e19f644ea5c5932a2549f4d4cce050ed8060f70831d2dc14c23d95bf9c12 2024-03-21T08:20:08.917 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:08 smithi171 systemd[1]: Started Ceph mon.c for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:20:09.079 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T08:20:09.079 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T08:20:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: mon.a calling monitor election 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: mon.c calling monitor election 2024-03-21T08:20:13.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: fsmap 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: mgrmap e13: x(active, since 68s) 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: overall HEALTH_OK 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:13.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:14.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:20:14.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: mon.a calling monitor election 2024-03-21T08:20:14.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: mon.c calling monitor election 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: fsmap 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: mgrmap e13: x(active, since 68s) 2024-03-21T08:20:14.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: overall HEALTH_OK 2024-03-21T08:20:14.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:14.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:14.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:14.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:14.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:14.537 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.533+0000 7fe8c5600700 1 -- 172.21.15.171:0/3880093875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 msgr2=0x7fe8a4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:14.537 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.533+0000 7fe8c5600700 1 --2- 172.21.15.171:0/3880093875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8a4005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0005fa0 tx=0x7fe8b000ffc0 comp rx=0 tx=0).stop 2024-03-21T08:20:14.537 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.533+0000 7fe8c5600700 1 -- 172.21.15.171:0/3880093875 shutdown_connections 2024-03-21T08:20:14.537 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.533+0000 7fe8c5600700 1 --2- 172.21.15.171:0/3880093875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8a4005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:14.537 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.533+0000 7fe8c5600700 1 -- 172.21.15.171:0/3880093875 >> 172.21.15.171:0/3880093875 conn(0x7fe8c00fc640 msgr2=0x7fe8c00fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:14.537 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.534+0000 7fe8c5600700 1 -- 172.21.15.171:0/3880093875 shutdown_connections 2024-03-21T08:20:14.537 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.534+0000 7fe8c5600700 1 -- 172.21.15.171:0/3880093875 wait complete. 2024-03-21T08:20:14.538 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.536+0000 7fe8c5600700 1 Processor -- start 2024-03-21T08:20:14.538 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.536+0000 7fe8c5600700 1 -- start start 2024-03-21T08:20:14.538 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.536+0000 7fe8c5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8c0100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:14.538 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.536+0000 7fe8c5600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 0x7fe8c0101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:14.538 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.536+0000 7fe8c5600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0109ba0 con 0x7fe8c0102e10 2024-03-21T08:20:14.539 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.536+0000 7fe8c5600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0109a20 con 0x7fe8c0100fb0 2024-03-21T08:20:14.539 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.537+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8c0100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:14.539 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.537+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8c0100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.171:58518/0 (socket says 172.21.15.171:58518) 2024-03-21T08:20:14.539 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.537+0000 7fe8bea00700 1 -- 172.21.15.171:0/4268930079 learned_addr learned my addr 172.21.15.171:0/4268930079 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:20:14.539 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.537+0000 7fe8be000700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 0x7fe8c0101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:14.540 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.537+0000 7fe8be000700 1 -- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 msgr2=0x7fe8c0101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-21T08:20:14.540 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.537+0000 7fe8be000700 1 -- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 msgr2=0x7fe8c0101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T08:20:14.540 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.537+0000 7fe8be000700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 0x7fe8c0101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T08:20:14.540 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.538+0000 7fe8be000700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 0x7fe8c0101430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:20:14.540 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.538+0000 7fe8bea00700 1 -- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 msgr2=0x7fe8c0101430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:20:14.540 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.538+0000 7fe8bea00700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 0x7fe8c0101430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:14.540 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.538+0000 7fe8bea00700 1 -- 172.21.15.171:0/4268930079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c0102990 con 0x7fe8c0102e10 2024-03-21T08:20:14.541 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.538+0000 7fe8bea00700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8c0100a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00137f0 tx=0x7fe8b0004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:14.541 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.539+0000 7fe8b7400700 1 -- 172.21.15.171:0/4268930079 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe8b0005230 con 0x7fe8c0102e10 2024-03-21T08:20:14.541 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.539+0000 7fe8b7400700 1 -- 172.21.15.171:0/4268930079 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8b001b430 con 0x7fe8c0102e10 2024-03-21T08:20:14.541 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.539+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c01a39a0 con 0x7fe8c0102e10 2024-03-21T08:20:14.541 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.539+0000 7fe8b7400700 1 -- 172.21.15.171:0/4268930079 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe8b0011670 con 0x7fe8c0102e10 2024-03-21T08:20:14.541 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.539+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c01a3ee0 con 0x7fe8c0102e10 2024-03-21T08:20:14.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.540+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c006a3c0 con 0x7fe8c0102e10 2024-03-21T08:20:14.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.540+0000 7fe8b7400700 1 -- 172.21.15.171:0/4268930079 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7fe8b001b5e0 con 0x7fe8c0102e10 2024-03-21T08:20:14.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.541+0000 7fe8b7400700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8a0038fe0 0x7fe8a003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:14.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.541+0000 7fe8b7400700 1 -- 172.21.15.171:0/4268930079 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe8b004e860 con 0x7fe8c0102e10 2024-03-21T08:20:14.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.542+0000 7fe8be000700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8a0038fe0 0x7fe8a003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:14.545 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.543+0000 7fe8be000700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8a0038fe0 0x7fe8a003b4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009850 tx=0x7fe8ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:14.561 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.559+0000 7fe8b7400700 1 -- 172.21.15.171:0/4268930079 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8b0022070 con 0x7fe8c0102e10 2024-03-21T08:20:14.883 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.881+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe8c002da30 con 0x7fe8c0102e10 2024-03-21T08:20:14.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:14 smithi171 ceph-mon[35388]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:14.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:14 smithi171 ceph-mon[35388]: Deploying daemon mon.b on smithi084 2024-03-21T08:20:14.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:14 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:14.884 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.882+0000 7fe8b7400700 1 -- 172.21.15.171:0/4268930079 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7fe8b00199c0 con 0x7fe8c0102e10 2024-03-21T08:20:14.884 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:20:14.885 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","modified":"2024-03-21T08:20:08.586166Z","created":"2024-03-21T08:17:46.599812Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T08:20:14.887 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8a0038fe0 msgr2=0x7fe8a003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:14.887 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8a0038fe0 0x7fe8a003b4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009850 tx=0x7fe8ac008040 comp rx=0 tx=0).stop 2024-03-21T08:20:14.887 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 msgr2=0x7fe8c0100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8c0100a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00137f0 tx=0x7fe8b0004d60 comp rx=0 tx=0).stop 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 shutdown_connections 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8a0038fe0 0x7fe8a003b4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8c0102e10 0x7fe8c0100a70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 --2- 172.21.15.171:0/4268930079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8c0100fb0 0x7fe8c0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 >> 172.21.15.171:0/4268930079 conn(0x7fe8c00fc640 msgr2=0x7fe8c00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 shutdown_connections 2024-03-21T08:20:14.888 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:14.885+0000 7fe8c5600700 1 -- 172.21.15.171:0/4268930079 wait complete. 2024-03-21T08:20:14.889 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-03-21T08:20:15.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:14 smithi003 ceph-mon[31675]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:15.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:14 smithi003 ceph-mon[31675]: Deploying daemon mon.b on smithi084 2024-03-21T08:20:15.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:14 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:15.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:15 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/4268930079' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:15.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:15 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/4268930079' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:15.927 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:20:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:20:15.587+0000 7f46d3c00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-21T08:20:16.935 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T08:20:16.936 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T08:20:17.024 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:16 smithi171 ceph-mon[35388]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:17.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:16 smithi003 ceph-mon[31675]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:17.354 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:20:18.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:18 smithi171 ceph-mon[35388]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.040 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:18 smithi003 ceph-mon[31675]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.218+0000 7fe614400700 1 -- 172.21.15.171:0/1593125349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c102e10 msgr2=0x7fe60c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:19.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.218+0000 7fe614400700 1 --2- 172.21.15.171:0/1593125349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c102e10 0x7fe60c1031f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00b0a0 tx=0x7fe5fc00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:20:19.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.218+0000 7fe614400700 1 -- 172.21.15.171:0/1593125349 shutdown_connections 2024-03-21T08:20:19.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.218+0000 7fe614400700 1 --2- 172.21.15.171:0/1593125349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c102e10 0x7fe60c1031f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:19.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.218+0000 7fe614400700 1 -- 172.21.15.171:0/1593125349 >> 172.21.15.171:0/1593125349 conn(0x7fe60c0fc640 msgr2=0x7fe60c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:19.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.218+0000 7fe614400700 1 -- 172.21.15.171:0/1593125349 shutdown_connections 2024-03-21T08:20:19.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.218+0000 7fe614400700 1 -- 172.21.15.171:0/1593125349 wait complete. 2024-03-21T08:20:19.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.219+0000 7fe614400700 1 Processor -- start 2024-03-21T08:20:19.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.219+0000 7fe614400700 1 -- start start 2024-03-21T08:20:19.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.220+0000 7fe614400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe60c102e10 0x7fe60c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:19.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.220+0000 7fe614400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c19dad0 0x7fe60c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:19.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.220+0000 7fe614400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c10c1b0 con 0x7fe60c19dad0 2024-03-21T08:20:19.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.220+0000 7fe614400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c10c030 con 0x7fe60c102e10 2024-03-21T08:20:19.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.220+0000 7fe611000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c19dad0 0x7fe60c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:19.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.221+0000 7fe611000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c19dad0 0x7fe60c197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.171:58532/0 (socket says 172.21.15.171:58532) 2024-03-21T08:20:19.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.221+0000 7fe611000700 1 -- 172.21.15.171:0/2779144336 learned_addr learned my addr 172.21.15.171:0/2779144336 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:20:19.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.221+0000 7fe611a00700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe60c102e10 0x7fe60c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:19.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.221+0000 7fe611000700 1 -- 172.21.15.171:0/2779144336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe60c102e10 msgr2=0x7fe60c19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:19.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.221+0000 7fe611000700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe60c102e10 0x7fe60c19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:19.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.221+0000 7fe611000700 1 -- 172.21.15.171:0/2779144336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c19e050 con 0x7fe60c19dad0 2024-03-21T08:20:19.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.222+0000 7fe611000700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c19dad0 0x7fe60c197660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b700 tx=0x7fe5f800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:19.225 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.222+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe5f8011630 con 0x7fe60c19dad0 2024-03-21T08:20:19.225 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.222+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c197c00 con 0x7fe60c19dad0 2024-03-21T08:20:19.225 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.222+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe5f8011c80 con 0x7fe60c19dad0 2024-03-21T08:20:19.225 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.222+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe5f8010c80 con 0x7fe60c19dad0 2024-03-21T08:20:19.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.222+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c198190 con 0x7fe60c19dad0 2024-03-21T08:20:19.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.223+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c06a3c0 con 0x7fe60c19dad0 2024-03-21T08:20:19.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.224+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7fe5f8018020 con 0x7fe60c19dad0 2024-03-21T08:20:19.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.224+0000 7fe60a000700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe5f0039030 0x7fe5f003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:19.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.224+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe5f804b720 con 0x7fe60c19dad0 2024-03-21T08:20:19.227 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.225+0000 7fe611a00700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe5f0039030 0x7fe5f003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:19.227 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.226+0000 7fe611a00700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe5f0039030 0x7fe5f003b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009fd0 tx=0x7fe5fc0076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:19.242 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.240+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe5f801f070 con 0x7fe60c19dad0 2024-03-21T08:20:19.343 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 podman[35457]: 2024-03-21 08:20:19.10304168 +0000 UTC m=+0.127071857 container create 9f1b9919572d1ded4e001299cfd906db9a7e46637aaf18858ef3648525ccc14e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-21T08:20:19.343 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 podman[35457]: 2024-03-21 08:20:19.339705706 +0000 UTC m=+0.363735907 container init 9f1b9919572d1ded4e001299cfd906db9a7e46637aaf18858ef3648525ccc14e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T08:20:19.464 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.462+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80184f0 con 0x7fe60c19dad0 2024-03-21T08:20:19.554 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:19.552+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe60c198500 con 0x7fe60c19dad0 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 podman[35457]: 2024-03-21 08:20:19.348970537 +0000 UTC m=+0.373000709 container start 9f1b9919572d1ded4e001299cfd906db9a7e46637aaf18858ef3648525ccc14e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pidfile_write: ignore empty --pid-file 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: load: jerasure load: lrc 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: RocksDB version: 7.9.2 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Git sha 0 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: DB SUMMARY 2024-03-21T08:20:19.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: DB Session ID: F8GC7NTFF9BU7H7B5UVH 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: CURRENT file: CURRENT 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.error_if_exists: 0 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.create_if_missing: 0 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.paranoid_checks: 1 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T08:20:19.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.env: 0x55837f3afb20 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.info_log: 0x55838104b960 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.statistics: (nil) 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.use_fsync: 0 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_log_file_size: 0 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T08:20:19.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.allow_fallocate: 1 2024-03-21T08:20:19.631 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T08:20:19.631 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T08:20:19.631 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.use_direct_reads: 0 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.db_log_dir: 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.wal_dir: 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T08:20:19.632 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.write_buffer_manager: 0x55838101e1e0 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T08:20:19.633 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.unordered_write: 0 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.row_cache: None 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.wal_filter: None 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.two_write_queues: 0 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T08:20:19.634 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.wal_compression: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.atomic_flush: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.log_readahead_size: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_background_jobs: 2 2024-03-21T08:20:19.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_background_compactions: -1 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_subcompactions: 1 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_open_files: -1 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T08:20:19.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_background_flushes: -1 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Compression algorithms supported: 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kZSTD supported: 0 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kXpressCompression supported: 0 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kZlibCompression supported: 1 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kSnappyCompression supported: 1 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kLZ4Compression supported: 1 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: kBZip2Compression supported: 0 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T08:20:19.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.merge_operator: 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_filter: None 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55838104ba60) 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout: cache_index_and_filter_blocks: 1 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout: pin_top_level_index_and_filter: 1 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_type: 0 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout: data_block_index_type: 0 2024-03-21T08:20:19.638 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_shortening: 1 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: checksum: 4 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: no_block_cache: 0 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache: 0x558381051090 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_name: BinnedLRUCache 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_options: 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: capacity : 536870912 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: num_shard_bits : 4 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: strict_capacity_limit : 0 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: high_pri_pool_ratio: 0.000 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_compressed: (nil) 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: persistent_cache: (nil) 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_size: 4096 2024-03-21T08:20:19.639 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_size_deviation: 10 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_restart_interval: 16 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_block_restart_interval: 1 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: metadata_block_size: 4096 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: partition_filters: 0 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: use_delta_encoding: 1 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: filter_policy: bloomfilter 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: whole_key_filtering: 1 2024-03-21T08:20:19.640 INFO:journalctl@ceph.mon.b.smithi084.stdout: verify_compression: 0 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: read_amp_bytes_per_bit: 0 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: format_version: 5 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: enable_index_compression: 1 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_align: 0 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: max_auto_readahead_size: 262144 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: prepopulate_block_cache: 0 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: initial_auto_readahead_size: 8192 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T08:20:19.641 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression: NoCompression 2024-03-21T08:20:19.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T08:20:19.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T08:20:19.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T08:20:19.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.num_levels: 7 2024-03-21T08:20:19.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T08:20:19.643 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:20:19.644 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T08:20:19.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T08:20:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T08:20:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.table_properties_collectors: 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.inplace_update_support: 0 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T08:20:19.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.bloom_locality: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.max_successive_merges: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.ttl: 2592000 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.enable_blob_files: false 2024-03-21T08:20:19.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.min_blob_size: 0 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T08:20:19.650 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 59bf619b-fa40-4df9-aae9-1e24dbbcef3f 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009219407393, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009219408547, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711009219, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "59bf619b-fa40-4df9-aae9-1e24dbbcef3f", "db_session_id": "F8GC7NTFF9BU7H7B5UVH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711009219408854, "job": 1, "event": "recovery_finished"} 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558381144000 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: DB pointer 0x55838112e000 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T08:20:19.651 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** DB Stats ** 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** Compaction Stats [default] ** 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T08:20:19.652 INFO:journalctl@ceph.mon.b.smithi084.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** Compaction Stats [default] ** 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T08:20:19.653 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: Block cache BinnedLRUCache@0x558381051090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-21T08:20:19.654 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: using public_addr v2:172.21.15.84:0/0 -> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: starting mon.b rank -1 at public addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] at bind addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(???) e0 preinit fsid 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 bash[35457]: 9f1b9919572d1ded4e001299cfd906db9a7e46637aaf18858ef3648525ccc14e 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 systemd[1]: Started Ceph mon.b for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).mds e1 new map 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout: e1 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout: legacy client fscid: -1 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-03-21T08:20:19.655 INFO:journalctl@ceph.mon.b.smithi084.stdout: No filesystems configured 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:19.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Deploying cephadm binary to smithi084 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Added host smithi084 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Deploying cephadm binary to smithi171 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:19.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Added host smithi171 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3399190596' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3399190596' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c;count:3 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Deploying daemon mon.c on smithi171 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/3589045064' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.a calling monitor election 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.c calling monitor election 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: fsmap 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mgrmap e13: x(active, since 68s) 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: overall HEALTH_OK 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:19.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:19.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: Deploying daemon mon.b on smithi084 2024-03-21T08:20:19.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:19.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/4268930079' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:19.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:19.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:19 smithi084 ceph-mon[35477]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-21T08:20:24.486 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.483+0000 7fe60a000700 1 -- 172.21.15.171:0/2779144336 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1275 (secure 0 0 0) 0x7fe5f800f540 con 0x7fe60c19dad0 2024-03-21T08:20:24.487 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-21T08:20:24.487 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","modified":"2024-03-21T08:20:19.459928Z","created":"2024-03-21T08:17:46.599812Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T08:20:24.489 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe5f0039030 msgr2=0x7fe5f003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:24.489 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe5f0039030 0x7fe5f003b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009fd0 tx=0x7fe5fc0076a0 comp rx=0 tx=0).stop 2024-03-21T08:20:24.489 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c19dad0 msgr2=0x7fe60c197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:24.489 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c19dad0 0x7fe60c197660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b700 tx=0x7fe5f800ba10 comp rx=0 tx=0).stop 2024-03-21T08:20:24.489 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 shutdown_connections 2024-03-21T08:20:24.490 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe5f0039030 0x7fe5f003b4f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:24.490 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe60c102e10 0x7fe60c19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:24.490 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 --2- 172.21.15.171:0/2779144336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60c19dad0 0x7fe60c197660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:24.490 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 >> 172.21.15.171:0/2779144336 conn(0x7fe60c0fc640 msgr2=0x7fe60c105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:24.490 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 shutdown_connections 2024-03-21T08:20:24.490 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:24.487+0000 7fe614400700 1 -- 172.21.15.171:0/2779144336 wait complete. 2024-03-21T08:20:24.491 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-03-21T08:20:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:20:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: mon.a calling monitor election 2024-03-21T08:20:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: mon.c calling monitor election 2024-03-21T08:20:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/2779144336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: fsmap 2024-03-21T08:20:24.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: mgrmap e13: x(active, since 79s) 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: mon.b (rank 2) addr [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] is down (out of quorum) 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:24.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:24.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: mon.a calling monitor election 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: mon.c calling monitor election 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/2779144336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: fsmap 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: mgrmap e13: x(active, since 79s) 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: mon.b (rank 2) addr [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] is down (out of quorum) 2024-03-21T08:20:24.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:24.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:24.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:24.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:25.408 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T08:20:25.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T08:20:25.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:25.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:25.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:25.864 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:20:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.776 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:26.776 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:20:26.776 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.a calling monitor election 2024-03-21T08:20:26.776 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.c calling monitor election 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/2779144336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: fsmap 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:26.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mgrmap e13: x(active, since 79s) 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.b (rank 2) addr [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] is down (out of quorum) 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.b calling monitor election 2024-03-21T08:20:26.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.b calling monitor election 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.a calling monitor election 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.c calling monitor election 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: fsmap 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: mgrmap e13: x(active, since 81s) 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: Cluster is now healthy 2024-03-21T08:20:26.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:26 smithi084 ceph-mon[35477]: overall HEALTH_OK 2024-03-21T08:20:26.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: mon.b calling monitor election 2024-03-21T08:20:26.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: mon.b calling monitor election 2024-03-21T08:20:26.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: mon.a calling monitor election 2024-03-21T08:20:26.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: mon.c calling monitor election 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: fsmap 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: mgrmap e13: x(active, since 81s) 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: Cluster is now healthy 2024-03-21T08:20:26.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:26 smithi171 ceph-mon[35388]: overall HEALTH_OK 2024-03-21T08:20:26.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: mon.b calling monitor election 2024-03-21T08:20:26.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: mon.b calling monitor election 2024-03-21T08:20:26.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: mon.a calling monitor election 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: mon.c calling monitor election 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: fsmap 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: mgrmap e13: x(active, since 81s) 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: Cluster is now healthy 2024-03-21T08:20:26.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:26 smithi003 ceph-mon[31675]: overall HEALTH_OK 2024-03-21T08:20:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.746+0000 7f41c4800700 1 -- 172.21.15.3:0/325688401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc102d20 msgr2=0x7f41bc103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.746+0000 7f41c4800700 1 --2- 172.21.15.3:0/325688401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc102d20 0x7f41bc103100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009910 tx=0x7f41b0009c20 comp rx=0 tx=0).stop 2024-03-21T08:20:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.746+0000 7f41c4800700 1 -- 172.21.15.3:0/325688401 shutdown_connections 2024-03-21T08:20:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.746+0000 7f41c4800700 1 --2- 172.21.15.3:0/325688401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc102d20 0x7f41bc103100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.746+0000 7f41c4800700 1 -- 172.21.15.3:0/325688401 >> 172.21.15.3:0/325688401 conn(0x7f41bc0776d0 msgr2=0x7f41bc0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.747+0000 7f41c4800700 1 -- 172.21.15.3:0/325688401 shutdown_connections 2024-03-21T08:20:27.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.747+0000 7f41c4800700 1 -- 172.21.15.3:0/325688401 wait complete. 2024-03-21T08:20:27.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 Processor -- start 2024-03-21T08:20:27.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 -- start start 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 0x7f41bc19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41bc19fc50 0x7f41bc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc199e10 0x7f41bc19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc10c0c0 con 0x7f41bc199e10 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc10bf40 con 0x7f41bc102d20 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.748+0000 7f41c4800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc10c240 con 0x7f41bc19fc50 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c2800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc199e10 0x7f41bc19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c2800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc199e10 0x7f41bc19a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45300/0 (socket says 172.21.15.3:45300) 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c2800700 1 -- 172.21.15.3:0/990748782 learned_addr learned my addr 172.21.15.3:0/990748782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:20:27.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c1400700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41bc19fc50 0x7f41bc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c1e00700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 0x7f41bc19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c1e00700 1 -- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 msgr2=0x7f41bc19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c1e00700 1 -- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 msgr2=0x7f41bc19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c1e00700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 0x7f41bc19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c1400700 1 -- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 msgr2=0x7f41bc19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.749+0000 7f41c1e00700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 0x7f41bc19f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41c1400700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 0x7f41bc19f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41c1400700 1 -- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc199e10 msgr2=0x7f41bc19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41c1400700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc199e10 0x7f41bc19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41c1400700 1 -- 172.21.15.3:0/990748782 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc1a65e0 con 0x7f41bc19fc50 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41c2800700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc199e10 0x7f41bc19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41c1400700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41bc19fc50 0x7f41bc1998d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00b550 tx=0x7f41ac00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41b6a00700 1 -- 172.21.15.3:0/990748782 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac004230 con 0x7f41bc19fc50 2024-03-21T08:20:27.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.750+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc1a6810 con 0x7f41bc19fc50 2024-03-21T08:20:27.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.751+0000 7f41b6a00700 1 -- 172.21.15.3:0/990748782 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41ac0043d0 con 0x7f41bc19fc50 2024-03-21T08:20:27.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.751+0000 7f41b6a00700 1 -- 172.21.15.3:0/990748782 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac011b80 con 0x7f41bc19fc50 2024-03-21T08:20:27.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.751+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1a6da0 con 0x7f41bc19fc50 2024-03-21T08:20:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.752+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc069e60 con 0x7f41bc19fc50 2024-03-21T08:20:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.752+0000 7f41b6a00700 1 -- 172.21.15.3:0/990748782 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f41ac0036a0 con 0x7f41bc19fc50 2024-03-21T08:20:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.752+0000 7f41b6a00700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f41a40396b0 0x7f41a403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.753+0000 7f41b6a00700 1 -- 172.21.15.3:0/990748782 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f41ac04bda0 con 0x7f41bc19fc50 2024-03-21T08:20:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.753+0000 7f41c1e00700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f41a40396b0 0x7f41a403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:27.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.754+0000 7f41c1e00700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f41a40396b0 0x7f41a403bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009750 tx=0x7f41b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:27.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:27.761+0000 7f41b6a00700 1 -- 172.21.15.3:0/990748782 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41ac0151c0 con 0x7f41bc19fc50 2024-03-21T08:20:27.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: Updating smithi003:/etc/ceph/ceph.conf 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:20:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.018+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f41bc02da30 con 0x7f41bc19fc50 2024-03-21T08:20:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.019+0000 7f41b6a00700 1 -- 172.21.15.3:0/990748782 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7f41ac021e00 con 0x7f41bc19fc50 2024-03-21T08:20:28.021 INFO:teuthology.orchestra.run.smithi003.stdout:# minimal ceph.conf for 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:20:28.021 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-03-21T08:20:28.021 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:20:28.021 INFO:teuthology.orchestra.run.smithi003.stdout: mon_host = [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f41a40396b0 msgr2=0x7f41a403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f41a40396b0 0x7f41a403bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009750 tx=0x7f41b001a040 comp rx=0 tx=0).stop 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41bc19fc50 msgr2=0x7f41bc1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41bc19fc50 0x7f41bc1998d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00b550 tx=0x7f41ac00b860 comp rx=0 tx=0).stop 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 shutdown_connections 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f41a40396b0 0x7f41a403bb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41bc199e10 0x7f41bc19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41bc102d20 0x7f41bc19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 --2- 172.21.15.3:0/990748782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41bc19fc50 0x7f41bc1998d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 >> 172.21.15.3:0/990748782 conn(0x7f41bc0776d0 msgr2=0x7f41bc108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:28.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.022+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 shutdown_connections 2024-03-21T08:20:28.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:28.023+0000 7f41c4800700 1 -- 172.21.15.3:0/990748782 wait complete. 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: Updating smithi003:/etc/ceph/ceph.conf 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:20:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: Updating smithi003:/etc/ceph/ceph.conf 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:20:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:20:28.604 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:20:28.461+0000 7f46d3c00700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-21T08:20:28.657 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T08:20:28.657 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:20:28.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T08:20:28.741 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:20:28.741 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:28.856 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/990748782' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.857 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:28.867 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:20:28.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T08:20:28.969 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:20:28.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:29.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:29.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:29.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:29.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:29.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/990748782' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:29.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.094 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.094 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.094 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.094 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.094 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.094 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.105 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:20:29.105 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T08:20:29.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:29.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:29.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:29.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:20:29.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/990748782' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:29.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.204 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:20:29.204 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:20:29.340 INFO:tasks.cephadm:Adding mgr.x on smithi003 2024-03-21T08:20:29.340 INFO:tasks.cephadm:Adding mgr.y on smithi084 2024-03-21T08:20:29.341 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-03-21T08:20:29.341 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi003=x;smithi084=y;smithi171=z' 2024-03-21T08:20:29.818 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:29.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:29.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:29 smithi003 ceph-mon[31675]: Reconfiguring daemon mon.a on smithi003 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:30.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:29 smithi171 ceph-mon[35388]: Reconfiguring daemon mon.a on smithi003 2024-03-21T08:20:30.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T08:20:30.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:30.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:30.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:30.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:29 smithi084 ceph-mon[35477]: Reconfiguring daemon mon.a on smithi003 2024-03-21T08:20:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:30 smithi003 ceph-mon[31675]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:30.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:30.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:30.970 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:30 smithi171 ceph-mon[35388]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:30.970 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.970 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:30.970 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:30.970 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:30.970 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:30 smithi084 ceph-mon[35477]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:31.827 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.825+0000 7efe14200700 1 -- 172.21.15.171:0/1894782467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 msgr2=0x7efe0c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:31.827 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.825+0000 7efe14200700 1 --2- 172.21.15.171:0/1894782467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 0x7efe0c074620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efe00009910 tx=0x7efe00009c20 comp rx=0 tx=0).stop 2024-03-21T08:20:31.827 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.825+0000 7efe14200700 1 -- 172.21.15.171:0/1894782467 shutdown_connections 2024-03-21T08:20:31.827 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.825+0000 7efe14200700 1 --2- 172.21.15.171:0/1894782467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 0x7efe0c074620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:31.827 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.825+0000 7efe14200700 1 -- 172.21.15.171:0/1894782467 >> 172.21.15.171:0/1894782467 conn(0x7efe0c0fd260 msgr2=0x7efe0c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:31.827 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.826+0000 7efe14200700 1 -- 172.21.15.171:0/1894782467 shutdown_connections 2024-03-21T08:20:31.827 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.826+0000 7efe14200700 1 -- 172.21.15.171:0/1894782467 wait complete. 2024-03-21T08:20:31.828 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.827+0000 7efe14200700 1 Processor -- start 2024-03-21T08:20:31.828 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.827+0000 7efe14200700 1 -- start start 2024-03-21T08:20:31.829 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe14200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 0x7efe0c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:31.829 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe14200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe0c19df80 0x7efe0c1a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:31.829 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe14200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe0c19e400 0x7efe0c1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:31.829 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe14200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c10e2a0 con 0x7efe0c0761c0 2024-03-21T08:20:31.829 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe14200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c105870 con 0x7efe0c19e400 2024-03-21T08:20:31.829 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe14200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c10e420 con 0x7efe0c19df80 2024-03-21T08:20:31.830 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe10e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe0c19df80 0x7efe0c1a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:31.830 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe10e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe0c19df80 0x7efe0c1a2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:38774/0 (socket says 172.21.15.171:38774) 2024-03-21T08:20:31.830 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe10e00700 1 -- 172.21.15.171:0/2529301222 learned_addr learned my addr 172.21.15.171:0/2529301222 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:20:31.830 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe11800700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 0x7efe0c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:31.830 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.828+0000 7efe12200700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe0c19e400 0x7efe0c1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.829+0000 7efe10e00700 1 -- 172.21.15.171:0/2529301222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe0c19e400 msgr2=0x7efe0c1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.829+0000 7efe10e00700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe0c19e400 0x7efe0c1a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.829+0000 7efe10e00700 1 -- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 msgr2=0x7efe0c19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.829+0000 7efe10e00700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 0x7efe0c19da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.829+0000 7efe10e00700 1 -- 172.21.15.171:0/2529301222 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c1a5e50 con 0x7efe0c19df80 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.829+0000 7efe11800700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 0x7efe0c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.829+0000 7efe10e00700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe0c19df80 0x7efe0c1a2ff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b780 tx=0x7efdf800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.830+0000 7efe0a000700 1 -- 172.21.15.171:0/2529301222 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8004230 con 0x7efe0c19df80 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.830+0000 7efe0a000700 1 -- 172.21.15.171:0/2529301222 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efdf80043d0 con 0x7efe0c19df80 2024-03-21T08:20:31.831 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.830+0000 7efe0a000700 1 -- 172.21.15.171:0/2529301222 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8011b80 con 0x7efe0c19df80 2024-03-21T08:20:31.832 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.830+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c1a6140 con 0x7efe0c19df80 2024-03-21T08:20:31.832 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.830+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c1a66b0 con 0x7efe0c19df80 2024-03-21T08:20:31.832 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.831+0000 7efe0a000700 1 -- 172.21.15.171:0/2529301222 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7efdf80036a0 con 0x7efe0c19df80 2024-03-21T08:20:31.832 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.831+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c0637c0 con 0x7efe0c19df80 2024-03-21T08:20:31.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.832+0000 7efe0a000700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efdf40396b0 0x7efdf403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:31.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.832+0000 7efe0a000700 1 -- 172.21.15.171:0/2529301222 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efdf804bfd0 con 0x7efe0c19df80 2024-03-21T08:20:31.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.832+0000 7efe11800700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efdf40396b0 0x7efdf403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:31.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.833+0000 7efe11800700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efdf40396b0 0x7efdf403bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe000052c0 tx=0x7efe0001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:31.839 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:31.838+0000 7efe0a000700 1 -- 172.21.15.171:0/2529301222 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efdf801e1a0 con 0x7efe0c19df80 2024-03-21T08:20:31.972 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:31 smithi084 ceph-mon[35477]: Reconfiguring mon.b (monmap changed)... 2024-03-21T08:20:31.972 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:31 smithi084 ceph-mon[35477]: Reconfiguring daemon mon.b on smithi084 2024-03-21T08:20:32.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.073+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi084=y;smithi171=z", "target": ["mon-mgr", ""]}) v1 -- 0x7efe0c0625d0 con 0x7efdf40396b0 2024-03-21T08:20:32.075 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:31 smithi171 ceph-mon[35388]: Reconfiguring mon.b (monmap changed)... 2024-03-21T08:20:32.075 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:31 smithi171 ceph-mon[35388]: Reconfiguring daemon mon.b on smithi084 2024-03-21T08:20:32.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.088+0000 7efe0a000700 1 -- 172.21.15.171:0/2529301222 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efe0c0625d0 con 0x7efdf40396b0 2024-03-21T08:20:32.089 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-03-21T08:20:32.092 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efdf40396b0 msgr2=0x7efdf403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:32.092 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efdf40396b0 0x7efdf403bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe000052c0 tx=0x7efe0001a040 comp rx=0 tx=0).stop 2024-03-21T08:20:32.092 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe0c19df80 msgr2=0x7efe0c1a2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:32.092 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe0c19df80 0x7efe0c1a2ff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b780 tx=0x7efdf800bb40 comp rx=0 tx=0).stop 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 shutdown_connections 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efdf40396b0 0x7efdf403bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe0c0761c0 0x7efe0c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe0c19e400 0x7efe0c1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 --2- 172.21.15.171:0/2529301222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe0c19df80 0x7efe0c1a2ff0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 >> 172.21.15.171:0/2529301222 conn(0x7efe0c0fd260 msgr2=0x7efe0c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.091+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 shutdown_connections 2024-03-21T08:20:32.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:20:32.092+0000 7efe14200700 1 -- 172.21.15.171:0/2529301222 wait complete. 2024-03-21T08:20:32.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:31 smithi003 ceph-mon[31675]: Reconfiguring mon.b (monmap changed)... 2024-03-21T08:20:32.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:31 smithi003 ceph-mon[31675]: Reconfiguring daemon mon.b on smithi084 2024-03-21T08:20:32.793 DEBUG:teuthology.orchestra.run.smithi084:mgr.y> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.y.service 2024-03-21T08:20:32.801 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.z.service 2024-03-21T08:20:32.807 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T08:20:32.808 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:20:32.808 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T08:20:32.871 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T08:20:32.871 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-03-21T08:20:32.928 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T08:20:32.928 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:32.929 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-21T08:20:32.929 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:32.929 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:32.929 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-03-21 08:19:21.717347746 +0000 2024-03-21T08:20:32.929 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-03-21 08:13:00.087875212 +0000 2024-03-21T08:20:32.929 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-03-21 08:13:00.087875212 +0000 2024-03-21T08:20:32.929 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-03-21 08:13:00.087875212 +0000 2024-03-21T08:20:32.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T08:20:33.024 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-03-21T08:20:33.025 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-03-21T08:20:33.025 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00039094 s, 1.3 MB/s 2024-03-21T08:20:33.029 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T08:20:33.087 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-03-21T08:20:33.138 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T08:20:33.138 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:33.138 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-21T08:20:33.138 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:33.138 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:33.138 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-03-21 08:19:21.780346926 +0000 2024-03-21T08:20:33.138 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-03-21 08:13:00.677867425 +0000 2024-03-21T08:20:33.139 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-03-21 08:13:00.677867425 +0000 2024-03-21T08:20:33.139 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-03-21 08:13:00.677867425 +0000 2024-03-21T08:20:33.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T08:20:33.226 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-03-21T08:20:33.226 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-03-21T08:20:33.226 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000380521 s, 1.3 MB/s 2024-03-21T08:20:33.231 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T08:20:33.291 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-03-21T08:20:33.343 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T08:20:33.343 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:33.343 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-21T08:20:33.343 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:33.343 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:33.343 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-03-21 08:19:21.831346262 +0000 2024-03-21T08:20:33.344 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-03-21 08:13:01.257859769 +0000 2024-03-21T08:20:33.344 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-03-21 08:13:01.257859769 +0000 2024-03-21T08:20:33.344 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-03-21 08:13:01.257859769 +0000 2024-03-21T08:20:33.344 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi084=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: Saving service mgr spec with placement smithi003=x;smithi084=y;smithi171=z;count:3 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:33.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi084=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: Saving service mgr spec with placement smithi003=x;smithi084=y;smithi171=z;count:3 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:33.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi084=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: Saving service mgr spec with placement smithi003=x;smithi084=y;smithi171=z;count:3 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:20:33.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:33.425 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-03-21T08:20:33.425 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-03-21T08:20:33.426 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000351224 s, 1.5 MB/s 2024-03-21T08:20:33.430 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T08:20:33.489 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-03-21 08:19:21.894345442 +0000 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-03-21 08:13:01.851851929 +0000 2024-03-21T08:20:33.540 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-03-21 08:13:01.851851929 +0000 2024-03-21T08:20:33.541 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-03-21 08:13:01.851851929 +0000 2024-03-21T08:20:33.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T08:20:33.628 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-03-21T08:20:33.629 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-03-21T08:20:33.629 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00038556 s, 1.3 MB/s 2024-03-21T08:20:33.633 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T08:20:33.694 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-03-21T08:20:33.694 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T08:20:33.750 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T08:20:33.750 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-03-21T08:20:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T08:20:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-21T08:20:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-03-21 08:20:18.615203452 +0000 2024-03-21T08:20:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-03-21 08:12:50.312246111 +0000 2024-03-21T08:20:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-03-21 08:12:50.312246111 +0000 2024-03-21T08:20:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-03-21 08:12:50.312246111 +0000 2024-03-21T08:20:33.802 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T08:20:33.892 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-03-21T08:20:33.892 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-03-21T08:20:33.892 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000366402 s, 1.4 MB/s 2024-03-21T08:20:33.896 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T08:20:33.951 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-03-21 08:20:18.616203434 +0000 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-03-21 08:12:50.869235876 +0000 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-03-21 08:12:50.869235876 +0000 2024-03-21T08:20:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-03-21 08:12:50.869235876 +0000 2024-03-21T08:20:34.002 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T08:20:34.086 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-03-21T08:20:34.087 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-03-21T08:20:34.087 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000360401 s, 1.4 MB/s 2024-03-21T08:20:34.091 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T08:20:34.146 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-03-21T08:20:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-03-21 08:20:18.617203416 +0000 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-03-21 08:12:51.456225089 +0000 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-03-21 08:12:51.456225089 +0000 2024-03-21T08:20:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-03-21 08:12:51.456225089 +0000 2024-03-21T08:20:34.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T08:20:34.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:34 smithi084 ceph-mon[35477]: Reconfiguring mon.c (monmap changed)... 2024-03-21T08:20:34.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:34 smithi084 ceph-mon[35477]: Reconfiguring daemon mon.c on smithi171 2024-03-21T08:20:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-03-21T08:20:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-03-21T08:20:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000345802 s, 1.5 MB/s 2024-03-21T08:20:34.278 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T08:20:34.331 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-03-21T08:20:34.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:34 smithi171 ceph-mon[35388]: Reconfiguring mon.c (monmap changed)... 2024-03-21T08:20:34.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:34 smithi171 ceph-mon[35388]: Reconfiguring daemon mon.c on smithi171 2024-03-21T08:20:34.381 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T08:20:34.381 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:34.381 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-21T08:20:34.381 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:34.381 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:34.381 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-03-21 08:20:18.618203398 +0000 2024-03-21T08:20:34.381 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-03-21 08:12:52.031214523 +0000 2024-03-21T08:20:34.382 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-03-21 08:12:52.031214523 +0000 2024-03-21T08:20:34.382 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-03-21 08:12:52.031214523 +0000 2024-03-21T08:20:34.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T08:20:34.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:34 smithi003 ceph-mon[31675]: Reconfiguring mon.c (monmap changed)... 2024-03-21T08:20:34.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:34 smithi003 ceph-mon[31675]: Reconfiguring daemon mon.c on smithi171 2024-03-21T08:20:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-03-21T08:20:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-03-21T08:20:34.467 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000337533 s, 1.5 MB/s 2024-03-21T08:20:34.471 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T08:20:34.525 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-21T08:20:34.525 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T08:20:34.579 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T08:20:34.580 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-03-21T08:20:34.626 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T08:20:34.626 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:34.626 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-21T08:20:34.626 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:34.626 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:34.626 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-21 08:20:07.296571936 +0000 2024-03-21T08:20:34.626 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-21 08:12:50.953119754 +0000 2024-03-21T08:20:34.627 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-21 08:12:50.953119754 +0000 2024-03-21T08:20:34.627 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-21 08:12:50.953119754 +0000 2024-03-21T08:20:34.627 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T08:20:34.712 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-21T08:20:34.712 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-21T08:20:34.712 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.00042916 s, 1.2 MB/s 2024-03-21T08:20:34.716 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T08:20:34.763 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-03-21T08:20:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T08:20:34.852 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:34.853 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-21T08:20:34.853 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:34.853 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:34.853 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-21 08:20:07.296571936 +0000 2024-03-21T08:20:34.853 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-21 08:12:51.543108293 +0000 2024-03-21T08:20:34.853 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-21 08:12:51.543108293 +0000 2024-03-21T08:20:34.854 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-21 08:12:51.543108293 +0000 2024-03-21T08:20:34.854 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T08:20:34.931 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-21T08:20:34.932 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-21T08:20:34.932 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000313193 s, 1.6 MB/s 2024-03-21T08:20:34.935 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T08:20:34.977 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-03-21T08:20:35.057 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T08:20:35.057 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:35.057 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-21T08:20:35.057 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:35.057 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:35.057 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-21 08:20:07.297571916 +0000 2024-03-21T08:20:35.058 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-21 08:12:52.110097278 +0000 2024-03-21T08:20:35.058 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-21 08:12:52.110097278 +0000 2024-03-21T08:20:35.058 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-21 08:12:52.110097278 +0000 2024-03-21T08:20:35.058 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T08:20:35.182 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-21T08:20:35.182 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-21T08:20:35.182 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000329119 s, 1.6 MB/s 2024-03-21T08:20:35.187 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T08:20:35.236 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-03-21T08:20:35.282 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-21 08:20:07.297571916 +0000 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-21 08:12:52.671086380 +0000 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-21 08:12:52.671086380 +0000 2024-03-21T08:20:35.283 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-21 08:12:52.671086380 +0000 2024-03-21T08:20:35.283 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T08:20:35.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:35.367 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-21T08:20:35.367 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-21T08:20:35.367 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000352021 s, 1.5 MB/s 2024-03-21T08:20:35.372 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:35.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:20:35.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:35.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:35.423 INFO:tasks.cephadm:Deploying osd.0 on smithi003 with /dev/vg_nvme/lv_4... 2024-03-21T08:20:35.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T08:20:35.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:35.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:35.881 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:20:36.306 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:36 smithi003 ceph-mon[31675]: Deploying daemon mgr.z on smithi171 2024-03-21T08:20:36.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:36 smithi084 ceph-mon[35477]: Deploying daemon mgr.z on smithi171 2024-03-21T08:20:36.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:36 smithi171 ceph-mon[35388]: Deploying daemon mgr.z on smithi171 2024-03-21T08:20:37.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:37 smithi084 ceph-mon[35477]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:37.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:37 smithi171 ceph-mon[35388]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:37.418 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:37 smithi003 ceph-mon[31675]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:37.884 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:37 smithi171 systemd[1]: Starting Ceph mgr.z for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:20:38.201 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:37 smithi171 podman[37219]: 2024-03-21 08:20:37.825144095 +0000 UTC m=+0.035028241 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:20:38.202 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:37 smithi171 podman[37219]: 2024-03-21 08:20:37.930135958 +0000 UTC m=+0.140020113 container create 2645ed24aaae1b1b9a49fb19b16532413b458e3002836d63b67c62b757f61fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T08:20:38.202 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:38 smithi171 podman[37219]: 2024-03-21 08:20:38.200103853 +0000 UTC m=+0.409988065 container init 2645ed24aaae1b1b9a49fb19b16532413b458e3002836d63b67c62b757f61fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-21T08:20:38.635 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:38 smithi171 podman[37219]: 2024-03-21 08:20:38.210336827 +0000 UTC m=+0.420220966 container start 2645ed24aaae1b1b9a49fb19b16532413b458e3002836d63b67c62b757f61fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:20:38.645 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:38 smithi171 ceph-mgr[37238]: -- 172.21.15.171:0/31391644 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f149a45a0 con 0x563f14978c00 2024-03-21T08:20:38.645 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:38 smithi171 bash[37219]: 2645ed24aaae1b1b9a49fb19b16532413b458e3002836d63b67c62b757f61fbc 2024-03-21T08:20:38.645 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:38 smithi171 systemd[1]: Started Ceph mgr.z for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:20:38.650 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T08:20:38.650 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T08:20:38.650 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-03-21T08:20:38.650 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-03-21T08:20:38.650 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0363889 s, 288 MB/s 2024-03-21T08:20:38.651 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:38 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:38.946+0000 7f5c62a63200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:39.065+0000 7f5c62a63200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:20:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:39.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:39.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:39.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:39.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:39.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:20:39.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:39.389 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:39.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi003:vg_nvme/lv_4 2024-03-21T08:20:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:20:40.129 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:39.857+0000 7f5c62a63200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:20:40.129 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:39.977+0000 7f5c62a63200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:20:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:40 smithi171 ceph-mon[35388]: Deploying daemon mgr.y on smithi084 2024-03-21T08:20:40.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:40.161+0000 7f5c62a63200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:20:40.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:40.283+0000 7f5c62a63200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:20:40.394 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:40 smithi084 ceph-mon[35477]: Deploying daemon mgr.y on smithi084 2024-03-21T08:20:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:40 smithi003 ceph-mon[31675]: Deploying daemon mgr.y on smithi084 2024-03-21T08:20:40.724 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:40.392+0000 7f5c62a63200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:20:41.095 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:40.722+0000 7f5c62a63200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:20:41.095 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:40.861+0000 7f5c62a63200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:20:41.095 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:40.970+0000 7f5c62a63200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:20:41.210 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:41 smithi084 ceph-mon[35477]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:41.235 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:41 smithi003 ceph-mon[31675]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:41.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:41 smithi171 ceph-mon[35388]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:41.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:41.092+0000 7f5c62a63200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:20:41.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:41.309+0000 7f5c62a63200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:20:41.553 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 systemd[1]: Starting Ceph mgr.y for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:20:41.818 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 podman[36832]: 2024-03-21 08:20:41.45549267 +0000 UTC m=+0.032999149 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:20:41.818 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 podman[36832]: 2024-03-21 08:20:41.561864417 +0000 UTC m=+0.139370920 container create 8fa6f4e455c9e770e487c633613b7be5ddfa88d5cf93b194f1b07c40db44c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:20:41.818 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 podman[36832]: 2024-03-21 08:20:41.81581034 +0000 UTC m=+0.393316863 container init 8fa6f4e455c9e770e487c633613b7be5ddfa88d5cf93b194f1b07c40db44c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-21T08:20:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.024+0000 7f35da200700 1 -- 172.21.15.3:0/416434382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d4103d60 msgr2=0x7f35d410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.024+0000 7f35da200700 1 --2- 172.21.15.3:0/416434382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d4103d60 0x7f35d410a610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b0d0 tx=0x7f35cc00b3e0 comp rx=0 tx=0).stop 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.025+0000 7f35da200700 1 -- 172.21.15.3:0/416434382 shutdown_connections 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.025+0000 7f35da200700 1 --2- 172.21.15.3:0/416434382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d4103d60 0x7f35d410a610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.025+0000 7f35da200700 1 --2- 172.21.15.3:0/416434382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35d41021c0 0x7f35d41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.025+0000 7f35da200700 1 --2- 172.21.15.3:0/416434382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35d41033c0 0x7f35d4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.025+0000 7f35da200700 1 -- 172.21.15.3:0/416434382 >> 172.21.15.3:0/416434382 conn(0x7f35d40fd7a0 msgr2=0x7f35d40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.025+0000 7f35da200700 1 -- 172.21.15.3:0/416434382 shutdown_connections 2024-03-21T08:20:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.025+0000 7f35da200700 1 -- 172.21.15.3:0/416434382 wait complete. 2024-03-21T08:20:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.026+0000 7f35da200700 1 Processor -- start 2024-03-21T08:20:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35da200700 1 -- start start 2024-03-21T08:20:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35da200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d41021c0 0x7f35d419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35da200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35d41033c0 0x7f35d419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35da200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35d4103d60 0x7f35d41a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35da200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d410dcc0 con 0x7f35d41021c0 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35da200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d410db40 con 0x7f35d41033c0 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35da200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d410de40 con 0x7f35d4103d60 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35d3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d41021c0 0x7f35d419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35d3400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d41021c0 0x7f35d419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52474/0 (socket says 172.21.15.3:52474) 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35d3400700 1 -- 172.21.15.3:0/2303264271 learned_addr learned my addr 172.21.15.3:0/2303264271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.027+0000 7f35d3e00700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35d4103d60 0x7f35d41a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.028+0000 7f35d2a00700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35d41033c0 0x7f35d419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.028+0000 7f35d3400700 1 -- 172.21.15.3:0/2303264271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35d4103d60 msgr2=0x7f35d41a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.028+0000 7f35d3400700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35d4103d60 0x7f35d41a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.028+0000 7f35d3400700 1 -- 172.21.15.3:0/2303264271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35d41033c0 msgr2=0x7f35d419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:20:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.028+0000 7f35d3400700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35d41033c0 0x7f35d419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:20:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.028+0000 7f35d3400700 1 -- 172.21.15.3:0/2303264271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d41a21e0 con 0x7f35d41021c0 2024-03-21T08:20:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.028+0000 7f35d3400700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d41021c0 0x7f35d419a580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e9a0 tx=0x7f35c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.029+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4004220 con 0x7f35d41021c0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.029+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f35c40043c0 con 0x7f35d41021c0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.029+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d41a24d0 con 0x7f35d41021c0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.029+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4003ce0 con 0x7f35d41021c0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.029+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d41a2a10 con 0x7f35d41021c0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.030+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 46349+0+0 (secure 0 0 0) 0x7f35c400c810 con 0x7f35d41021c0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.030+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d4068e50 con 0x7f35d41021c0 2024-03-21T08:20:42.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.030+0000 7f35cbe00700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35b8039460 0x7f35b803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:20:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.030+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f35c404c6d0 con 0x7f35d41021c0 2024-03-21T08:20:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.031+0000 7f35d2a00700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35b8039460 0x7f35b803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:20:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.031+0000 7f35d2a00700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35b8039460 0x7f35b803b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009c60 tx=0x7f35c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:20:42.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.037+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35c4016330 con 0x7f35d41021c0 2024-03-21T08:20:42.126 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 podman[36832]: 2024-03-21 08:20:41.826203415 +0000 UTC m=+0.403709948 container start 8fa6f4e455c9e770e487c633613b7be5ddfa88d5cf93b194f1b07c40db44c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:20:42.126 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 ceph-mgr[36851]: -- 172.21.15.84:0/2479594360 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ba37d5e5a0 con 0x55ba37d32800 2024-03-21T08:20:42.127 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 bash[36832]: 8fa6f4e455c9e770e487c633613b7be5ddfa88d5cf93b194f1b07c40db44c1fa 2024-03-21T08:20:42.127 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:41 smithi084 systemd[1]: Started Ceph mgr.y for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:20:42.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:42.283+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f35d4075ae0 con 0x7f35b8039460 2024-03-21T08:20:42.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:42.578+0000 7fd1228df200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:20:42.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:42.700+0000 7fd1228df200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:20:42.884 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:42.619+0000 7f5c62a63200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:20:42.884 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:42.797+0000 7f5c62a63200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:43.235+0000 7f5c62a63200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:43.365+0000 7f5c62a63200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:20:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:20:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:43.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:43.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:20:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:20:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:43.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:43.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:43.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:20:43.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:20:43.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:43.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:43.865 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:43.545+0000 7fd1228df200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:20:43.865 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:43.666+0000 7fd1228df200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:20:43.865 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:43.855+0000 7fd1228df200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:20:43.883 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:43.624+0000 7f5c62a63200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:20:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:44 smithi084 ceph-mon[35477]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:44.129 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:43.977+0000 7fd1228df200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:20:44.129 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:44.087+0000 7fd1228df200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:20:44.139 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:43.880+0000 7f5c62a63200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:20:44.139 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:44 smithi171 ceph-mon[35388]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:44.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:44 smithi003 ceph-mon[31675]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:20:44.581 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:44.258+0000 7f5c62a63200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:20:44.694 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:44.433+0000 7fd1228df200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:20:44.694 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:44.577+0000 7fd1228df200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:20:44.882 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:44.579+0000 7f5c62a63200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:20:45.042 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:44.691+0000 7fd1228df200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:20:45.042 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:44.816+0000 7fd1228df200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:20:45.042 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:45.040+0000 7fd1228df200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:20:45.134 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:44.879+0000 7f5c62a63200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:20:45.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:45 smithi171 ceph-mon[35388]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:45.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:45 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:45 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:45 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:45 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:45 smithi003 ceph-mon[31675]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:45 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:45 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:45 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:45 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:45 smithi084 ceph-mon[35477]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:45 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:45 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:45 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:45 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:46.252 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:45.915+0000 7f5c62a63200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:20:46.516 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:46.347+0000 7fd1228df200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:20:46.517 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:46.517 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:46.517 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:46.517 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:46.517 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:46.517 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T08:20:46.517 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: Reconfiguring daemon mgr.x on smithi003 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3665789408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]: dispatch 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]: dispatch 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]': finished 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T08:20:46.518 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:20:46.631 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:46.628+0000 7f5c62a63200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:20:46.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:46.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:46.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:46.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:46.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:46.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T08:20:46.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: Reconfiguring daemon mgr.x on smithi003 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3665789408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]: dispatch 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]: dispatch 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]': finished 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T08:20:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:20:46.651 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: Reconfiguring daemon mgr.x on smithi003 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3665789408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]: dispatch 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]: dispatch 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94762ab2-9404-4097-911f-0f6a15f4788a"}]': finished 2024-03-21T08:20:46.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T08:20:46.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:46 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:20:46.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:46.514+0000 7fd1228df200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:20:46.883 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:46.756+0000 7f5c62a63200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:20:47.265 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:46.924+0000 7fd1228df200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:20:47.265 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:47.040+0000 7fd1228df200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:47.276+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 14) v1 ==== 92073+0+0 (secure 0 0 0) 0x7f35c4018cd0 con 0x7f35d41021c0 2024-03-21T08:20:47.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:20:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:20:47.033+0000 7f5c62a63200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:20:47.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:47 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4252200088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:20:47.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:47 smithi171 ceph-mon[35388]: Standby manager daemon z started 2024-03-21T08:20:47.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:47 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:20:47.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:47 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:20:47.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:47 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:20:47.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:47 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:20:47.530 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:47.289+0000 7fd1228df200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:20:47.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4252200088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:20:47.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-mon[35477]: Standby manager daemon z started 2024-03-21T08:20:47.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:20:47.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:20:47.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:20:47.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:20:47.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:47 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4252200088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:20:47.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:47 smithi003 ceph-mon[31675]: Standby manager daemon z started 2024-03-21T08:20:47.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:47 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:20:47.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:47 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:20:47.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:47 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:20:47.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:47 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/1075513867' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:20:47.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:47.528+0000 7fd1228df200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:20:48.209 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:47.894+0000 7fd1228df200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:20:48.209 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:48.206+0000 7fd1228df200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:20:48.509 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:48 smithi084 ceph-mon[35477]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:48.509 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:48 smithi084 ceph-mon[35477]: mgrmap e14: x(active, since 102s), standbys: z 2024-03-21T08:20:48.509 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:20:48.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:48 smithi171 ceph-mon[35388]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:48.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:48 smithi171 ceph-mon[35388]: mgrmap e14: x(active, since 102s), standbys: z 2024-03-21T08:20:48.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:20:48.668 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:48 smithi003 ceph-mon[31675]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:48.668 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:48 smithi003 ceph-mon[31675]: mgrmap e14: x(active, since 102s), standbys: z 2024-03-21T08:20:48.668 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:20:48.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:48.506+0000 7fd1228df200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:20:49.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:49.511+0000 7fd1228df200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:20:50.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:50 smithi084 ceph-mon[35477]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:50.593 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:50.191+0000 7fd1228df200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:20:50.594 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:50.316+0000 7fd1228df200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:20:50.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:50 smithi171 ceph-mon[35388]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:50.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:50 smithi003 ceph-mon[31675]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:50.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:20:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:20:50.590+0000 7fd1228df200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:20:51.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:20:51.306+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 15) v1 ==== 137797+0+0 (secure 0 0 0) 0x7f35c40173d0 con 0x7f35d41021c0 2024-03-21T08:20:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:51 smithi003 ceph-mon[31675]: Standby manager daemon y started 2024-03-21T08:20:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:51 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:20:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:51 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:20:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:51 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:20:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:51 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:20:51.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:51 smithi084 ceph-mon[35477]: Standby manager daemon y started 2024-03-21T08:20:51.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:51 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:20:51.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:51 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:20:51.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:51 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:20:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:51 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:20:51.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:51 smithi171 ceph-mon[35388]: Standby manager daemon y started 2024-03-21T08:20:51.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:51 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:20:51.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:51 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:20:51.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:51 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:20:51.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:51 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2172286992' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:20:52.474 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:52 smithi003 ceph-mon[31675]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:52.475 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:52 smithi003 ceph-mon[31675]: mgrmap e15: x(active, since 106s), standbys: z, y 2024-03-21T08:20:52.475 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:20:52.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:52 smithi084 ceph-mon[35477]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:52.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:52 smithi084 ceph-mon[35477]: mgrmap e15: x(active, since 106s), standbys: z, y 2024-03-21T08:20:52.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:20:52.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:52 smithi171 ceph-mon[35388]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:52.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:52 smithi171 ceph-mon[35388]: mgrmap e15: x(active, since 106s), standbys: z, y 2024-03-21T08:20:52.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:20:53.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:53.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:53.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:53.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:53.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:53.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:53.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:53.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:20:53.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:53.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:20:53.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:20:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:54 smithi084 ceph-mon[35477]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:54 smithi171 ceph-mon[35388]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:54.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:54 smithi003 ceph-mon[31675]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:55.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T08:20:55.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:55.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T08:20:55.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:55.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T08:20:55.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:20:56.665 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:56 smithi003 ceph-mon[31675]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:56.665 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:56 smithi003 ceph-mon[31675]: Deploying daemon osd.0 on smithi003 2024-03-21T08:20:56.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:56 smithi084 ceph-mon[35477]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:56.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:56 smithi084 ceph-mon[35477]: Deploying daemon osd.0 on smithi003 2024-03-21T08:20:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:56 smithi171 ceph-mon[35388]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:56 smithi171 ceph-mon[35388]: Deploying daemon osd.0 on smithi003 2024-03-21T08:20:58.526 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:20:58 smithi003 ceph-mon[31675]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:58.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:20:58 smithi084 ceph-mon[35477]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:20:58.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:20:58 smithi171 ceph-mon[35388]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:00 smithi003 ceph-mon[31675]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:00.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:00 smithi084 ceph-mon[35477]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:00.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:00 smithi171 ceph-mon[35388]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:02.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:02 smithi003 ceph-mon[31675]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:02.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:02 smithi084 ceph-mon[35477]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:02.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:02 smithi171 ceph-mon[35388]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:04.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:04 smithi003 ceph-mon[31675]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:04.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:04 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:04.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:04 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:04.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:04 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:04.878 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:04 smithi084 ceph-mon[35477]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:04.879 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:04 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:04.879 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:04 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:04.879 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:04 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:04.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:04 smithi171 ceph-mon[35388]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:04.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:04 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:04.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:04 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:04.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:04 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:06.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:06 smithi003 ceph-mon[31675]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:06.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:06 smithi084 ceph-mon[35477]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:06.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:06 smithi171 ceph-mon[35388]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.847+0000 7f35cbe00700 1 -- 172.21.15.3:0/2303264271 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f35d4075ae0 con 0x7f35b8039460 2024-03-21T08:21:07.850 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 0 on host 'smithi003' 2024-03-21T08:21:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35b8039460 msgr2=0x7f35b803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35b8039460 0x7f35b803b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009c60 tx=0x7f35c0009260 comp rx=0 tx=0).stop 2024-03-21T08:21:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d41021c0 msgr2=0x7f35d419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d41021c0 0x7f35d419a580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e9a0 tx=0x7f35c400ecb0 comp rx=0 tx=0).stop 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 shutdown_connections 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35b8039460 0x7f35b803b920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35d41021c0 0x7f35d419a580 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35d4103d60 0x7f35d41a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 --2- 172.21.15.3:0/2303264271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35d41033c0 0x7f35d419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 >> 172.21.15.3:0/2303264271 conn(0x7f35d40fd7a0 msgr2=0x7f35d40743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.851+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 shutdown_connections 2024-03-21T08:21:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:21:07.852+0000 7f35da200700 1 -- 172.21.15.3:0/2303264271 wait complete. 2024-03-21T08:21:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:08.689 DEBUG:teuthology.orchestra.run.smithi003:osd.0> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T08:21:08.694 INFO:tasks.cephadm:Deploying osd.1 on smithi084 with /dev/vg_nvme/lv_4... 2024-03-21T08:21:08.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T08:21:08.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:08 smithi084 ceph-mon[35477]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:08.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:08 smithi084 ceph-mon[35477]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:21:08.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:08 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:08.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:08 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:08.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:08 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:08.860 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:08 smithi003 ceph-mon[31675]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:08.860 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:08 smithi003 ceph-mon[31675]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:21:08.861 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:08 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:08.861 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:08 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:08.861 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:08 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:08 smithi171 ceph-mon[35388]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:08 smithi171 ceph-mon[35388]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:21:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:21:09.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:09 smithi084 ceph-mon[35477]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T08:21:09.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:09 smithi084 ceph-mon[35477]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T08:21:09.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:09 smithi084 ceph-mon[35477]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:21:09.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:09 smithi171 ceph-mon[35388]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T08:21:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:09 smithi171 ceph-mon[35388]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T08:21:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:09 smithi171 ceph-mon[35388]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:21:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:09.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:09 smithi003 ceph-mon[31675]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T08:21:09.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:09 smithi003 ceph-mon[31675]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T08:21:09.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:09 smithi003 ceph-mon[31675]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:21:09.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:10.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-mon[31675]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:10.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-mon[31675]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:21:10.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-mon[31675]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T08:21:10.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:10.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:10.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-mon[31675]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' 2024-03-21T08:21:10.678 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0[46192]: 2024-03-21T08:21:10.478+0000 7f1a15c00700 -1 osd.0 0 waiting for initial osdmap 2024-03-21T08:21:10.678 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:21:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0[46192]: 2024-03-21T08:21:10.500+0000 7f1a0b000700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:21:10.711 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:10 smithi084 ceph-mon[35477]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:10.711 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:10 smithi084 ceph-mon[35477]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:21:10.712 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:10 smithi084 ceph-mon[35477]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T08:21:10.712 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:10 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:10.712 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:10 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:10.712 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:10 smithi084 ceph-mon[35477]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' 2024-03-21T08:21:10.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:10 smithi171 ceph-mon[35388]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:10.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:10 smithi171 ceph-mon[35388]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:21:10.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:10 smithi171 ceph-mon[35388]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T08:21:10.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:10 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:10.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:10 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:10.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:10 smithi171 ceph-mon[35388]: from='osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195]' entity='osd.0' 2024-03-21T08:21:11.809 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T08:21:11.809 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T08:21:11.809 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-03-21T08:21:11.809 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-03-21T08:21:11.810 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0360618 s, 291 MB/s 2024-03-21T08:21:11.810 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-03-21T08:21:11.810 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:11 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:21:11.810 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:11 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:21:11.810 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:11 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:11.810 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:11 smithi084 ceph-mon[35477]: osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] boot 2024-03-21T08:21:11.810 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:11 smithi084 ceph-mon[35477]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T08:21:11.810 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:11 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:11.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:11 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:21:11.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:11 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:21:11.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:11 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:11.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:11 smithi171 ceph-mon[35388]: osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] boot 2024-03-21T08:21:11.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:11 smithi171 ceph-mon[35388]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T08:21:11.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:11 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:11.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:11 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:21:11.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:11 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:21:11.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:11 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:11.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:11 smithi003 ceph-mon[31675]: osd.0 [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] boot 2024-03-21T08:21:11.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:11 smithi003 ceph-mon[31675]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T08:21:11.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:11 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:21:12.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi084:vg_nvme/lv_4 2024-03-21T08:21:12.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:12 smithi084 ceph-mon[35477]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:12.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:12 smithi171 ceph-mon[35388]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:12.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:12 smithi003 ceph-mon[31675]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:21:12.982 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:13.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:13.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:13.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:14.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:14 smithi084 ceph-mon[35477]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:14.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:14 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi003 2024-03-21T08:21:14.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:14 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi003 to 17095M 2024-03-21T08:21:14.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:14 smithi171 ceph-mon[35388]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:14.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:14 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi003 2024-03-21T08:21:14.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:14 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi003 to 17095M 2024-03-21T08:21:14.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:14 smithi003 ceph-mon[31675]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:14.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:14 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi003 2024-03-21T08:21:14.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:14 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi003 to 17095M 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.945+0000 7fcb15400700 1 -- 172.21.15.84:0/2619303765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105fb0 msgr2=0x7fcb1010c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.945+0000 7fcb15400700 1 --2- 172.21.15.84:0/2619303765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105fb0 0x7fcb1010c860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800b0d0 tx=0x7fcb0800b3e0 comp rx=0 tx=0).stop 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.945+0000 7fcb15400700 1 -- 172.21.15.84:0/2619303765 shutdown_connections 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.945+0000 7fcb15400700 1 --2- 172.21.15.84:0/2619303765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb10104410 0x7fcb10104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.945+0000 7fcb15400700 1 --2- 172.21.15.84:0/2619303765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105fb0 0x7fcb1010c860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.946+0000 7fcb15400700 1 --2- 172.21.15.84:0/2619303765 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb10105610 0x7fcb10105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.946+0000 7fcb15400700 1 -- 172.21.15.84:0/2619303765 >> 172.21.15.84:0/2619303765 conn(0x7fcb100ff9b0 msgr2=0x7fcb10101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.946+0000 7fcb15400700 1 -- 172.21.15.84:0/2619303765 shutdown_connections 2024-03-21T08:21:14.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.946+0000 7fcb15400700 1 -- 172.21.15.84:0/2619303765 wait complete. 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 Processor -- start 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 -- start start 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb10104410 0x7fcb1019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105610 0x7fcb1019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb10105fb0 0x7fcb101a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1010fea0 con 0x7fcb10105fb0 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1010fd20 con 0x7fcb10105610 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb15400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10110020 con 0x7fcb10104410 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.947+0000 7fcb0de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105610 0x7fcb1019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:14.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105610 0x7fcb1019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41680/0 (socket says 172.21.15.84:41680) 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 -- 172.21.15.84:0/1505416286 learned_addr learned my addr 172.21.15.84:0/1505416286 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0e800700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb10104410 0x7fcb1019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0f200700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb10105fb0 0x7fcb101a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 -- 172.21.15.84:0/1505416286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb10104410 msgr2=0x7fcb1019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb10104410 0x7fcb1019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 -- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb10105fb0 msgr2=0x7fcb101a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb10105fb0 0x7fcb101a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 -- 172.21.15.84:0/1505416286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb101a20e0 con 0x7fcb10105610 2024-03-21T08:21:14.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb0de00700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105610 0x7fcb1019a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e9c0 tx=0x7fcafc00ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:21:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.948+0000 7fcb07400700 1 -- 172.21.15.84:0/1505416286 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00cbd0 con 0x7fcb10105610 2024-03-21T08:21:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.949+0000 7fcb07400700 1 -- 172.21.15.84:0/1505416286 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcafc00cd70 con 0x7fcb10105610 2024-03-21T08:21:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.949+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb101a23d0 con 0x7fcb10105610 2024-03-21T08:21:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.949+0000 7fcb07400700 1 -- 172.21.15.84:0/1505416286 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc010430 con 0x7fcb10105610 2024-03-21T08:21:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.949+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb101a2910 con 0x7fcb10105610 2024-03-21T08:21:14.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.950+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb10068e50 con 0x7fcb10105610 2024-03-21T08:21:14.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.952+0000 7fcb07400700 1 -- 172.21.15.84:0/1505416286 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 15) v1 ==== 137797+0+0 (secure 0 0 0) 0x7fcafc004750 con 0x7fcb10105610 2024-03-21T08:21:14.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.954+0000 7fcb07400700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fcaf40a2fa0 0x7fcaf40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:14.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.954+0000 7fcb07400700 1 -- 172.21.15.84:0/1505416286 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fcafc0cdfe0 con 0x7fcb10105610 2024-03-21T08:21:14.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.954+0000 7fcb0e800700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fcaf40a2fa0 0x7fcaf40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:14.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.956+0000 7fcb0e800700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fcaf40a2fa0 0x7fcaf40a5460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb10105470 tx=0x7fcb00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:21:14.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:14.957+0000 7fcb07400700 1 -- 172.21.15.84:0/1505416286 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcafc053740 con 0x7fcb10105610 2024-03-21T08:21:15.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:15.174+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb100625d0 con 0x7fcaf40a2fa0 2024-03-21T08:21:15.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:21:15.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:21:15.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:15.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:21:15.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:21:15.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:15.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:21:15.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:21:15.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:16 smithi084 ceph-mon[35477]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:21:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:16 smithi084 ceph-mon[35477]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:16.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:16 smithi171 ceph-mon[35388]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:21:16.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:16 smithi171 ceph-mon[35388]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:16.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:16 smithi003 ceph-mon[31675]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:21:16.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:16 smithi003 ceph-mon[31675]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:18 smithi084 ceph-mon[35477]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:18.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:18 smithi171 ceph-mon[35388]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:18 smithi003 ceph-mon[31675]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:19.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:19 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/2001983054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]: dispatch 2024-03-21T08:21:19.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:19 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]: dispatch 2024-03-21T08:21:19.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:19 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]': finished 2024-03-21T08:21:19.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:19 smithi171 ceph-mon[35388]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T08:21:19.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:19 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:19.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/2001983054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]: dispatch 2024-03-21T08:21:19.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:19 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]: dispatch 2024-03-21T08:21:19.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:19 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]': finished 2024-03-21T08:21:19.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:19 smithi084 ceph-mon[35477]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T08:21:19.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:19 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/2001983054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]: dispatch 2024-03-21T08:21:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:19 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]: dispatch 2024-03-21T08:21:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:19 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae4ad7a-eb28-42c8-a31a-26241b629dae"}]': finished 2024-03-21T08:21:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:19 smithi003 ceph-mon[31675]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T08:21:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:19 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:20.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:20 smithi084 ceph-mon[35477]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:20.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:20 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/3298201777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:21:20.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:20 smithi171 ceph-mon[35388]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:20.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:20 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/3298201777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:21:20.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:20 smithi003 ceph-mon[31675]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:20.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:20 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/3298201777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:21:22.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:22 smithi084 ceph-mon[35477]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:22.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:22 smithi171 ceph-mon[35388]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:22.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:22 smithi003 ceph-mon[31675]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:24.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:24 smithi084 ceph-mon[35477]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:24.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:24 smithi171 ceph-mon[35388]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:24.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:24 smithi003 ceph-mon[31675]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:26.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:26 smithi084 ceph-mon[35477]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:26.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:26 smithi171 ceph-mon[35388]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:26.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:26 smithi003 ceph-mon[31675]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:27.854 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T08:21:27.854 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:27.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T08:21:27.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T08:21:28.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:28.757 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:28 smithi084 ceph-mon[35477]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:28.757 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:28 smithi084 ceph-mon[35477]: Deploying daemon osd.1 on smithi084 2024-03-21T08:21:28.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:28 smithi003 ceph-mon[31675]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:28.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:28 smithi003 ceph-mon[31675]: Deploying daemon osd.1 on smithi084 2024-03-21T08:21:29.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:28 smithi171 ceph-mon[35388]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:29.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:28 smithi171 ceph-mon[35388]: Deploying daemon osd.1 on smithi084 2024-03-21T08:21:30.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:30 smithi084 ceph-mon[35477]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:30.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:30 smithi003 ceph-mon[31675]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:31.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:30 smithi171 ceph-mon[35388]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:32.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:32 smithi084 ceph-mon[35477]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:32 smithi003 ceph-mon[31675]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:32 smithi171 ceph-mon[35388]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:34.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:34 smithi003 ceph-mon[31675]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:35.085 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:34 smithi084 ceph-mon[35477]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:34 smithi171 ceph-mon[35388]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:36.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:36 smithi084 ceph-mon[35477]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:36.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:36.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:36.899 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:37.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:36 smithi171 ceph-mon[35388]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:37.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:37.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:37.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:37.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:36 smithi003 ceph-mon[31675]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:37.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:37.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:37.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:38 smithi084 ceph-mon[35477]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:38 smithi171 ceph-mon[35388]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:39.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:38 smithi003 ceph-mon[31675]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:40.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:40.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:40.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:40 smithi084 ceph-mon[35477]: from='osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:21:40.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:40 smithi084 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:21:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:40 smithi171 ceph-mon[35388]: from='osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:21:40.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:40 smithi171 ceph-mon[35388]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:21:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:40 smithi003 ceph-mon[31675]: from='osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:21:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:40 smithi003 ceph-mon[31675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:21:40.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.579+0000 7fcb07400700 1 -- 172.21.15.84:0/1505416286 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb100625d0 con 0x7fcaf40a2fa0 2024-03-21T08:21:40.582 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 1 on host 'smithi084' 2024-03-21T08:21:40.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fcaf40a2fa0 msgr2=0x7fcaf40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:40.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fcaf40a2fa0 0x7fcaf40a5460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb10105470 tx=0x7fcb00009260 comp rx=0 tx=0).stop 2024-03-21T08:21:40.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105610 msgr2=0x7fcb1019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:40.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105610 0x7fcb1019a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e9c0 tx=0x7fcafc00ed80 comp rx=0 tx=0).stop 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 shutdown_connections 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fcaf40a2fa0 0x7fcaf40a5460 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb10105fb0 0x7fcb101a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb10104410 0x7fcb1019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 --2- 172.21.15.84:0/1505416286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb10105610 0x7fcb1019a9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.583+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 >> 172.21.15.84:0/1505416286 conn(0x7fcb100ff9b0 msgr2=0x7fcb10108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.584+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 shutdown_connections 2024-03-21T08:21:40.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:21:40.584+0000 7fcb15400700 1 -- 172.21.15.84:0/1505416286 wait complete. 2024-03-21T08:21:41.357 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:41.357 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T08:21:41.357 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T08:21:41.357 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:41.357 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: from='osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:21:41.357 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:21:41.357 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:41.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:41.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:41.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:41.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T08:21:41.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T08:21:41.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:41.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: from='osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:21:41.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:21:41.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:41.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:41.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:41.421 DEBUG:teuthology.orchestra.run.smithi084:osd.1> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T08:21:41.425 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2024-03-21T08:21:41.425 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: from='osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:21:41.863 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:21:42.312 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:42 smithi171 ceph-mon[35388]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:21:42.312 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:42 smithi171 ceph-mon[35388]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T08:21:42.312 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:42.312 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:42.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:42 smithi084 ceph-mon[35477]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:21:42.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:42 smithi084 ceph-mon[35477]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T08:21:42.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:42.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:42.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:42 smithi003 ceph-mon[31675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:21:42.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:42 smithi003 ceph-mon[31675]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T08:21:42.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:42.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:43 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:21:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:43 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:21:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:43 smithi171 ceph-mon[35388]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:43 smithi171 ceph-mon[35388]: osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] boot 2024-03-21T08:21:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:43 smithi171 ceph-mon[35388]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T08:21:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:43 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:21:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:43 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:21:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:43 smithi003 ceph-mon[31675]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:43 smithi003 ceph-mon[31675]: osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] boot 2024-03-21T08:21:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:43 smithi003 ceph-mon[31675]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T08:21:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:43.485 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:43 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:21:43.485 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:43 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:21:43.485 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:43 smithi084 ceph-mon[35477]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:21:43.485 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:43 smithi084 ceph-mon[35477]: osd.1 [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] boot 2024-03-21T08:21:43.485 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:43 smithi084 ceph-mon[35477]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T08:21:43.485 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:21:44.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:44 smithi003 ceph-mon[31675]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T08:21:44.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:44 smithi003 ceph-mon[31675]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:44.581 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:44 smithi171 ceph-mon[35388]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T08:21:44.582 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:44 smithi171 ceph-mon[35388]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:44.582 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T08:21:44.582 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T08:21:44.582 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-21T08:21:44.582 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-21T08:21:44.582 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350435 s, 299 MB/s 2024-03-21T08:21:44.582 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 2024-03-21T08:21:44.582 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-21T08:21:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:44 smithi084 ceph-mon[35477]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T08:21:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:44 smithi084 ceph-mon[35477]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:45.259 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-03-21T08:21:45.670 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:21:47.266 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:47.266 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi084 2024-03-21T08:21:47.266 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.267 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.267 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:21:47.267 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi084 to 17091M 2024-03-21T08:21:47.267 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.267 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:47.267 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:47.267 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:47.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi084 2024-03-21T08:21:47.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:21:47.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi084 to 17091M 2024-03-21T08:21:47.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:47.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:47.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi084 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi084 to 17091M 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:21:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:21:47.759 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.756+0000 7f3342400700 1 -- 172.21.15.171:0/2053777908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 msgr2=0x7f333c10c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:47.759 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.756+0000 7f3342400700 1 --2- 172.21.15.171:0/2053777908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c10c860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f333400b0a0 tx=0x7f333400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:21:47.759 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.757+0000 7f3342400700 1 -- 172.21.15.171:0/2053777908 shutdown_connections 2024-03-21T08:21:47.759 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.757+0000 7f3342400700 1 --2- 172.21.15.171:0/2053777908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f333c104410 0x7f333c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:47.759 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.757+0000 7f3342400700 1 --2- 172.21.15.171:0/2053777908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c10c860 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:47.759 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.757+0000 7f3342400700 1 --2- 172.21.15.171:0/2053777908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f333c105610 0x7f333c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:47.759 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.757+0000 7f3342400700 1 -- 172.21.15.171:0/2053777908 >> 172.21.15.171:0/2053777908 conn(0x7f333c0ff9b0 msgr2=0x7f333c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:21:47.760 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.757+0000 7f3342400700 1 -- 172.21.15.171:0/2053777908 shutdown_connections 2024-03-21T08:21:47.760 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.757+0000 7f3342400700 1 -- 172.21.15.171:0/2053777908 wait complete. 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 Processor -- start 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 -- start start 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f333c104410 0x7f333c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f333c105610 0x7f333c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c10ff00 con 0x7f333c105610 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c10fd80 con 0x7f333c104410 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.758+0000 7f3342400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c110080 con 0x7f333c105fb0 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:47.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c1a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:33934/0 (socket says 172.21.15.171:33934) 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 -- 172.21.15.171:0/3893442766 learned_addr learned my addr 172.21.15.171:0/3893442766 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333b400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f333c104410 0x7f333c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333aa00700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f333c105610 0x7f333c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 -- 172.21.15.171:0/3893442766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f333c104410 msgr2=0x7f333c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f333c104410 0x7f333c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 -- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f333c105610 msgr2=0x7f333c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f333c105610 0x7f333c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 -- 172.21.15.171:0/3893442766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c1a2120 con 0x7f333c105fb0 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333aa00700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f333c105610 0x7f333c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.759+0000 7f333be00700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c1a1a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f333400b070 tx=0x7f333400cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.760+0000 7f3333e00700 1 -- 172.21.15.171:0/3893442766 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334021070 con 0x7f333c105fb0 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.760+0000 7f3333e00700 1 -- 172.21.15.171:0/3893442766 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3334004020 con 0x7f333c105fb0 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.760+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c1a23b0 con 0x7f333c105fb0 2024-03-21T08:21:47.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.760+0000 7f3333e00700 1 -- 172.21.15.171:0/3893442766 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333401d6b0 con 0x7f333c105fb0 2024-03-21T08:21:47.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.760+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c1a2860 con 0x7f333c105fb0 2024-03-21T08:21:47.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.761+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c068e50 con 0x7f333c105fb0 2024-03-21T08:21:47.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.765+0000 7f3333e00700 1 -- 172.21.15.171:0/3893442766 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) v1 ==== 137797+0+0 (secure 0 0 0) 0x7f3334004560 con 0x7f333c105fb0 2024-03-21T08:21:47.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.766+0000 7f3333e00700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f33200a2fa0 0x7f33200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:21:47.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.766+0000 7f3333e00700 1 -- 172.21.15.171:0/3893442766 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f33340cf1c0 con 0x7f333c105fb0 2024-03-21T08:21:47.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.767+0000 7f333b400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f33200a2fa0 0x7f33200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:21:47.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.768+0000 7f333b400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f33200a2fa0 0x7f33200a5460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f332c005950 tx=0x7f332c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:21:47.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:47.769+0000 7f3333e00700 1 -- 172.21.15.171:0/3893442766 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f333409d830 con 0x7f333c105fb0 2024-03-21T08:21:48.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:21:48.002+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f333c0625d0 con 0x7f33200a2fa0 2024-03-21T08:21:48.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:21:48.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:21:48.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:48.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:21:48.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:21:48.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:48.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:21:48.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:21:48.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:21:49.254 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:49 smithi171 ceph-mon[35388]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:49.254 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:49 smithi171 ceph-mon[35388]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:21:49.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:49 smithi084 ceph-mon[35477]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:49.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:49 smithi084 ceph-mon[35477]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:21:49.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:49 smithi003 ceph-mon[31675]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:49.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:49 smithi003 ceph-mon[31675]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:21:51.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:51 smithi084 ceph-mon[35477]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:51 smithi171 ceph-mon[35388]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:51.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:51 smithi003 ceph-mon[31675]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:52.308 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:52 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/2167946179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]: dispatch 2024-03-21T08:21:52.308 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:52 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]: dispatch 2024-03-21T08:21:52.308 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:52 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]': finished 2024-03-21T08:21:52.308 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:52 smithi171 ceph-mon[35388]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T08:21:52.308 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:21:52.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:52 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/2167946179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]: dispatch 2024-03-21T08:21:52.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:52 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]: dispatch 2024-03-21T08:21:52.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:52 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]': finished 2024-03-21T08:21:52.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:52 smithi084 ceph-mon[35477]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T08:21:52.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:21:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:52 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/2167946179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]: dispatch 2024-03-21T08:21:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:52 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]: dispatch 2024-03-21T08:21:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:52 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72b1933d-fa5d-423e-9711-162d319d2c10"}]': finished 2024-03-21T08:21:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:52 smithi003 ceph-mon[31675]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T08:21:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:21:53.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:53 smithi084 ceph-mon[35477]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:53.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:53 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/3803656542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:21:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:53 smithi171 ceph-mon[35388]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:53 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/3803656542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:21:53.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:53 smithi003 ceph-mon[31675]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:53 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/3803656542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:21:55.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:55 smithi084 ceph-mon[35477]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:55.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:55 smithi171 ceph-mon[35388]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:55.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:55 smithi003 ceph-mon[31675]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:57.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:57 smithi084 ceph-mon[35477]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:57.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:57 smithi171 ceph-mon[35388]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:57.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:57 smithi003 ceph-mon[31675]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:59.209 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:21:59 smithi171 ceph-mon[35388]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:59.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:21:59 smithi084 ceph-mon[35477]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:21:59.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:21:59 smithi003 ceph-mon[31675]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:01.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:01 smithi084 ceph-mon[35477]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:01.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:01 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T08:22:01.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:01 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:01.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:01 smithi171 ceph-mon[35388]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:01.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:01 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T08:22:01.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:01 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:01 smithi003 ceph-mon[31675]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:01.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:01 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T08:22:01.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:01 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:02.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:02 smithi084 ceph-mon[35477]: Deploying daemon osd.2 on smithi171 2024-03-21T08:22:02.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:02 smithi171 ceph-mon[35388]: Deploying daemon osd.2 on smithi171 2024-03-21T08:22:02.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:02 smithi003 ceph-mon[31675]: Deploying daemon osd.2 on smithi171 2024-03-21T08:22:03.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:03 smithi084 ceph-mon[35477]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:03.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:03 smithi171 ceph-mon[35388]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:03.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:03 smithi003 ceph-mon[31675]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:05.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:05 smithi171 ceph-mon[35388]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:05.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:05 smithi084 ceph-mon[35477]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:05.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:05 smithi003 ceph-mon[31675]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:07.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:07 smithi084 ceph-mon[35477]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:07.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:07 smithi171 ceph-mon[35388]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:07.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:07 smithi003 ceph-mon[31675]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:09 smithi084 ceph-mon[35477]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:09.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:09 smithi171 ceph-mon[35388]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:09.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:09.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:09.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:09 smithi003 ceph-mon[31675]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:11.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:11 smithi084 ceph-mon[35477]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:11.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:11 smithi171 ceph-mon[35388]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:11.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:11 smithi003 ceph-mon[31675]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:12.618 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.617+0000 7f3333e00700 1 -- 172.21.15.171:0/3893442766 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f333c0625d0 con 0x7f33200a2fa0 2024-03-21T08:22:12.618 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f33200a2fa0 msgr2=0x7f33200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f33200a2fa0 0x7f33200a5460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f332c005950 tx=0x7f332c0058e0 comp rx=0 tx=0).stop 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 msgr2=0x7f333c1a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c1a1a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f333400b070 tx=0x7f333400cf70 comp rx=0 tx=0).stop 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 shutdown_connections 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f33200a2fa0 0x7f33200a5460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f333c105fb0 0x7f333c1a1a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f333c104410 0x7f333c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:12.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 --2- 172.21.15.171:0/3893442766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f333c105610 0x7f333c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:12.624 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 >> 172.21.15.171:0/3893442766 conn(0x7f333c0ff9b0 msgr2=0x7f333c1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:22:12.624 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 shutdown_connections 2024-03-21T08:22:12.624 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:22:12.622+0000 7f3342400700 1 -- 172.21.15.171:0/3893442766 wait complete. 2024-03-21T08:22:13.378 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:13 smithi171 ceph-mon[35388]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:13.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:13 smithi171 ceph-mon[35388]: from='osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:22:13.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:13 smithi171 ceph-mon[35388]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:22:13.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:13 smithi003 ceph-mon[31675]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:13 smithi003 ceph-mon[31675]: from='osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:22:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:13 smithi003 ceph-mon[31675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:22:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.440 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T08:22:13.447 INFO:tasks.cephadm:Deploying osd.3 on smithi003 with /dev/vg_nvme/lv_3... 2024-03-21T08:22:13.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T08:22:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:13 smithi084 ceph-mon[35477]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:13 smithi084 ceph-mon[35477]: from='osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:22:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:13 smithi084 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:22:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:13.902 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:22:14.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:14 smithi003 ceph-mon[31675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T08:22:14.331 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:14 smithi003 ceph-mon[31675]: from='osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:22:14.331 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:14 smithi003 ceph-mon[31675]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T08:22:14.331 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:14 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:14.331 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:14 smithi003 ceph-mon[31675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:22:14.548 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:14 smithi171 ceph-mon[35388]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T08:22:14.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:14 smithi171 ceph-mon[35388]: from='osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:22:14.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:14 smithi171 ceph-mon[35388]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T08:22:14.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:14 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:14.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:14 smithi171 ceph-mon[35388]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:22:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:14 smithi084 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T08:22:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:14 smithi084 ceph-mon[35477]: from='osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:22:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:14 smithi084 ceph-mon[35477]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T08:22:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:14 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:14 smithi084 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:22:15.126 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2[40491]: 2024-03-21T08:22:14.964+0000 7f6721600700 -1 osd.2 0 waiting for initial osdmap 2024-03-21T08:22:15.127 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2[40491]: 2024-03-21T08:22:14.982+0000 7f6716a00700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:22:15.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:15 smithi003 ceph-mon[31675]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:15.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:15 smithi003 ceph-mon[31675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:22:15.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:15 smithi003 ceph-mon[31675]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T08:22:15.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:15.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:15.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:15 smithi003 ceph-mon[31675]: from='osd.2 ' entity='osd.2' 2024-03-21T08:22:15.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:15 smithi171 ceph-mon[35388]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:15.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:15 smithi171 ceph-mon[35388]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:22:15.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:15 smithi171 ceph-mon[35388]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T08:22:15.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:15.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:15.380 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:15 smithi171 ceph-mon[35388]: from='osd.2 ' entity='osd.2' 2024-03-21T08:22:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:15 smithi084 ceph-mon[35477]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:15 smithi084 ceph-mon[35477]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:22:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:15 smithi084 ceph-mon[35477]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T08:22:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:15 smithi084 ceph-mon[35477]: from='osd.2 ' entity='osd.2' 2024-03-21T08:22:16.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:16 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:22:16.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:16 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:22:16.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:16 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:16.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:16 smithi171 ceph-mon[35388]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:16.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:16 smithi171 ceph-mon[35388]: osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] boot 2024-03-21T08:22:16.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:16 smithi171 ceph-mon[35388]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T08:22:16.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:16 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:16 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:22:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:16 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:22:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:16 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:16 smithi003 ceph-mon[31675]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:16 smithi003 ceph-mon[31675]: osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] boot 2024-03-21T08:22:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:16 smithi003 ceph-mon[31675]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T08:22:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:16 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:16 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:22:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:16 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:22:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:16 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:16 smithi084 ceph-mon[35477]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:22:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:16 smithi084 ceph-mon[35477]: osd.2 [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] boot 2024-03-21T08:22:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:16 smithi084 ceph-mon[35477]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T08:22:16.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:16 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:22:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T08:22:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T08:22:16.934 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-03-21T08:22:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-03-21T08:22:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0371552 s, 282 MB/s 2024-03-21T08:22:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-03-21T08:22:17.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi003:vg_nvme/lv_3 2024-03-21T08:22:18.192 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:22:18.344 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:17 smithi003 ceph-mon[31675]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T08:22:18.345 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:22:18.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:17 smithi084 ceph-mon[35477]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T08:22:18.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:22:18.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:17 smithi171 ceph-mon[35388]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T08:22:18.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:22:19.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:19 smithi003 ceph-mon[31675]: pgmap v105: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:19.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:19 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T08:22:19.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:19 smithi003 ceph-mon[31675]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T08:22:19.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:19 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:22:19.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:19 smithi084 ceph-mon[35477]: pgmap v105: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:19.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T08:22:19.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:19 smithi084 ceph-mon[35477]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T08:22:19.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:22:19.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:19 smithi171 ceph-mon[35388]: pgmap v105: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:19.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:19 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T08:22:19.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:19 smithi171 ceph-mon[35388]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T08:22:19.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:19 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:22:19.926 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:19 smithi003 sudo[50848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:22:19.927 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:19 smithi003 sudo[50848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:19.927 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:19 smithi003 sudo[50848]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:19.927 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:19 smithi003 sudo[50848]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:20.370 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T08:22:20.370 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T08:22:20.370 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi171 2024-03-21T08:22:20.370 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.370 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.370 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi171 to 17091M 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:20.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:20 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi171 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi171 to 17091M 2024-03-21T08:22:20.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:20.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:20.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:20.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:20.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:20.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:20 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi171 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi171 to 17091M 2024-03-21T08:22:20.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:20.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:20.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:20.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:20.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:20.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:20 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:20.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.537+0000 7fdda7200700 1 -- 172.21.15.3:0/2909428679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8104310 msgr2=0x7fdda8104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:20.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.537+0000 7fdda7200700 1 --2- 172.21.15.3:0/2909428679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8104310 0x7fdda8104730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdd980099c0 tx=0x7fdd98009cd0 comp rx=0 tx=0).stop 2024-03-21T08:22:20.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.537+0000 7fdda7200700 1 -- 172.21.15.3:0/2909428679 shutdown_connections 2024-03-21T08:22:20.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.537+0000 7fdda7200700 1 --2- 172.21.15.3:0/2909428679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8104310 0x7fdda8104730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:20.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.537+0000 7fdda7200700 1 --2- 172.21.15.3:0/2909428679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda8105eb0 0x7fdda810c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:20.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.537+0000 7fdda7200700 1 --2- 172.21.15.3:0/2909428679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdda8105510 0x7fdda8105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:20.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.537+0000 7fdda7200700 1 -- 172.21.15.3:0/2909428679 >> 172.21.15.3:0/2909428679 conn(0x7fdda80ff8b0 msgr2=0x7fdda8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:22:20.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.538+0000 7fdda7200700 1 -- 172.21.15.3:0/2909428679 shutdown_connections 2024-03-21T08:22:20.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.538+0000 7fdda7200700 1 -- 172.21.15.3:0/2909428679 wait complete. 2024-03-21T08:22:20.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.539+0000 7fdda7200700 1 Processor -- start 2024-03-21T08:22:20.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.539+0000 7fdda7200700 1 -- start start 2024-03-21T08:22:20.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.539+0000 7fdda7200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdda8104310 0x7fdda819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:20.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda7200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda8105510 0x7fdda819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:20.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda7200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8105eb0 0x7fdda81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:20.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda7200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda810fe00 con 0x7fdda8105eb0 2024-03-21T08:22:20.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda7200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda810fc80 con 0x7fdda8104310 2024-03-21T08:22:20.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda7200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda810ff80 con 0x7fdda8105510 2024-03-21T08:22:20.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8105eb0 0x7fdda81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:20.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8105eb0 0x7fdda81a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44810/0 (socket says 172.21.15.3:44810) 2024-03-21T08:22:20.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 -- 172.21.15.3:0/1762633959 learned_addr learned my addr 172.21.15.3:0/1762633959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:22:20.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda5e00700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdda8104310 0x7fdda819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdd9fe00700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda8105510 0x7fdda819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 -- 172.21.15.3:0/1762633959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda8105510 msgr2=0x7fdda819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda8105510 0x7fdda819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 -- 172.21.15.3:0/1762633959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdda8104310 msgr2=0x7fdda819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdda8104310 0x7fdda819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdda6800700 1 -- 172.21.15.3:0/1762633959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda81a20f0 con 0x7fdda8105eb0 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.540+0000 7fdd9fe00700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda8105510 0x7fdda819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.541+0000 7fdda6800700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8105eb0 0x7fdda81a1a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdda000add0 tx=0x7fdda000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.541+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0017020 con 0x7fdda8105eb0 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.541+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdda000cdf0 con 0x7fdda8105eb0 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.541+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0005460 con 0x7fdda8105eb0 2024-03-21T08:22:20.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.541+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda81a23e0 con 0x7fdda8105eb0 2024-03-21T08:22:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.541+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda81a28d0 con 0x7fdda8105eb0 2024-03-21T08:22:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.542+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda8068e50 con 0x7fdda8105eb0 2024-03-21T08:22:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.546+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 15) v1 ==== 137797+0+0 (secure 0 0 0) 0x7fdda0004ad0 con 0x7fdda8105eb0 2024-03-21T08:22:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.547+0000 7fdd9ea00700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdd8c0a2f20 0x7fdd8c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.547+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fdda00cdd40 con 0x7fdda8105eb0 2024-03-21T08:22:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.547+0000 7fdda5e00700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdd8c0a2f20 0x7fdd8c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.548+0000 7fdda5e00700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdd8c0a2f20 0x7fdd8c0a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:22:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.550+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdda0021020 con 0x7fdda8105eb0 2024-03-21T08:22:20.676 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:20 smithi003 sudo[50860]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:22:20.676 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:20 smithi003 sudo[50860]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:20.677 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:20 smithi003 sudo[50860]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:20.677 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:22:20 smithi003 sudo[50860]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:20.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:20.788+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdda80625d0 con 0x7fdd8c0a2f20 2024-03-21T08:22:21.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:21 smithi003 ceph-mon[31675]: pgmap v108: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:21.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:21 smithi003 ceph-mon[31675]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T08:22:21.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:21.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:22:21.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:22:21.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:21.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:21 smithi084 ceph-mon[35477]: pgmap v108: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:21 smithi084 ceph-mon[35477]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T08:22:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:22:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:22:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:21.378 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43857]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:22:21.378 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43857]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:21.379 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43857]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:21.379 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43857]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:21.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:21 smithi171 ceph-mon[35388]: pgmap v108: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:21.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:21 smithi171 ceph-mon[35388]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T08:22:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:22:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:22:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:22.047 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43860]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:22:22.047 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43860]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:22.047 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43860]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:22.047 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:22:21 smithi084 sudo[43860]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:22.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:22.058+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fdda0097f80 con 0x7fdda8105eb0 2024-03-21T08:22:22.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:22 smithi084 ceph-mon[35477]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:22:22.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:22 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:22.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:22 smithi171 ceph-mon[35388]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:22:22.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:22 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:22.384 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44388]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:22:22.384 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44388]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:22.384 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44388]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:22.384 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44388]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:22 smithi003 ceph-mon[31675]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:22:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:22 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:23.134 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44391]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:22:23.134 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44391]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:23.134 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44391]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:23.134 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:22:22 smithi171 sudo[44391]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:23.571 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:23 smithi003 sudo[51049]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:22:23.571 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:23 smithi003 sudo[51049]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:23.571 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:23 smithi003 sudo[51049]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:23.571 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:23 smithi003 sudo[51049]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:24 smithi084 ceph-mon[35477]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:24 smithi084 ceph-mon[35477]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-21T08:22:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:24 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:24 smithi084 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:24.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 ceph-mon[35388]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 ceph-mon[35388]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-21T08:22:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 ceph-mon[35388]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 sudo[51066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:22:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 sudo[51066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 sudo[51066]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 sudo[51066]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 ceph-mon[31675]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 ceph-mon[31675]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-21T08:22:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:24 smithi003 ceph-mon[31675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 sudo[44394]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:22:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 sudo[44394]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 sudo[44394]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:24.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:24 smithi171 sudo[44394]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 sudo[44397]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 sudo[44397]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 sudo[44397]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 sudo[44397]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:25.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:25 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/901500847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d03f7-da4c-4c3e-b98d-1da6de58611f"}]: dispatch 2024-03-21T08:22:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:25.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:25.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:25 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/901500847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d03f7-da4c-4c3e-b98d-1da6de58611f"}]: dispatch 2024-03-21T08:22:26.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:25 smithi084 sudo[43863]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:22:26.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:25 smithi084 sudo[43863]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:26.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:25 smithi084 sudo[43863]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:26.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:25 smithi084 sudo[43863]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:26.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 sudo[43866]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 sudo[43866]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 sudo[43866]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 sudo[43866]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/901500847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d03f7-da4c-4c3e-b98d-1da6de58611f"}]: dispatch 2024-03-21T08:22:26.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/901500847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097d03f7-da4c-4c3e-b98d-1da6de58611f"}]': finished 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:26.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:26 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/896564392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/901500847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097d03f7-da4c-4c3e-b98d-1da6de58611f"}]': finished 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:26.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:26 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/896564392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/901500847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097d03f7-da4c-4c3e-b98d-1da6de58611f"}]': finished 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:22:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:26.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:26 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/896564392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:22:27.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:27 smithi084 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:27.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:27 smithi084 ceph-mon[35477]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:27.531 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:27 smithi084 ceph-mon[35477]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:27.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:27 smithi171 ceph-mon[35388]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:27.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:27 smithi171 ceph-mon[35388]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:27.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:27 smithi171 ceph-mon[35388]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:27.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:27 smithi003 ceph-mon[31675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:22:27.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:27 smithi003 ceph-mon[31675]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:27.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:27 smithi003 ceph-mon[31675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:22:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:28 smithi084 ceph-mon[35477]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:28.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:28 smithi171 ceph-mon[35388]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:28.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:28 smithi003 ceph-mon[31675]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:30.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:30 smithi084 ceph-mon[35477]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-21T08:22:30.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:30 smithi171 ceph-mon[35388]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-21T08:22:30.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:30 smithi003 ceph-mon[31675]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-21T08:22:32.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:32 smithi003 ceph-mon[31675]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-21T08:22:32.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:32 smithi084 ceph-mon[35477]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-21T08:22:32.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:32 smithi171 ceph-mon[35388]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-21T08:22:34.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:34 smithi084 ceph-mon[35477]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T08:22:34.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:34 smithi171 ceph-mon[35388]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T08:22:34.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:34 smithi003 ceph-mon[31675]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T08:22:35.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T08:22:35.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:35 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:35.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:35 smithi084 ceph-mon[35477]: Deploying daemon osd.3 on smithi003 2024-03-21T08:22:35.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T08:22:35.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:35 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:35.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:35 smithi171 ceph-mon[35388]: Deploying daemon osd.3 on smithi003 2024-03-21T08:22:35.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T08:22:35.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:35 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:35.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:35 smithi003 ceph-mon[31675]: Deploying daemon osd.3 on smithi003 2024-03-21T08:22:36.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:36 smithi003 ceph-mon[31675]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1016 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T08:22:36.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:36 smithi084 ceph-mon[35477]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1016 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T08:22:36.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:36 smithi171 ceph-mon[35388]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1016 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T08:22:38.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:38 smithi003 ceph-mon[31675]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T08:22:38.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:38 smithi084 ceph-mon[35477]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T08:22:38.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:38 smithi171 ceph-mon[35388]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T08:22:40.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:40 smithi003 ceph-mon[31675]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T08:22:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:40 smithi084 ceph-mon[35477]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T08:22:40.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:40 smithi171 ceph-mon[35388]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T08:22:42.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:42 smithi084 ceph-mon[35477]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:42.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:42 smithi171 ceph-mon[35388]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:42.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:42 smithi003 ceph-mon[31675]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:44.565 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:44 smithi003 ceph-mon[31675]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:44.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:44.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:44.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:44 smithi084 ceph-mon[35477]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:44.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:44 smithi171 ceph-mon[35388]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:44.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:44.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:44.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:46.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:46 smithi003 ceph-mon[31675]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:46.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:46 smithi084 ceph-mon[35477]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:46 smithi171 ceph-mon[35388]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:48.000 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:48.000 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:48.000 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:47 smithi003 ceph-mon[31675]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T08:22:48.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.281+0000 7fdd9ea00700 1 -- 172.21.15.3:0/1762633959 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdda80625d0 con 0x7fdd8c0a2f20 2024-03-21T08:22:48.284 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 3 on host 'smithi003' 2024-03-21T08:22:48.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.284+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdd8c0a2f20 msgr2=0x7fdd8c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:48.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.284+0000 7fdda7200700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdd8c0a2f20 0x7fdd8c0a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd98005fb0 comp rx=0 tx=0).stop 2024-03-21T08:22:48.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8105eb0 msgr2=0x7fdda81a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:48.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8105eb0 0x7fdda81a1a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdda000add0 tx=0x7fdda000c330 comp rx=0 tx=0).stop 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 shutdown_connections 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdd8c0a2f20 0x7fdd8c0a53e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdda8105eb0 0x7fdda81a1a50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdda8104310 0x7fdda819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 --2- 172.21.15.3:0/1762633959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdda8105510 0x7fdda819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 >> 172.21.15.3:0/1762633959 conn(0x7fdda80ff8b0 msgr2=0x7fdda81088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 shutdown_connections 2024-03-21T08:22:48.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:22:48.285+0000 7fdda7200700 1 -- 172.21.15.3:0/1762633959 wait complete. 2024-03-21T08:22:48.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:48.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:48.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:47 smithi084 ceph-mon[35477]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T08:22:48.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:48.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:48.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:47 smithi171 ceph-mon[35388]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T08:22:49.112 DEBUG:teuthology.orchestra.run.smithi003:osd.3> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T08:22:49.118 INFO:tasks.cephadm:Deploying osd.4 on smithi084 with /dev/vg_nvme/lv_3... 2024-03-21T08:22:49.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T08:22:49.360 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-mon[31675]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:49.360 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-mon[31675]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T08:22:49.360 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-mon[31675]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T08:22:49.360 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-mon[31675]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:22:49.361 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:49.361 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:49.361 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:49.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:49 smithi084 ceph-mon[35477]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:49.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:49 smithi084 ceph-mon[35477]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T08:22:49.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:49 smithi084 ceph-mon[35477]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T08:22:49.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:49 smithi084 ceph-mon[35477]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:22:49.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:49 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:49.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:49 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:49.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:49 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:49.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:49 smithi171 ceph-mon[35388]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:49.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:49 smithi171 ceph-mon[35388]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T08:22:49.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:49 smithi171 ceph-mon[35388]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T08:22:49.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:49 smithi171 ceph-mon[35388]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:22:49.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:49 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:49.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:49 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:49.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:49 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:49.557 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:22:50.040 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:22:49.895+0000 7f305a200700 -1 osd.3 0 waiting for initial osdmap 2024-03-21T08:22:50.040 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:22:49 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:22:49.928+0000 7f3050000700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:22:50.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:50 smithi084 ceph-mon[35477]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:22:50.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:50 smithi084 ceph-mon[35477]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T08:22:50.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:50 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:50.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:50 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:50.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:50 smithi084 ceph-mon[35477]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' 2024-03-21T08:22:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:50 smithi171 ceph-mon[35388]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:22:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:50 smithi171 ceph-mon[35388]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T08:22:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:50 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:50 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:50 smithi171 ceph-mon[35388]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' 2024-03-21T08:22:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:50 smithi003 ceph-mon[31675]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:22:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:50 smithi003 ceph-mon[31675]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T08:22:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:50 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:50.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:50 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:50.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:50 smithi003 ceph-mon[31675]: from='osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001]' entity='osd.3' 2024-03-21T08:22:51.110 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:51 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:22:51.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:51 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:22:51.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:51 smithi003 ceph-mon[31675]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:51.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:51 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:51.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:51 smithi003 ceph-mon[31675]: osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] boot 2024-03-21T08:22:51.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:51 smithi003 ceph-mon[31675]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T08:22:51.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:51 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:51.359 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:51 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:22:51.359 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:51 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:22:51.359 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:51 smithi084 ceph-mon[35477]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:51.359 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:51 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:51.360 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:51 smithi084 ceph-mon[35477]: osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] boot 2024-03-21T08:22:51.360 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:51 smithi084 ceph-mon[35477]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T08:22:51.360 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:51 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:51 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:22:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:51 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:22:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:51 smithi171 ceph-mon[35388]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:22:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:51 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:51 smithi171 ceph-mon[35388]: osd.3 [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] boot 2024-03-21T08:22:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:51 smithi171 ceph-mon[35388]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T08:22:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:51 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:22:52.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:52 smithi084 ceph-mon[35477]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T08:22:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:52 smithi171 ceph-mon[35388]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T08:22:52.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:52 smithi003 ceph-mon[31675]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T08:22:52.573 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T08:22:52.574 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T08:22:52.574 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-03-21T08:22:52.574 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-03-21T08:22:52.574 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.035149 s, 298 MB/s 2024-03-21T08:22:52.574 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-03-21T08:22:53.276 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:53 smithi084 ceph-mon[35477]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:22:53.276 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:53 smithi084 ceph-mon[35477]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T08:22:53.276 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:53 smithi084 ceph-mon[35477]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T08:22:53.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:53 smithi003 ceph-mon[31675]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:22:53.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:53 smithi003 ceph-mon[31675]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T08:22:53.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:53 smithi003 ceph-mon[31675]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T08:22:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:53 smithi171 ceph-mon[35388]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:22:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:53 smithi171 ceph-mon[35388]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T08:22:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:53 smithi171 ceph-mon[35388]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T08:22:53.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi084:vg_nvme/lv_3 2024-03-21T08:22:53.838 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:22:54.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T08:22:54.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-03-21T08:22:54.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi003 2024-03-21T08:22:54.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:54.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:54.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi003 to 8547M 2024-03-21T08:22:54.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:54.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:54.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T08:22:54.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi003 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi003 to 8547M 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T08:22:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-03-21T08:22:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi003 2024-03-21T08:22:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi003 to 8547M 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:54.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:55.442 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:55.442 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:55.442 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:55.442 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:55.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:22:55.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:55.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:22:55.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:22:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.936+0000 7f8b50200700 1 -- 172.21.15.84:0/1675624689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b48074bf0 msgr2=0x7f8b48075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.936+0000 7f8b50200700 1 --2- 172.21.15.84:0/1675624689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b48074bf0 0x7f8b48075070 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009990 tx=0x7f8b34009ca0 comp rx=0 tx=0).stop 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.937+0000 7f8b50200700 1 -- 172.21.15.84:0/1675624689 shutdown_connections 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.937+0000 7f8b50200700 1 --2- 172.21.15.84:0/1675624689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b4810e7a0 0x7f8b48110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.937+0000 7f8b50200700 1 --2- 172.21.15.84:0/1675624689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b480761c0 0x7f8b48074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.937+0000 7f8b50200700 1 --2- 172.21.15.84:0/1675624689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b48074bf0 0x7f8b48075070 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.937+0000 7f8b50200700 1 -- 172.21.15.84:0/1675624689 >> 172.21.15.84:0/1675624689 conn(0x7f8b480fd740 msgr2=0x7f8b480ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.937+0000 7f8b50200700 1 -- 172.21.15.84:0/1675624689 shutdown_connections 2024-03-21T08:22:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.937+0000 7f8b50200700 1 -- 172.21.15.84:0/1675624689 wait complete. 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 Processor -- start 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 -- start start 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48074bf0 0x7f8b4819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b480761c0 0x7f8b4819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b4810e7a0 0x7f8b481a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48114270 con 0x7f8b48074bf0 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481140f0 con 0x7f8b4810e7a0 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.938+0000 7f8b50200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481143f0 con 0x7f8b480761c0 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b4810e7a0 0x7f8b481a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b4810e7a0 0x7f8b481a5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51526/0 (socket says 172.21.15.84:51526) 2024-03-21T08:22:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 -- 172.21.15.84:0/3646725315 learned_addr learned my addr 172.21.15.84:0/3646725315 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4ce00700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b480761c0 0x7f8b4819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4d800700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48074bf0 0x7f8b4819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 -- 172.21.15.84:0/3646725315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b480761c0 msgr2=0x7f8b4819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b480761c0 0x7f8b4819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 -- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48074bf0 msgr2=0x7f8b4819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48074bf0 0x7f8b4819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4e200700 1 -- 172.21.15.84:0/3646725315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b481a65f0 con 0x7f8b4810e7a0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4ce00700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b480761c0 0x7f8b4819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.939+0000 7f8b4d800700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48074bf0 0x7f8b4819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.940+0000 7f8b4e200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b4810e7a0 0x7f8b481a5f50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b40007ee0 tx=0x7f8b4000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.940+0000 7f8b46000700 1 -- 172.21.15.84:0/3646725315 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000dbd0 con 0x7f8b4810e7a0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.940+0000 7f8b46000700 1 -- 172.21.15.84:0/3646725315 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b4000dd70 con 0x7f8b4810e7a0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.940+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b481a68e0 con 0x7f8b4810e7a0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.940+0000 7f8b46000700 1 -- 172.21.15.84:0/3646725315 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40005a00 con 0x7f8b4810e7a0 2024-03-21T08:22:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.940+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b481a6dd0 con 0x7f8b4810e7a0 2024-03-21T08:22:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.941+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48068e50 con 0x7f8b4810e7a0 2024-03-21T08:22:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.943+0000 7f8b46000700 1 -- 172.21.15.84:0/3646725315 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8b4001a020 con 0x7f8b4810e7a0 2024-03-21T08:22:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.945+0000 7f8b46000700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b300a2fb0 0x7f8b300a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:22:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.945+0000 7f8b46000700 1 -- 172.21.15.84:0/3646725315 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f8b400ce0f0 con 0x7f8b4810e7a0 2024-03-21T08:22:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.945+0000 7f8b4d800700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b300a2fb0 0x7f8b300a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:22:55.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.946+0000 7f8b4d800700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b300a2fb0 0x7f8b300a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b480765a0 tx=0x7f8b3c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:22:55.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:55.948+0000 7f8b46000700 1 -- 172.21.15.84:0/3646725315 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b40010070 con 0x7f8b4810e7a0 2024-03-21T08:22:56.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:22:56.181+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b480625d0 con 0x7f8b300a2fb0 2024-03-21T08:22:56.500 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:56 smithi003 ceph-mon[31675]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-03-21T08:22:56.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:56 smithi084 ceph-mon[35477]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-03-21T08:22:56.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:56 smithi171 ceph-mon[35388]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-03-21T08:22:57.478 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:57 smithi084 ceph-mon[35477]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:22:57.478 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:22:57.478 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:22:57.478 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:57 smithi171 ceph-mon[35388]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:22:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:22:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:22:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:57.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:57 smithi003 ceph-mon[31675]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:22:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:22:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:22:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:22:58.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:22:58 smithi084 ceph-mon[35477]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:22:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:22:58 smithi171 ceph-mon[35388]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:22:58.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:22:58 smithi003 ceph-mon[31675]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:23:00.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:00 smithi084 ceph-mon[35477]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T08:23:00.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:00 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/3479036636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]: dispatch 2024-03-21T08:23:00.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:00 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]: dispatch 2024-03-21T08:23:00.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:00 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]': finished 2024-03-21T08:23:00.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:00 smithi084 ceph-mon[35477]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T08:23:00.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:00 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:00 smithi171 ceph-mon[35388]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T08:23:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:00 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/3479036636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]: dispatch 2024-03-21T08:23:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:00 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]: dispatch 2024-03-21T08:23:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:00 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]': finished 2024-03-21T08:23:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:00 smithi171 ceph-mon[35388]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T08:23:00.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:00 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:00 smithi003 ceph-mon[31675]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T08:23:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:00 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/3479036636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]: dispatch 2024-03-21T08:23:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:00 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]: dispatch 2024-03-21T08:23:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:00 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73fe75be-a893-4d19-b3ec-9d75c169f300"}]': finished 2024-03-21T08:23:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:00 smithi003 ceph-mon[31675]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T08:23:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:01.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:01 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/2133699603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:23:01.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:01 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/2133699603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:23:01.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:01 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/2133699603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:23:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:02 smithi084 ceph-mon[35477]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:02.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:02 smithi171 ceph-mon[35388]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:02.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:02 smithi003 ceph-mon[31675]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:04.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:04 smithi084 ceph-mon[35477]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:04.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:04 smithi171 ceph-mon[35388]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:04.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:04 smithi003 ceph-mon[31675]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:06.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:06 smithi084 ceph-mon[35477]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:06 smithi171 ceph-mon[35388]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:06.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:06 smithi003 ceph-mon[31675]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:08.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:08 smithi084 ceph-mon[35477]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:08.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:08 smithi171 ceph-mon[35388]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:08.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:08 smithi003 ceph-mon[31675]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:10.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:10 smithi003 ceph-mon[31675]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:10.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:10 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T08:23:10.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:10 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:10.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:10 smithi003 ceph-mon[31675]: Deploying daemon osd.4 on smithi084 2024-03-21T08:23:10.580 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:10 smithi084 ceph-mon[35477]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:10.581 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:10 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T08:23:10.581 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:10 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:10.581 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:10 smithi084 ceph-mon[35477]: Deploying daemon osd.4 on smithi084 2024-03-21T08:23:10.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:10 smithi171 ceph-mon[35388]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:10.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:10 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T08:23:10.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:10 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:10.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:10 smithi171 ceph-mon[35388]: Deploying daemon osd.4 on smithi084 2024-03-21T08:23:12.523 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:12 smithi084 ceph-mon[35477]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:12.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:12 smithi171 ceph-mon[35388]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:12.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:12 smithi003 ceph-mon[31675]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:14.561 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:14 smithi084 ceph-mon[35477]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:14.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:14 smithi171 ceph-mon[35388]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:14.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:14 smithi003 ceph-mon[31675]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:16.558 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:16 smithi084 ceph-mon[35477]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:16 smithi171 ceph-mon[35388]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:16.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:16 smithi003 ceph-mon[31675]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:18.788 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:18 smithi084 ceph-mon[35477]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:18.788 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:18.788 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:18.788 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:18.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:18 smithi171 ceph-mon[35388]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:18.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:18.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:18.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:18.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:18 smithi003 ceph-mon[31675]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:20.550 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:20 smithi171 ceph-mon[35388]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:20.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:20 smithi003 ceph-mon[31675]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:20.709 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:20 smithi084 ceph-mon[35477]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:22.121 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:22 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:22.121 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:22 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:22.121 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:22 smithi084 ceph-mon[35477]: from='osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:23:22.121 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:22 smithi084 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:23:22.264 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.262+0000 7f8b46000700 1 -- 172.21.15.84:0/3646725315 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8b480625d0 con 0x7f8b300a2fb0 2024-03-21T08:23:22.265 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 4 on host 'smithi084' 2024-03-21T08:23:22.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.265+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b300a2fb0 msgr2=0x7f8b300a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:23:22.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.265+0000 7f8b50200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b300a2fb0 0x7f8b300a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b480765a0 tx=0x7f8b3c00b2c0 comp rx=0 tx=0).stop 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.265+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b4810e7a0 msgr2=0x7f8b481a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.265+0000 7f8b50200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b4810e7a0 0x7f8b481a5f50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b40007ee0 tx=0x7f8b4000d3b0 comp rx=0 tx=0).stop 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 shutdown_connections 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b300a2fb0 0x7f8b300a5470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48074bf0 0x7f8b4819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b4810e7a0 0x7f8b481a5f50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 --2- 172.21.15.84:0/3646725315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b480761c0 0x7f8b4819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 >> 172.21.15.84:0/3646725315 conn(0x7f8b480fd740 msgr2=0x7f8b48101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 shutdown_connections 2024-03-21T08:23:22.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:23:22.266+0000 7f8b50200700 1 -- 172.21.15.84:0/3646725315 wait complete. 2024-03-21T08:23:22.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:22 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:22.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:22 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:22.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:22 smithi171 ceph-mon[35388]: from='osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:23:22.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:22 smithi171 ceph-mon[35388]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:23:22.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:22 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:22 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:22 smithi003 ceph-mon[31675]: from='osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:23:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:22 smithi003 ceph-mon[31675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:23:23.126 DEBUG:teuthology.orchestra.run.smithi084:osd.4> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T08:23:23.134 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2024-03-21T08:23:23.134 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T08:23:23.331 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:23.331 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T08:23:23.331 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T08:23:23.331 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:23.332 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: from='osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:23:23.332 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:23:23.332 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:23.332 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: from='osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:23.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: from='osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:23.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:23.570 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:23:24.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:24 smithi084 ceph-mon[35477]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:23:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:24 smithi084 ceph-mon[35477]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T08:23:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:24 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:24 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:24 smithi171 ceph-mon[35388]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:23:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:24 smithi171 ceph-mon[35388]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T08:23:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:24 smithi003 ceph-mon[31675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:23:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:24 smithi003 ceph-mon[31675]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T08:23:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:24.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:25.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:25 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:23:25.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:25 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:23:25.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:25 smithi084 ceph-mon[35477]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:25.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:25 smithi084 ceph-mon[35477]: osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] boot 2024-03-21T08:23:25.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:25 smithi084 ceph-mon[35477]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T08:23:25.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:25.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:25 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:23:25.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:25 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:23:25.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:25 smithi171 ceph-mon[35388]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:25.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:25 smithi171 ceph-mon[35388]: osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] boot 2024-03-21T08:23:25.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:25 smithi171 ceph-mon[35388]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T08:23:25.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:25.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:25 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:23:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:25 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:23:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:25 smithi003 ceph-mon[31675]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-21T08:23:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:25 smithi003 ceph-mon[31675]: osd.4 [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] boot 2024-03-21T08:23:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:25 smithi003 ceph-mon[31675]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T08:23:25.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:23:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:26 smithi084 ceph-mon[35477]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T08:23:26.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:26 smithi171 ceph-mon[35388]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T08:23:26.389 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T08:23:26.389 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T08:23:26.389 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-21T08:23:26.389 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-21T08:23:26.389 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367081 s, 286 MB/s 2024-03-21T08:23:26.389 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-21T08:23:26.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:26 smithi003 ceph-mon[31675]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T08:23:27.279 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:27 smithi084 ceph-mon[35477]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:27.288 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-03-21T08:23:27.372 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:27 smithi171 ceph-mon[35388]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:27.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:27 smithi003 ceph-mon[31675]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:27.724 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:23:29.298 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:29.298 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi084 2024-03-21T08:23:29.298 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.298 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.298 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:23:29.298 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:23:29.298 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi084 to 8545M 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:29.299 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi084 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi084 to 8545M 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:29.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:29.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:29.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi084 2024-03-21T08:23:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi084 to 8545M 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:29.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:29.785 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 -- 172.21.15.171:0/2092524009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105eb0 msgr2=0x7f054c10c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:23:29.785 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 --2- 172.21.15.171:0/2092524009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105eb0 0x7f054c10c760 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f054400b0a0 tx=0x7f054400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:23:29.785 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 -- 172.21.15.171:0/2092524009 shutdown_connections 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 --2- 172.21.15.171:0/2092524009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105eb0 0x7f054c10c760 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 --2- 172.21.15.171:0/2092524009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c104310 0x7f054c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 --2- 172.21.15.171:0/2092524009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f054c105510 0x7f054c105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 -- 172.21.15.171:0/2092524009 >> 172.21.15.171:0/2092524009 conn(0x7f054c0ff8b0 msgr2=0x7f054c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 -- 172.21.15.171:0/2092524009 shutdown_connections 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.783+0000 7f0552c00700 1 -- 172.21.15.171:0/2092524009 wait complete. 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 Processor -- start 2024-03-21T08:23:29.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 -- start start 2024-03-21T08:23:29.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c104310 0x7f054c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:23:29.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105510 0x7f054c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:23:29.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f054c105eb0 0x7f054c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:23:29.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c10fde0 con 0x7f054c105eb0 2024-03-21T08:23:29.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c10fc60 con 0x7f054c104310 2024-03-21T08:23:29.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.785+0000 7f0552c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c10ff60 con 0x7f054c105510 2024-03-21T08:23:29.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105510 0x7f054c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105510 0x7f054c19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:54442/0 (socket says 172.21.15.171:54442) 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 -- 172.21.15.171:0/1471979510 learned_addr learned my addr 172.21.15.171:0/1471979510 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0551800700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c104310 0x7f054c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0552200700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f054c105eb0 0x7f054c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 -- 172.21.15.171:0/1471979510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c104310 msgr2=0x7f054c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c104310 0x7f054c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 -- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f054c105eb0 msgr2=0x7f054c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f054c105eb0 0x7f054c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0550e00700 1 -- 172.21.15.171:0/1471979510 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c1a2000 con 0x7f054c105510 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.786+0000 7f0552200700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f054c105eb0 0x7f054c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:23:29.789 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.787+0000 7f0550e00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105510 0x7f054c19a8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f053800d980 tx=0x7f053800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:23:29.790 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.787+0000 7f054a000700 1 -- 172.21.15.171:0/1471979510 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538016070 con 0x7f054c105510 2024-03-21T08:23:29.790 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.787+0000 7f054a000700 1 -- 172.21.15.171:0/1471979510 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0538010460 con 0x7f054c105510 2024-03-21T08:23:29.790 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.787+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c1a22f0 con 0x7f054c105510 2024-03-21T08:23:29.790 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.787+0000 7f054a000700 1 -- 172.21.15.171:0/1471979510 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800f620 con 0x7f054c105510 2024-03-21T08:23:29.790 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.787+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c1a2830 con 0x7f054c105510 2024-03-21T08:23:29.790 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.788+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c068e50 con 0x7f054c105510 2024-03-21T08:23:29.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.790+0000 7f054a000700 1 -- 172.21.15.171:0/1471979510 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f0538019070 con 0x7f054c105510 2024-03-21T08:23:29.793 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.792+0000 7f054a000700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05340a2fb0 0x7f05340a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:23:29.793 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.792+0000 7f054a000700 1 -- 172.21.15.171:0/1471979510 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f05380cdfb0 con 0x7f054c105510 2024-03-21T08:23:29.794 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.792+0000 7f0551800700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05340a2fb0 0x7f05340a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:23:29.795 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.793+0000 7f0551800700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05340a2fb0 0x7f05340a5470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0540005fd0 tx=0x7f0540005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:23:29.797 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:29.795+0000 7f054a000700 1 -- 172.21.15.171:0/1471979510 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f053809bdb0 con 0x7f054c105510 2024-03-21T08:23:30.032 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:30.030+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f054c0625d0 con 0x7f05340a2fb0 2024-03-21T08:23:31.181 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:31 smithi171 ceph-mon[35388]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:31.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:31 smithi171 ceph-mon[35388]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:23:31.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:23:31.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:23:31.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:31.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:31 smithi084 ceph-mon[35477]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:31.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:31 smithi084 ceph-mon[35477]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:23:31.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:23:31.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:23:31.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:31 smithi003 ceph-mon[31675]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:31 smithi003 ceph-mon[31675]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:23:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:23:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:23:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:33.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:33 smithi084 ceph-mon[35477]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:33.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:33 smithi171 ceph-mon[35388]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:33.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:33 smithi003 ceph-mon[31675]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:34.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:34 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/1872991008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]: dispatch 2024-03-21T08:23:34.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:34 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]: dispatch 2024-03-21T08:23:34.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:34 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]': finished 2024-03-21T08:23:34.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:34 smithi084 ceph-mon[35477]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T08:23:34.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:34.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:34 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/1872991008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]: dispatch 2024-03-21T08:23:34.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:34 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]: dispatch 2024-03-21T08:23:34.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:34 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]': finished 2024-03-21T08:23:34.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:34 smithi171 ceph-mon[35388]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T08:23:34.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:34.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:34 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/1872991008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]: dispatch 2024-03-21T08:23:34.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:34 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]: dispatch 2024-03-21T08:23:34.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:34 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51c9f7b7-be7e-46db-b99c-ea76625620dc"}]': finished 2024-03-21T08:23:34.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:34 smithi003 ceph-mon[31675]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T08:23:34.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:35 smithi084 ceph-mon[35477]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:35.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:35 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/1337139348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:23:35.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:35 smithi171 ceph-mon[35388]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:35.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:35 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/1337139348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:23:35.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:35 smithi003 ceph-mon[31675]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:35.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:35 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/1337139348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:23:37.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:37 smithi084 ceph-mon[35477]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:37.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:37 smithi171 ceph-mon[35388]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:37.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:37 smithi003 ceph-mon[31675]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:39 smithi171 ceph-mon[35388]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:39.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:39 smithi084 ceph-mon[35477]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:39.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:39 smithi003 ceph-mon[31675]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:41.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:41 smithi084 ceph-mon[35477]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:41.376 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:41 smithi171 ceph-mon[35388]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:41.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:41 smithi003 ceph-mon[31675]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:43 smithi084 ceph-mon[35477]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T08:23:43.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:43 smithi171 ceph-mon[35388]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T08:23:43.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:43.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:43 smithi003 ceph-mon[31675]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T08:23:43.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:44.293 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:44 smithi171 ceph-mon[35388]: Deploying daemon osd.5 on smithi171 2024-03-21T08:23:44.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:44 smithi084 ceph-mon[35477]: Deploying daemon osd.5 on smithi171 2024-03-21T08:23:44.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:44 smithi003 ceph-mon[31675]: Deploying daemon osd.5 on smithi171 2024-03-21T08:23:45.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:45 smithi084 ceph-mon[35477]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:45.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:45 smithi171 ceph-mon[35388]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:45.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:45 smithi003 ceph-mon[31675]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:47.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:47 smithi084 ceph-mon[35477]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:47.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:47 smithi171 ceph-mon[35388]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:47.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:47 smithi003 ceph-mon[31675]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:49.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:49 smithi084 ceph-mon[35477]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:49.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:49 smithi171 ceph-mon[35388]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:49.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:49 smithi003 ceph-mon[31675]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:51.317 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:51 smithi171 ceph-mon[35388]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:51.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:51 smithi084 ceph-mon[35477]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:51.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:51 smithi003 ceph-mon[31675]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:52.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:52.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:52.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:52 smithi171 ceph-mon[35388]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:52.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:52.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:52.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:52.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:52 smithi084 ceph-mon[35477]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:52.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:52.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:52.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:52.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:52 smithi003 ceph-mon[31675]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:54.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:54 smithi084 ceph-mon[35477]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:54.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:54 smithi171 ceph-mon[35388]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:54.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:54 smithi003 ceph-mon[31675]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:55.856 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.854+0000 7f054a000700 1 -- 172.21.15.171:0/1471979510 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f054c0625d0 con 0x7f05340a2fb0 2024-03-21T08:23:55.856 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2024-03-21T08:23:55.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05340a2fb0 msgr2=0x7f05340a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:23:55.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05340a2fb0 0x7f05340a5470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0540005fd0 tx=0x7f0540005dd0 comp rx=0 tx=0).stop 2024-03-21T08:23:55.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105510 msgr2=0x7f054c19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:23:55.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105510 0x7f054c19a8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f053800d980 tx=0x7f053800dd40 comp rx=0 tx=0).stop 2024-03-21T08:23:55.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 shutdown_connections 2024-03-21T08:23:55.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05340a2fb0 0x7f05340a5470 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:55.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f054c105eb0 0x7f054c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:55.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c104310 0x7f054c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:55.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 --2- 172.21.15.171:0/1471979510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f054c105510 0x7f054c19a8e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:23:55.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 >> 172.21.15.171:0/1471979510 conn(0x7f054c0ff8b0 msgr2=0x7f054c108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:23:55.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 shutdown_connections 2024-03-21T08:23:55.860 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:23:55.858+0000 7f0552c00700 1 -- 172.21.15.171:0/1471979510 wait complete. 2024-03-21T08:23:55.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:55.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:55.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:55.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:55.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:55.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:55 smithi003 ceph-mon[31675]: from='osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:23:55.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:55 smithi003 ceph-mon[31675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:23:56.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:56.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:56.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:55 smithi084 ceph-mon[35477]: from='osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:23:56.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:55 smithi084 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:23:56.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:23:56.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:23:56.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:55 smithi171 ceph-mon[35388]: from='osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:23:56.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:55 smithi171 ceph-mon[35388]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:23:56.683 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T08:23:56.689 INFO:tasks.cephadm:Deploying osd.6 on smithi003 with /dev/vg_nvme/lv_2... 2024-03-21T08:23:56.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T08:23:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T08:23:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T08:23:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:56.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: from='osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:23:56.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:23:56.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:56.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:56 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:57.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:57.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T08:23:57.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T08:23:57.030 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:57.030 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: from='osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:23:57.030 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:23:57.030 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:57.030 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:57.030 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:56 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:57.135 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:23:57.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:57.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T08:23:57.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T08:23:57.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:57.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: from='osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:23:57.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:23:57.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:57.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:23:57.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:56 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:23:57.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:57 smithi003 ceph-mon[31675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:23:57.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:57 smithi003 ceph-mon[31675]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T08:23:57.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:57.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:57.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:57 smithi003 ceph-mon[31675]: osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] boot 2024-03-21T08:23:57.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:57 smithi003 ceph-mon[31675]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T08:23:57.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:58.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:57 smithi084 ceph-mon[35477]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:23:58.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:57 smithi084 ceph-mon[35477]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T08:23:58.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:58.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:58.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:57 smithi084 ceph-mon[35477]: osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] boot 2024-03-21T08:23:58.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:57 smithi084 ceph-mon[35477]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T08:23:58.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:58.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:57 smithi171 ceph-mon[35388]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:23:58.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:57 smithi171 ceph-mon[35388]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T08:23:58.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:58.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:58.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:57 smithi171 ceph-mon[35388]: osd.5 [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] boot 2024-03-21T08:23:58.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:57 smithi171 ceph-mon[35388]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T08:23:58.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:23:59.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:58 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:23:59.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:58 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:23:59.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:58 smithi003 ceph-mon[31675]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:59.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:23:58 smithi003 ceph-mon[31675]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T08:23:59.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:58 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:23:59.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:58 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:23:59.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:58 smithi084 ceph-mon[35477]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:59.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:23:58 smithi084 ceph-mon[35477]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T08:23:59.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:58 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:23:59.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:58 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:23:59.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:58 smithi171 ceph-mon[35388]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:23:59.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:23:58 smithi171 ceph-mon[35388]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T08:24:00.263 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T08:24:00.263 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T08:24:00.263 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-03-21T08:24:00.263 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-03-21T08:24:00.263 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0329658 s, 318 MB/s 2024-03-21T08:24:00.263 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-03-21T08:24:00.449 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:00 smithi171 ceph-mon[35388]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T08:24:00.449 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:00 smithi171 ceph-mon[35388]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:00.581 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:00 smithi003 ceph-mon[31675]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T08:24:00.582 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:00 smithi003 ceph-mon[31675]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:00.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:00 smithi084 ceph-mon[35477]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T08:24:00.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:00 smithi084 ceph-mon[35477]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:01.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi003:vg_nvme/lv_2 2024-03-21T08:24:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:01 smithi003 ceph-mon[31675]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T08:24:01.609 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:24:01.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:01 smithi084 ceph-mon[35477]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T08:24:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:01 smithi171 ceph-mon[35388]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T08:24:02.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T08:24:02.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:02.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi171 2024-03-21T08:24:02.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi171 to 8545M 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:02.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T08:24:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi171 2024-03-21T08:24:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:02.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:02.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi171 to 8545M 2024-03-21T08:24:02.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:02.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:02.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi171 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi171 to 8545M 2024-03-21T08:24:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:02.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:02.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:02.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.862+0000 7fed15e00700 1 -- 172.21.15.3:0/2355567566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10102c50 msgr2=0x7fed10105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.862+0000 7fed15e00700 1 --2- 172.21.15.3:0/2355567566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10102c50 0x7fed10105040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fed00009990 tx=0x7fed00009ca0 comp rx=0 tx=0).stop 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.862+0000 7fed15e00700 1 -- 172.21.15.3:0/2355567566 shutdown_connections 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.862+0000 7fed15e00700 1 --2- 172.21.15.3:0/2355567566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10102c50 0x7fed10105040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.862+0000 7fed15e00700 1 --2- 172.21.15.3:0/2355567566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed10107eb0 0x7fed1010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.862+0000 7fed15e00700 1 --2- 172.21.15.3:0/2355567566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10105580 0x7fed10107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.862+0000 7fed15e00700 1 -- 172.21.15.3:0/2355567566 >> 172.21.15.3:0/2355567566 conn(0x7fed100fc660 msgr2=0x7fed100feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:24:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.863+0000 7fed15e00700 1 -- 172.21.15.3:0/2355567566 shutdown_connections 2024-03-21T08:24:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.863+0000 7fed15e00700 1 -- 172.21.15.3:0/2355567566 wait complete. 2024-03-21T08:24:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 Processor -- start 2024-03-21T08:24:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 -- start start 2024-03-21T08:24:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10102c50 0x7fed101025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed10105580 0x7fed10100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10107eb0 0x7fed10101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1010dcd0 con 0x7fed10107eb0 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1010db50 con 0x7fed10105580 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.864+0000 7fed15e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1010de50 con 0x7fed10102c50 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10107eb0 0x7fed10101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10107eb0 0x7fed10101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50344/0 (socket says 172.21.15.3:50344) 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 -- 172.21.15.3:0/843497712 learned_addr learned my addr 172.21.15.3:0/843497712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0ea00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10102c50 0x7fed101025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0e000700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed10105580 0x7fed10100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 -- 172.21.15.3:0/843497712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10102c50 msgr2=0x7fed101025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10102c50 0x7fed101025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 -- 172.21.15.3:0/843497712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed10105580 msgr2=0x7fed10100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed10105580 0x7fed10100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0f400700 1 -- 172.21.15.3:0/843497712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed101019f0 con 0x7fed10107eb0 2024-03-21T08:24:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.865+0000 7fed0e000700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed10105580 0x7fed10100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:24:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.866+0000 7fed0f400700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10107eb0 0x7fed10101140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fed08007ec0 tx=0x7fed0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:24:03.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.866+0000 7fed14a00700 1 -- 172.21.15.3:0/843497712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800dbd0 con 0x7fed10107eb0 2024-03-21T08:24:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.866+0000 7fed14a00700 1 -- 172.21.15.3:0/843497712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed0800dd70 con 0x7fed10107eb0 2024-03-21T08:24:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.866+0000 7fed14a00700 1 -- 172.21.15.3:0/843497712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08004560 con 0x7fed10107eb0 2024-03-21T08:24:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.866+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed101a2560 con 0x7fed10107eb0 2024-03-21T08:24:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.866+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed101a2a20 con 0x7fed10107eb0 2024-03-21T08:24:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.867+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed10068e50 con 0x7fed10107eb0 2024-03-21T08:24:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.870+0000 7fed14a00700 1 -- 172.21.15.3:0/843497712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fed0801a030 con 0x7fed10107eb0 2024-03-21T08:24:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.871+0000 7fed14a00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecf40a2fb0 0x7fecf40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.871+0000 7fed14a00700 1 -- 172.21.15.3:0/843497712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fed080ce190 con 0x7fed10107eb0 2024-03-21T08:24:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.871+0000 7fed0ea00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecf40a2fb0 0x7fecf40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.872+0000 7fed0ea00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecf40a2fb0 0x7fecf40a5470 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fed0000b5c0 tx=0x7fed00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:24:03.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:03.874+0000 7fed14a00700 1 -- 172.21.15.3:0/843497712 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fed08021020 con 0x7fed10107eb0 2024-03-21T08:24:04.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:04.113+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fed100625d0 con 0x7fecf40a2fb0 2024-03-21T08:24:04.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:04 smithi003 ceph-mon[31675]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-21T08:24:04.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:04 smithi003 ceph-mon[31675]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:24:04.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:04 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:24:04.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:04 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:24:04.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:04 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:04.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:04 smithi084 ceph-mon[35477]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-21T08:24:04.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:04 smithi084 ceph-mon[35477]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:24:04.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:04 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:24:04.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:04 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:24:04.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:04 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:04.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:04 smithi171 ceph-mon[35388]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-21T08:24:04.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:04 smithi171 ceph-mon[35388]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:24:04.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:04 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:24:04.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:04 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:24:04.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:04 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:06.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:06 smithi084 ceph-mon[35477]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T08:24:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:06 smithi171 ceph-mon[35388]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T08:24:06.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:06 smithi003 ceph-mon[31675]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T08:24:08.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:08 smithi084 ceph-mon[35477]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:24:08.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:08 smithi171 ceph-mon[35388]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:24:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:08 smithi003 ceph-mon[31675]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:24:09.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:09 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1077152194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68131baf-10d4-4e02-a3cc-b7ec95fe3708"}]: dispatch 2024-03-21T08:24:09.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:09 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1077152194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68131baf-10d4-4e02-a3cc-b7ec95fe3708"}]': finished 2024-03-21T08:24:09.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:09 smithi084 ceph-mon[35477]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T08:24:09.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:09.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:09 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4008357353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:24:09.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:09 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1077152194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68131baf-10d4-4e02-a3cc-b7ec95fe3708"}]: dispatch 2024-03-21T08:24:09.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:09 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1077152194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68131baf-10d4-4e02-a3cc-b7ec95fe3708"}]': finished 2024-03-21T08:24:09.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:09 smithi171 ceph-mon[35388]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T08:24:09.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:09.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:09 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4008357353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:24:09.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:09 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1077152194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68131baf-10d4-4e02-a3cc-b7ec95fe3708"}]: dispatch 2024-03-21T08:24:09.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:09 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1077152194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68131baf-10d4-4e02-a3cc-b7ec95fe3708"}]': finished 2024-03-21T08:24:09.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:09 smithi003 ceph-mon[31675]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T08:24:09.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:09.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:09 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4008357353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:24:10.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:10 smithi084 ceph-mon[35477]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:24:10.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:10 smithi171 ceph-mon[35388]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:24:10.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:10 smithi003 ceph-mon[31675]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:24:12.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:12 smithi084 ceph-mon[35477]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T08:24:12.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:12 smithi171 ceph-mon[35388]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T08:24:12.644 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:12 smithi003 ceph-mon[31675]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T08:24:14.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:14 smithi171 ceph-mon[35388]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:14.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:14 smithi003 ceph-mon[31675]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:14.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:14 smithi084 ceph-mon[35477]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:16.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:16 smithi003 ceph-mon[31675]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:16.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:16 smithi084 ceph-mon[35477]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:16.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:16 smithi171 ceph-mon[35388]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:18.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:18 smithi003 ceph-mon[31675]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:18.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T08:24:18.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:18.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:18 smithi003 ceph-mon[31675]: Deploying daemon osd.6 on smithi003 2024-03-21T08:24:18.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:18 smithi171 ceph-mon[35388]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:18.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T08:24:18.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:18.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:18 smithi171 ceph-mon[35388]: Deploying daemon osd.6 on smithi003 2024-03-21T08:24:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:18 smithi084 ceph-mon[35477]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T08:24:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:18 smithi084 ceph-mon[35477]: Deploying daemon osd.6 on smithi003 2024-03-21T08:24:20.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:20 smithi171 ceph-mon[35388]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:20.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:20 smithi003 ceph-mon[31675]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:20.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:20 smithi084 ceph-mon[35477]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:22.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:22 smithi003 ceph-mon[31675]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:22.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:22 smithi084 ceph-mon[35477]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:22.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:22 smithi171 ceph-mon[35388]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:24.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:24 smithi003 ceph-mon[31675]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:24.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:24 smithi084 ceph-mon[35477]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:24.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:24 smithi171 ceph-mon[35388]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:26.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:26 smithi003 ceph-mon[31675]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:26.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:26 smithi084 ceph-mon[35477]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:26.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:26 smithi171 ceph-mon[35388]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:27.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:27.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:27.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:24:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:24:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:24:28.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:28 smithi003 ceph-mon[31675]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:29.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:28 smithi084 ceph-mon[35477]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:29.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:28 smithi171 ceph-mon[35388]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:30.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:30 smithi003 ceph-mon[31675]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:30 smithi003 ceph-mon[31675]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T08:24:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:30 smithi084 ceph-mon[35477]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:30 smithi084 ceph-mon[35477]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T08:24:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:31.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:31.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:30 smithi171 ceph-mon[35388]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:31.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:30 smithi171 ceph-mon[35388]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T08:24:31.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:31.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:31.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:31.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:31.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:31 smithi003 ceph-mon[31675]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T08:24:31.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:31 smithi003 ceph-mon[31675]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T08:24:31.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:31 smithi003 ceph-mon[31675]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:24:31.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:31.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:31.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:31 smithi003 ceph-mon[31675]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:24:32.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.016+0000 7fed14a00700 1 -- 172.21.15.3:0/843497712 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fed100625d0 con 0x7fecf40a2fb0 2024-03-21T08:24:32.018 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 6 on host 'smithi003' 2024-03-21T08:24:32.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.019+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecf40a2fb0 msgr2=0x7fecf40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:32.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.019+0000 7fed15e00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecf40a2fb0 0x7fecf40a5470 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fed0000b5c0 tx=0x7fed00005fd0 comp rx=0 tx=0).stop 2024-03-21T08:24:32.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10107eb0 msgr2=0x7fed10101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:32.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10107eb0 0x7fed10101140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fed08007ec0 tx=0x7fed0800d3b0 comp rx=0 tx=0).stop 2024-03-21T08:24:32.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 shutdown_connections 2024-03-21T08:24:32.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecf40a2fb0 0x7fecf40a5470 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:32.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed10107eb0 0x7fed10101140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:32.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed10102c50 0x7fed101025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:32.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 --2- 172.21.15.3:0/843497712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed10105580 0x7fed10100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:32.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 >> 172.21.15.3:0/843497712 conn(0x7fed100fc660 msgr2=0x7fed101035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:24:32.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 shutdown_connections 2024-03-21T08:24:32.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:24:32.020+0000 7fed15e00700 1 -- 172.21.15.3:0/843497712 wait complete. 2024-03-21T08:24:32.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:31 smithi084 ceph-mon[35477]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T08:24:32.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:31 smithi084 ceph-mon[35477]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T08:24:32.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:31 smithi084 ceph-mon[35477]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:24:32.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:32.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:32.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:31 smithi084 ceph-mon[35477]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:24:32.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:31 smithi171 ceph-mon[35388]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T08:24:32.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:31 smithi171 ceph-mon[35388]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T08:24:32.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:31 smithi171 ceph-mon[35388]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:24:32.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:32.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:32.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:31 smithi171 ceph-mon[35388]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:24:32.767 DEBUG:teuthology.orchestra.run.smithi003:osd.6> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.6.service 2024-03-21T08:24:32.772 INFO:tasks.cephadm:Deploying osd.7 on smithi084 with /dev/vg_nvme/lv_2... 2024-03-21T08:24:32.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T08:24:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T08:24:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:32.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:24:32.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:32.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:32.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:32 smithi003 ceph-mon[31675]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' 2024-03-21T08:24:33.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:33.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T08:24:33.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:33.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:33.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:24:33.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:33.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:33.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:32 smithi084 ceph-mon[35477]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:33.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:32 smithi171 ceph-mon[35388]: from='osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022]' entity='osd.6' 2024-03-21T08:24:33.203 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:24:33.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:33 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:24:33.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:33 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:24:33.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:33.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:33 smithi003 ceph-mon[31675]: osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] boot 2024-03-21T08:24:33.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:33 smithi003 ceph-mon[31675]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T08:24:33.930 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:34.123 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:33 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:24:34.123 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:33 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:24:34.124 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:34.124 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:33 smithi084 ceph-mon[35477]: osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] boot 2024-03-21T08:24:34.124 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:33 smithi084 ceph-mon[35477]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T08:24:34.124 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:34.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:33 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:24:34.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:33 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:24:34.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:34.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:33 smithi171 ceph-mon[35388]: osd.6 [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] boot 2024-03-21T08:24:34.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:33 smithi171 ceph-mon[35388]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T08:24:34.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:24:34.957 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:34 smithi084 ceph-mon[35477]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:34.957 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:34 smithi084 ceph-mon[35477]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T08:24:34.971 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:34 smithi003 ceph-mon[31675]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:34.971 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:34 smithi003 ceph-mon[31675]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T08:24:35.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:34 smithi171 ceph-mon[35388]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:24:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:34 smithi171 ceph-mon[35388]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T08:24:35.951 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T08:24:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T08:24:35.952 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-03-21T08:24:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-03-21T08:24:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0334385 s, 314 MB/s 2024-03-21T08:24:35.952 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-03-21T08:24:36.853 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi084:vg_nvme/lv_2 2024-03-21T08:24:36.926 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:36 smithi084 ceph-mon[35477]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:36.941 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:36 smithi003 ceph-mon[31675]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:37.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:36 smithi171 ceph-mon[35388]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:37.269 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi003 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi003 to 5698M 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:38.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:38.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:38 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.098 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:39.098 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi003 2024-03-21T08:24:39.098 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.098 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.098 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:39.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:39.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:39.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi003 to 5698M 2024-03-21T08:24:39.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:39.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:39.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:38 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi003 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:24:39.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi003 to 5698M 2024-03-21T08:24:39.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:39.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:24:39.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:38 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:24:39.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 -- 172.21.15.84:0/2874815598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 msgr2=0x7efc0810a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:39.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 --2- 172.21.15.84:0/2874815598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0810a5f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efc0000b0a0 tx=0x7efc0000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:24:39.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 -- 172.21.15.84:0/2874815598 shutdown_connections 2024-03-21T08:24:39.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 --2- 172.21.15.84:0/2874815598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0810a5f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:39.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 --2- 172.21.15.84:0/2874815598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc081003c0 0x7efc081007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:39.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 --2- 172.21.15.84:0/2874815598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc081033a0 0x7efc08103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:39.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 -- 172.21.15.84:0/2874815598 >> 172.21.15.84:0/2874815598 conn(0x7efc080fbd20 msgr2=0x7efc080fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:24:39.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 -- 172.21.15.84:0/2874815598 shutdown_connections 2024-03-21T08:24:39.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.436+0000 7efc0d800700 1 -- 172.21.15.84:0/2874815598 wait complete. 2024-03-21T08:24:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.437+0000 7efc0d800700 1 Processor -- start 2024-03-21T08:24:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.437+0000 7efc0d800700 1 -- start start 2024-03-21T08:24:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0d800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc081003c0 0x7efc081981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0d800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc081033a0 0x7efc081986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:39.441 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0d800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0d800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0810dbd0 con 0x7efc081033a0 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0d800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0810da50 con 0x7efc08103d40 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0d800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0810dd50 con 0x7efc081003c0 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53370/0 (socket says 172.21.15.84:53370) 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 -- 172.21.15.84:0/4008146323 learned_addr learned my addr 172.21.15.84:0/4008146323 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc07e00700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc081003c0 0x7efc081981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc07400700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc081033a0 0x7efc081986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 -- 172.21.15.84:0/4008146323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc081003c0 msgr2=0x7efc081981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc081003c0 0x7efc081981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:39.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 -- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc081033a0 msgr2=0x7efc081986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc081033a0 0x7efc081986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc0ce00700 1 -- 172.21.15.84:0/4008146323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0819fe10 con 0x7efc08103d40 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.438+0000 7efc07400700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc081033a0 0x7efc081986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.439+0000 7efc0ce00700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0819f770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efc0000d010 tx=0x7efc00004e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.439+0000 7efc04c00700 1 -- 172.21.15.84:0/4008146323 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00021070 con 0x7efc08103d40 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.439+0000 7efc04c00700 1 -- 172.21.15.84:0/4008146323 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efc00007d00 con 0x7efc08103d40 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.439+0000 7efc04c00700 1 -- 172.21.15.84:0/4008146323 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0001d450 con 0x7efc08103d40 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.439+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc081a00a0 con 0x7efc08103d40 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.439+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc081a0630 con 0x7efc08103d40 2024-03-21T08:24:39.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.440+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc08068e50 con 0x7efc08103d40 2024-03-21T08:24:39.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.443+0000 7efc04c00700 1 -- 172.21.15.84:0/4008146323 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7efc000040b0 con 0x7efc08103d40 2024-03-21T08:24:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.444+0000 7efc04c00700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efbec0a3030 0x7efbec0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:24:39.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.444+0000 7efc04c00700 1 -- 172.21.15.84:0/4008146323 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7efc0000f020 con 0x7efc08103d40 2024-03-21T08:24:39.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.445+0000 7efc07e00700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efbec0a3030 0x7efbec0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:24:39.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.446+0000 7efc07e00700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efbec0a3030 0x7efbec0a54f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc0806a4d0 tx=0x7efbf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:24:39.449 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.447+0000 7efc04c00700 1 -- 172.21.15.84:0/4008146323 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efc0002a020 con 0x7efc08103d40 2024-03-21T08:24:39.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:24:39.683+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7efc080625d0 con 0x7efbec0a3030 2024-03-21T08:24:40.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:24:40.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:24:40.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:40.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:24:40.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:24:40.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:40.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:24:40.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:24:40.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:40.984 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:40 smithi084 ceph-mon[35477]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:40.984 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:40 smithi084 ceph-mon[35477]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:24:41.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:40 smithi171 ceph-mon[35388]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:41.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:40 smithi171 ceph-mon[35388]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:24:41.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:40 smithi003 ceph-mon[31675]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:41.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:40 smithi003 ceph-mon[31675]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:24:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:42 smithi084 ceph-mon[35477]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:43.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:42 smithi171 ceph-mon[35388]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:43.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:42 smithi003 ceph-mon[31675]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:43 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/1834495543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]: dispatch 2024-03-21T08:24:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:43 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]: dispatch 2024-03-21T08:24:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:43 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]': finished 2024-03-21T08:24:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:43 smithi084 ceph-mon[35477]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T08:24:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:24:44.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:43 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/1834495543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]: dispatch 2024-03-21T08:24:44.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:43 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]: dispatch 2024-03-21T08:24:44.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:43 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]': finished 2024-03-21T08:24:44.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:43 smithi171 ceph-mon[35388]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T08:24:44.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:24:44.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:43 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/1834495543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]: dispatch 2024-03-21T08:24:44.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:43 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]: dispatch 2024-03-21T08:24:44.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:43 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06b9b8c1-ca04-4499-96d1-79e0e0f316d7"}]': finished 2024-03-21T08:24:44.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:43 smithi003 ceph-mon[31675]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T08:24:44.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:24:45.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:44 smithi171 ceph-mon[35388]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:45.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:44 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/3518451707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:24:45.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:44 smithi003 ceph-mon[31675]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:44 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/3518451707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:24:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:44 smithi084 ceph-mon[35477]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:44 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/3518451707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:24:47.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:46 smithi171 ceph-mon[35388]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:47.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:46 smithi003 ceph-mon[31675]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:47.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:46 smithi084 ceph-mon[35477]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:49.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:48 smithi084 ceph-mon[35477]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:49.156 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:48 smithi003 ceph-mon[31675]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:49.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:48 smithi171 ceph-mon[35388]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:51.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:50 smithi084 ceph-mon[35477]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:51.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:50 smithi003 ceph-mon[31675]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:51.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:50 smithi171 ceph-mon[35388]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:53.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:52 smithi084 ceph-mon[35477]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:53.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T08:24:53.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:53.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:52 smithi003 ceph-mon[31675]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:53.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T08:24:53.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:53.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:52 smithi171 ceph-mon[35388]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T08:24:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:24:54.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:53 smithi084 ceph-mon[35477]: Deploying daemon osd.7 on smithi084 2024-03-21T08:24:54.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:53 smithi003 ceph-mon[31675]: Deploying daemon osd.7 on smithi084 2024-03-21T08:24:54.386 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:53 smithi171 ceph-mon[35388]: Deploying daemon osd.7 on smithi084 2024-03-21T08:24:55.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:54 smithi084 ceph-mon[35477]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:55.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:54 smithi003 ceph-mon[31675]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:55.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:54 smithi171 ceph-mon[35388]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:57.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:56 smithi003 ceph-mon[31675]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:57.371 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:56 smithi084 ceph-mon[35477]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:57.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:56 smithi171 ceph-mon[35388]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:59.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:24:58 smithi084 ceph-mon[35477]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:59.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:24:58 smithi003 ceph-mon[31675]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:24:59.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:24:58 smithi171 ceph-mon[35388]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:01.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:00 smithi084 ceph-mon[35477]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:01.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:00 smithi171 ceph-mon[35388]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:00 smithi003 ceph-mon[31675]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:02.501 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:02.501 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:02.502 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:02 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:02.502 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:02 smithi084 ceph-mon[35477]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:02 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:02.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:02 smithi171 ceph-mon[35388]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:02.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:02.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:02.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:02 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:02.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:02 smithi003 ceph-mon[31675]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:04.444 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:04 smithi084 ceph-mon[35477]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:04.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:04 smithi171 ceph-mon[35388]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:04.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:04 smithi003 ceph-mon[31675]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:06.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:06.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:06 smithi084 ceph-mon[35477]: from='osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:25:06.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:06 smithi084 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:25:06.358 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:06.359 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:06 smithi171 ceph-mon[35388]: from='osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:25:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:06 smithi171 ceph-mon[35388]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:25:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:06.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:06.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:06 smithi003 ceph-mon[31675]: from='osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:25:06.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:06 smithi003 ceph-mon[31675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:25:06.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:06.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:06.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.459+0000 7efc04c00700 1 -- 172.21.15.84:0/4008146323 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efc080625d0 con 0x7efbec0a3030 2024-03-21T08:25:06.462 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 7 on host 'smithi084' 2024-03-21T08:25:06.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.463+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efbec0a3030 msgr2=0x7efbec0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.463+0000 7efc0d800700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efbec0a3030 0x7efbec0a54f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc0806a4d0 tx=0x7efbf8009260 comp rx=0 tx=0).stop 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.463+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 msgr2=0x7efc0819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.463+0000 7efc0d800700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0819f770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efc0000d010 tx=0x7efc00004e70 comp rx=0 tx=0).stop 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 shutdown_connections 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efbec0a3030 0x7efbec0a54f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc08103d40 0x7efc0819f770 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc081003c0 0x7efc081981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 --2- 172.21.15.84:0/4008146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc081033a0 0x7efc081986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 >> 172.21.15.84:0/4008146323 conn(0x7efc080fbd20 msgr2=0x7efc081066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 shutdown_connections 2024-03-21T08:25:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:25:06.464+0000 7efc0d800700 1 -- 172.21.15.84:0/4008146323 wait complete. 2024-03-21T08:25:07.318 DEBUG:teuthology.orchestra.run.smithi084:osd.7> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.7.service 2024-03-21T08:25:07.323 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2024-03-21T08:25:07.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: from='osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:07.412 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:07.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:07 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: from='osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:07 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: from='osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:07.593 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:07.770 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:25:08.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:08 smithi084 ceph-mon[35477]: from='osd.7 ' entity='osd.7' 2024-03-21T08:25:08.377 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:25:07.965+0000 7f1d5b600700 -1 osd.7 0 waiting for initial osdmap 2024-03-21T08:25:08.377 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:25:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:25:07.991+0000 7f1d51400700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:08 smithi003 ceph-mon[31675]: from='osd.7 ' entity='osd.7' 2024-03-21T08:25:08.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:25:08.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:25:08.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:25:08.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T08:25:08.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T08:25:08.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:08.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:08 smithi171 ceph-mon[35388]: from='osd.7 ' entity='osd.7' 2024-03-21T08:25:09.342 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:09 smithi171 ceph-mon[35388]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:09.342 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:09.342 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:09 smithi171 ceph-mon[35388]: osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] boot 2024-03-21T08:25:09.342 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:09 smithi171 ceph-mon[35388]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T08:25:09.342 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:09 smithi084 ceph-mon[35477]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:09 smithi084 ceph-mon[35477]: osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] boot 2024-03-21T08:25:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:09 smithi084 ceph-mon[35477]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T08:25:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:09.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:09 smithi003 ceph-mon[31675]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:25:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:09 smithi003 ceph-mon[31675]: osd.7 [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] boot 2024-03-21T08:25:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:09 smithi003 ceph-mon[31675]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T08:25:09.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:25:10.501 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:10 smithi171 ceph-mon[35388]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T08:25:10.501 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:10 smithi171 ceph-mon[35388]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:10.501 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T08:25:10.501 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T08:25:10.501 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-21T08:25:10.501 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-21T08:25:10.501 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0358359 s, 293 MB/s 2024-03-21T08:25:10.501 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 2024-03-21T08:25:10.501 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-21T08:25:10.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:10 smithi084 ceph-mon[35477]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T08:25:10.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:10 smithi084 ceph-mon[35477]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:10.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:10 smithi003 ceph-mon[31675]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T08:25:10.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:10 smithi003 ceph-mon[31675]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:11.412 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-03-21T08:25:11.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:11 smithi171 ceph-mon[35388]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-21T08:25:11.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:11 smithi171 ceph-mon[35388]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T08:25:11.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:11 smithi084 ceph-mon[35477]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-21T08:25:11.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:11 smithi084 ceph-mon[35477]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T08:25:11.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:11 smithi003 ceph-mon[31675]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-21T08:25:11.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:11 smithi003 ceph-mon[31675]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T08:25:11.851 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:25:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T08:25:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi084 2024-03-21T08:25:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi084 to 5697M 2024-03-21T08:25:12.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:12.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:12.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi084 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi084 to 5697M 2024-03-21T08:25:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:12.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:12.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi084 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi084 to 5697M 2024-03-21T08:25:12.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:12.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:12.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:12.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:13.515 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:13 smithi171 ceph-mon[35388]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T08:25:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:13 smithi084 ceph-mon[35477]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T08:25:13.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:13 smithi003 ceph-mon[31675]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T08:25:14.010 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.008+0000 7f80a7800700 1 -- 172.21.15.171:0/295581294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006fd30 msgr2=0x7f80a00705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:14.010 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.008+0000 7f80a7800700 1 --2- 172.21.15.171:0/295581294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006fd30 0x7f80a00705b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f809800b0a0 tx=0x7f809800b3b0 comp rx=0 tx=0).stop 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.008+0000 7f80a7800700 1 -- 172.21.15.171:0/295581294 shutdown_connections 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.008+0000 7f80a7800700 1 --2- 172.21.15.171:0/295581294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006fd30 0x7f80a00705b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.008+0000 7f80a7800700 1 --2- 172.21.15.171:0/295581294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a006e120 0x7f80a006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.008+0000 7f80a7800700 1 --2- 172.21.15.171:0/295581294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80a006f370 0x7f80a006f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.008+0000 7f80a7800700 1 -- 172.21.15.171:0/295581294 >> 172.21.15.171:0/295581294 conn(0x7f80a006cc60 msgr2=0x7f80a006d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.009+0000 7f80a7800700 1 -- 172.21.15.171:0/295581294 shutdown_connections 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.009+0000 7f80a7800700 1 -- 172.21.15.171:0/295581294 wait complete. 2024-03-21T08:25:14.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.010+0000 7f80a7800700 1 Processor -- start 2024-03-21T08:25:14.012 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.010+0000 7f80a7800700 1 -- start start 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a7800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006e120 0x7f80a01a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a7800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80a006f370 0x7f80a01a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a7800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a006fd30 0x7f80a01ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a7800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a011ca30 con 0x7f80a006f370 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a7800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a011c8b0 con 0x7f80a006fd30 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a7800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a011cbb0 con 0x7f80a006e120 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006e120 0x7f80a01a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006e120 0x7f80a01a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:39334/0 (socket says 172.21.15.171:39334) 2024-03-21T08:25:14.013 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 -- 172.21.15.171:0/139212031 learned_addr learned my addr 172.21.15.171:0/139212031 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:25:14.014 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f809fe00700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80a006f370 0x7f80a01a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:25:14.014 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 -- 172.21.15.171:0/139212031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a006fd30 msgr2=0x7f80a01ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:14.014 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a006fd30 0x7f80a01ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:14.014 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 -- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80a006f370 msgr2=0x7f80a01a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:14.014 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80a006f370 0x7f80a01a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:14.014 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f80a4e00700 1 -- 172.21.15.171:0/139212031 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a01aeca0 con 0x7f80a006e120 2024-03-21T08:25:14.014 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.011+0000 7f809fe00700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80a006f370 0x7f80a01a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:25:14.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.012+0000 7f80a4e00700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006e120 0x7f80a01a7040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f809400d9b0 tx=0x7f809400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:25:14.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.012+0000 7f809d600700 1 -- 172.21.15.171:0/139212031 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094017070 con 0x7f80a006e120 2024-03-21T08:25:14.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.012+0000 7f809d600700 1 -- 172.21.15.171:0/139212031 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8094004830 con 0x7f80a006e120 2024-03-21T08:25:14.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.012+0000 7f809d600700 1 -- 172.21.15.171:0/139212031 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940053a0 con 0x7f80a006e120 2024-03-21T08:25:14.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.012+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a01aef90 con 0x7f80a006e120 2024-03-21T08:25:14.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.012+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a01af390 con 0x7f80a006e120 2024-03-21T08:25:14.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.013+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a002da30 con 0x7f80a006e120 2024-03-21T08:25:14.017 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.016+0000 7f809d600700 1 -- 172.21.15.171:0/139212031 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f80940049e0 con 0x7f80a006e120 2024-03-21T08:25:14.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.018+0000 7f809d600700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f80880a3030 0x7f80880a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:14.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.018+0000 7f809d600700 1 -- 172.21.15.171:0/139212031 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f80940cec00 con 0x7f80a006e120 2024-03-21T08:25:14.020 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.019+0000 7f809fe00700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f80880a3030 0x7f80880a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:25:14.021 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.020+0000 7f809fe00700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f80880a3030 0x7f80880a54f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f808c005fd0 tx=0x7f808c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:25:14.023 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.022+0000 7f809d600700 1 -- 172.21.15.171:0/139212031 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8094020020 con 0x7f80a006e120 2024-03-21T08:25:14.256 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:14.255+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a00625d0 con 0x7f80880a3030 2024-03-21T08:25:14.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:14 smithi084 ceph-mon[35477]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:14.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:14 smithi171 ceph-mon[35388]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:14.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:14 smithi003 ceph-mon[31675]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:15.534 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:15 smithi171 ceph-mon[35388]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:25:15.534 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:25:15.534 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:25:15.534 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:15 smithi084 ceph-mon[35477]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:25:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:25:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:25:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:15.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:15 smithi003 ceph-mon[31675]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:25:15.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:25:15.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:25:15.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:16 smithi084 ceph-mon[35477]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:16 smithi171 ceph-mon[35388]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:16.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:16 smithi003 ceph-mon[31675]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:17.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:17 smithi084 ceph-mon[35477]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-21T08:25:17.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:17 smithi084 ceph-mon[35477]: Cluster is now healthy 2024-03-21T08:25:17.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:17 smithi171 ceph-mon[35388]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-21T08:25:17.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:17 smithi171 ceph-mon[35388]: Cluster is now healthy 2024-03-21T08:25:17.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:17 smithi003 ceph-mon[31675]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-21T08:25:17.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:17 smithi003 ceph-mon[31675]: Cluster is now healthy 2024-03-21T08:25:18.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:18 smithi084 ceph-mon[35477]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:18.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:18 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/2724795699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]: dispatch 2024-03-21T08:25:18.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:18 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]: dispatch 2024-03-21T08:25:18.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:18 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]': finished 2024-03-21T08:25:18.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:18 smithi084 ceph-mon[35477]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T08:25:18.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:18.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:18 smithi171 ceph-mon[35388]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:18.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:18 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/2724795699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]: dispatch 2024-03-21T08:25:18.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:18 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]: dispatch 2024-03-21T08:25:18.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:18 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]': finished 2024-03-21T08:25:18.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:18 smithi171 ceph-mon[35388]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T08:25:18.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:18.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:18 smithi003 ceph-mon[31675]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/2724795699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]: dispatch 2024-03-21T08:25:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:18 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]: dispatch 2024-03-21T08:25:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:18 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e75bbebb-99e2-47c0-8e5a-0842a58c1092"}]': finished 2024-03-21T08:25:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:18 smithi003 ceph-mon[31675]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T08:25:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:19.564 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:19 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/2454065011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:25:19.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/2454065011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:25:19.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:19 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/2454065011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:25:20.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:20 smithi084 ceph-mon[35477]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:20.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:20 smithi171 ceph-mon[35388]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:20.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:20 smithi003 ceph-mon[31675]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:22.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:22 smithi084 ceph-mon[35477]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:22.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:22 smithi171 ceph-mon[35388]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:22.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:22 smithi003 ceph-mon[31675]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:24.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:24 smithi084 ceph-mon[35477]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:24.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:24 smithi171 ceph-mon[35388]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:24.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:24 smithi003 ceph-mon[31675]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:26.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:26 smithi084 ceph-mon[35477]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:26.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:26 smithi171 ceph-mon[35388]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:26.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:26 smithi003 ceph-mon[31675]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:28 smithi084 ceph-mon[35477]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T08:25:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:28 smithi084 ceph-mon[35477]: Deploying daemon osd.8 on smithi171 2024-03-21T08:25:28.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:28 smithi171 ceph-mon[35388]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T08:25:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:28 smithi171 ceph-mon[35388]: Deploying daemon osd.8 on smithi171 2024-03-21T08:25:28.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:28 smithi003 ceph-mon[31675]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:28.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T08:25:28.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:28.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:28 smithi003 ceph-mon[31675]: Deploying daemon osd.8 on smithi171 2024-03-21T08:25:30.465 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:30 smithi171 ceph-mon[35388]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:30.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:30 smithi084 ceph-mon[35477]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:30.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:30 smithi003 ceph-mon[31675]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:32.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:32 smithi084 ceph-mon[35477]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:32.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:32 smithi171 ceph-mon[35388]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:32.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:32 smithi003 ceph-mon[31675]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:34.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:34 smithi084 ceph-mon[35477]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:34.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:34 smithi171 ceph-mon[35388]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:34.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:34 smithi003 ceph-mon[31675]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:37.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:37 smithi084 ceph-mon[35477]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:37.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:37 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:37.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:37 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:37.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:37 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:37.377 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:37 smithi171 ceph-mon[35388]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:37.378 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:37 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:37.378 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:37 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:37.378 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:37 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:37.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:37 smithi003 ceph-mon[31675]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:37.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:37.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:37.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:39.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:39 smithi171 ceph-mon[35388]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:39.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:39 smithi084 ceph-mon[35477]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:39.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:39 smithi003 ceph-mon[31675]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:41.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:40 smithi084 ceph-mon[35477]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:41.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:41.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:41.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:40 smithi084 ceph-mon[35477]: from='osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:25:41.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:40 smithi084 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:25:41.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:40 smithi171 ceph-mon[35388]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:41.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:41.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:41.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:40 smithi171 ceph-mon[35388]: from='osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:25:41.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:40 smithi171 ceph-mon[35388]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:25:41.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:40 smithi003 ceph-mon[31675]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:41.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:41.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:41.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:40 smithi003 ceph-mon[31675]: from='osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:25:41.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:40 smithi003 ceph-mon[31675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:25:41.283 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2024-03-21T08:25:41.284 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.281+0000 7f809d600700 1 -- 172.21.15.171:0/139212031 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f80a00625d0 con 0x7f80880a3030 2024-03-21T08:25:41.286 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.285+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f80880a3030 msgr2=0x7f80880a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:41.286 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.285+0000 7f80a7800700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f80880a3030 0x7f80880a54f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f808c005fd0 tx=0x7f808c005ee0 comp rx=0 tx=0).stop 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006e120 msgr2=0x7f80a01a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006e120 0x7f80a01a7040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f809400d9b0 tx=0x7f809400dd70 comp rx=0 tx=0).stop 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 shutdown_connections 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f80880a3030 0x7f80880a54f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a006fd30 0x7f80a01ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a006e120 0x7f80a01a7040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 --2- 172.21.15.171:0/139212031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80a006f370 0x7f80a01a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 >> 172.21.15.171:0/139212031 conn(0x7f80a006cc60 msgr2=0x7f80a010b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 shutdown_connections 2024-03-21T08:25:41.287 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:25:41.286+0000 7f80a7800700 1 -- 172.21.15.171:0/139212031 wait complete. 2024-03-21T08:25:42.022 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T08:25:42.022 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:25:42.022 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T08:25:42.022 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:42.022 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:25:42.022 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:42.023 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:42.023 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.023 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.023 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.023 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:42.071 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.8.service 2024-03-21T08:25:42.076 INFO:tasks.cephadm:Deploying osd.9 on smithi003 with /dev/vg_nvme/lv_1... 2024-03-21T08:25:42.077 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:42.172 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T08:25:42.172 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:42.173 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:25:42.562 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:25:42.953 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:42.954 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:25:42.954 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: osdmap e60: 9 total, 8 up, 9 in 2024-03-21T08:25:42.954 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:42.954 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:42.954 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: osdmap e61: 9 total, 8 up, 9 in 2024-03-21T08:25:42.954 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:42.954 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-mon[35388]: from='osd.8 ' entity='osd.8' 2024-03-21T08:25:42.955 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:25:42.720+0000 7f491a400700 -1 osd.8 0 waiting for initial osdmap 2024-03-21T08:25:42.955 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:25:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:25:42.766+0000 7f4910200700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: osdmap e60: 9 total, 8 up, 9 in 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: osdmap e61: 9 total, 8 up, 9 in 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:43.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:42 smithi003 ceph-mon[31675]: from='osd.8 ' entity='osd.8' 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: osdmap e60: 9 total, 8 up, 9 in 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: osdmap e61: 9 total, 8 up, 9 in 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:43.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:42 smithi084 ceph-mon[35477]: from='osd.8 ' entity='osd.8' 2024-03-21T08:25:43.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:43 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:25:43.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:43 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:25:43.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:43.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:43 smithi003 ceph-mon[31675]: osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] boot 2024-03-21T08:25:43.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:43 smithi003 ceph-mon[31675]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T08:25:43.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:43 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:44.069 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:43 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:25:44.069 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:43 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:25:44.070 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:44.070 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:43 smithi171 ceph-mon[35388]: osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] boot 2024-03-21T08:25:44.070 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:43 smithi171 ceph-mon[35388]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T08:25:44.070 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:43 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:43 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:25:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:43 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:25:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:43 smithi084 ceph-mon[35477]: osd.8 [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] boot 2024-03-21T08:25:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:43 smithi084 ceph-mon[35477]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T08:25:44.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:43 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:25:44.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:44 smithi003 ceph-mon[31675]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:44.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:44 smithi003 ceph-mon[31675]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T08:25:45.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:44 smithi084 ceph-mon[35477]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:45.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:44 smithi084 ceph-mon[35477]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T08:25:45.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:44 smithi171 ceph-mon[35388]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:25:45.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:44 smithi171 ceph-mon[35388]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T08:25:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T08:25:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T08:25:45.717 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-03-21T08:25:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-03-21T08:25:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0393034 s, 267 MB/s 2024-03-21T08:25:45.717 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-03-21T08:25:46.396 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:46 smithi003 ceph-mon[31675]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T08:25:46.396 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:46 smithi003 ceph-mon[31675]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:25:46.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi003:vg_nvme/lv_1 2024-03-21T08:25:46.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:46 smithi084 ceph-mon[35477]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T08:25:46.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:46 smithi084 ceph-mon[35477]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:25:46.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:46 smithi171 ceph-mon[35388]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T08:25:46.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:46 smithi171 ceph-mon[35388]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:25:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: osdmap e65: 9 total, 9 up, 9 in 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi171 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi171 to 5697M 2024-03-21T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.563 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:47.563 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:47.563 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:47 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: osdmap e65: 9 total, 9 up, 9 in 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi171 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi171 to 5697M 2024-03-21T08:25:47.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:47.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:47.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:47 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: osdmap e65: 9 total, 9 up, 9 in 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi171 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi171 to 5697M 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:47.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:47.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:25:47.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:47 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:25:48.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:48 smithi003 ceph-mon[31675]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:25:48.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:48 smithi003 ceph-mon[31675]: osdmap e66: 9 total, 9 up, 9 in 2024-03-21T08:25:48.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:48 smithi084 ceph-mon[35477]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:25:48.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:48 smithi084 ceph-mon[35477]: osdmap e66: 9 total, 9 up, 9 in 2024-03-21T08:25:48.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:48 smithi171 ceph-mon[35388]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:25:48.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:48 smithi171 ceph-mon[35388]: osdmap e66: 9 total, 9 up, 9 in 2024-03-21T08:25:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.267+0000 7f194fa00700 1 -- 172.21.15.3:0/1743609343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f194810a090 msgr2=0x7f194810c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.267+0000 7f194fa00700 1 --2- 172.21.15.3:0/1743609343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f194810a090 0x7f194810c520 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f194000b0a0 tx=0x7f194000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:25:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.267+0000 7f194fa00700 1 -- 172.21.15.3:0/1743609343 shutdown_connections 2024-03-21T08:25:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.267+0000 7f194fa00700 1 --2- 172.21.15.3:0/1743609343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f194810a090 0x7f194810c520 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.267+0000 7f194fa00700 1 --2- 172.21.15.3:0/1743609343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19480761c0 0x7f1948074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:49.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.267+0000 7f194fa00700 1 --2- 172.21.15.3:0/1743609343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1948074bf0 0x7f1948075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.267+0000 7f194fa00700 1 -- 172.21.15.3:0/1743609343 >> 172.21.15.3:0/1743609343 conn(0x7f19480fd7a0 msgr2=0x7f19480ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:25:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.268+0000 7f194fa00700 1 -- 172.21.15.3:0/1743609343 shutdown_connections 2024-03-21T08:25:49.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.268+0000 7f194fa00700 1 -- 172.21.15.3:0/1743609343 wait complete. 2024-03-21T08:25:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 Processor -- start 2024-03-21T08:25:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 -- start start 2024-03-21T08:25:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1948074bf0 0x7f194819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19480761c0 0x7f194819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f194810a090 0x7f19481a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194810fec0 con 0x7f1948074bf0 2024-03-21T08:25:49.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194810fd40 con 0x7f194810a090 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.269+0000 7f194fa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948110040 con 0x7f19480761c0 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1948074bf0 0x7f194819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1948074bf0 0x7f194819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39240/0 (socket says 172.21.15.3:39240) 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 -- 172.21.15.3:0/1569000788 learned_addr learned my addr 172.21.15.3:0/1569000788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f1947e00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19480761c0 0x7f194819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194da00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f194810a090 0x7f19481a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 -- 172.21.15.3:0/1569000788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19480761c0 msgr2=0x7f194819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19480761c0 0x7f194819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 -- 172.21.15.3:0/1569000788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f194810a090 msgr2=0x7f19481a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f194810a090 0x7f19481a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194d000700 1 -- 172.21.15.3:0/1569000788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19481a2170 con 0x7f1948074bf0 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.270+0000 7f194da00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f194810a090 0x7f19481a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.271+0000 7f194d000700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1948074bf0 0x7f194819a510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f193c00d9b0 tx=0x7f193c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:25:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.271+0000 7f1945600700 1 -- 172.21.15.3:0/1569000788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c017070 con 0x7f1948074bf0 2024-03-21T08:25:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.271+0000 7f1945600700 1 -- 172.21.15.3:0/1569000788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f193c009de0 con 0x7f1948074bf0 2024-03-21T08:25:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.271+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19481a2460 con 0x7f1948074bf0 2024-03-21T08:25:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.271+0000 7f1945600700 1 -- 172.21.15.3:0/1569000788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00f3c0 con 0x7f1948074bf0 2024-03-21T08:25:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.271+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19481a29f0 con 0x7f1948074bf0 2024-03-21T08:25:49.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.272+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1948068e50 con 0x7f1948074bf0 2024-03-21T08:25:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.275+0000 7f1945600700 1 -- 172.21.15.3:0/1569000788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f193c009930 con 0x7f1948074bf0 2024-03-21T08:25:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.277+0000 7f1945600700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19300a3030 0x7f19300a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:25:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.277+0000 7f1945600700 1 -- 172.21.15.3:0/1569000788 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f193c0ce550 con 0x7f1948074bf0 2024-03-21T08:25:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.277+0000 7f1947e00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19300a3030 0x7f19300a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:25:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.278+0000 7f1947e00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19300a3030 0x7f19300a54f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1948075ef0 tx=0x7f193400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:25:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.280+0000 7f1945600700 1 -- 172.21.15.3:0/1569000788 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f193c09bd10 con 0x7f1948074bf0 2024-03-21T08:25:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:25:49.521+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f19480625d0 con 0x7f19300a3030 2024-03-21T08:25:50.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:50 smithi084 ceph-mon[35477]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-21T08:25:50.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:50 smithi084 ceph-mon[35477]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:25:50.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:50 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:25:50.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:50 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:25:50.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:50 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:50.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:50 smithi171 ceph-mon[35388]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-21T08:25:50.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:50 smithi171 ceph-mon[35388]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:25:50.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:50 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:25:50.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:50 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:25:50.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:50 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:50.649 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:50 smithi003 ceph-mon[31675]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-21T08:25:50.649 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:50 smithi003 ceph-mon[31675]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:25:50.649 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:50 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:25:50.649 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:50 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:25:50.649 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:50 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:25:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:52 smithi003 ceph-mon[31675]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:25:52.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:52 smithi084 ceph-mon[35477]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:25:52.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:52 smithi171 ceph-mon[35388]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:25:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:54 smithi084 ceph-mon[35477]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:25:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:54 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/798205455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bad569e-33cd-4ec5-a11f-66e7b60cc9ae"}]: dispatch 2024-03-21T08:25:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:54 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/798205455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bad569e-33cd-4ec5-a11f-66e7b60cc9ae"}]': finished 2024-03-21T08:25:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:54 smithi084 ceph-mon[35477]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T08:25:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:25:54.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:54 smithi171 ceph-mon[35388]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:25:54.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:54 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/798205455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bad569e-33cd-4ec5-a11f-66e7b60cc9ae"}]: dispatch 2024-03-21T08:25:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:54 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/798205455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bad569e-33cd-4ec5-a11f-66e7b60cc9ae"}]': finished 2024-03-21T08:25:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:54 smithi171 ceph-mon[35388]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T08:25:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:25:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:54 smithi003 ceph-mon[31675]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:25:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:54 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/798205455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bad569e-33cd-4ec5-a11f-66e7b60cc9ae"}]: dispatch 2024-03-21T08:25:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:54 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/798205455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bad569e-33cd-4ec5-a11f-66e7b60cc9ae"}]': finished 2024-03-21T08:25:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:54 smithi003 ceph-mon[31675]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T08:25:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:25:55.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:55 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/244161893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:25:55.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:55 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/244161893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:25:55.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:55 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/244161893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:25:56.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:56 smithi084 ceph-mon[35477]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:25:56.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:56 smithi171 ceph-mon[35388]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:25:56.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:56 smithi003 ceph-mon[31675]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:25:58.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:25:58 smithi084 ceph-mon[35477]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T08:25:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:25:58 smithi171 ceph-mon[35388]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T08:25:58.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:25:58 smithi003 ceph-mon[31675]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T08:26:00.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:00 smithi084 ceph-mon[35477]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:00 smithi171 ceph-mon[35388]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:00.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:00 smithi003 ceph-mon[31675]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:02 smithi084 ceph-mon[35477]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:02.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:02 smithi171 ceph-mon[35388]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:02.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:02 smithi003 ceph-mon[31675]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:03.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:03 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T08:26:03.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:03 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:03.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:03 smithi084 ceph-mon[35477]: Deploying daemon osd.9 on smithi003 2024-03-21T08:26:03.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:03 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T08:26:03.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:03 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:03.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:03 smithi171 ceph-mon[35388]: Deploying daemon osd.9 on smithi003 2024-03-21T08:26:03.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:03 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T08:26:03.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:03 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:03.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:03 smithi003 ceph-mon[31675]: Deploying daemon osd.9 on smithi003 2024-03-21T08:26:04.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:04 smithi084 ceph-mon[35477]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:04.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:04 smithi171 ceph-mon[35388]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:04.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:04 smithi003 ceph-mon[31675]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:06.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:06 smithi084 ceph-mon[35477]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:06 smithi171 ceph-mon[35388]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:06.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:06 smithi003 ceph-mon[31675]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:08.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:08 smithi084 ceph-mon[35477]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:08.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:08 smithi171 ceph-mon[35388]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:08.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:08 smithi003 ceph-mon[31675]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:10.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:10 smithi084 ceph-mon[35477]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:10.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:10 smithi171 ceph-mon[35388]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:10.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:10 smithi003 ceph-mon[31675]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:12.880 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:12 smithi003 ceph-mon[31675]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:12.880 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:12.880 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:12.880 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:13.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:12 smithi084 ceph-mon[35477]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:13.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:13.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:13.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:13.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:12 smithi171 ceph-mon[35388]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:13.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:13.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:13.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:14.901 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:14 smithi003 ceph-mon[31675]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:15.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:14 smithi084 ceph-mon[35477]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:15.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:14 smithi171 ceph-mon[35388]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:16.827 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:16 smithi003 ceph-mon[31675]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:16.827 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:16 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:16.828 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:16 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:16.828 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:16 smithi003 ceph-mon[31675]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T08:26:16.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:16 smithi084 ceph-mon[35477]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:16.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:16 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:16.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:16 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:16.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:16 smithi084 ceph-mon[35477]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T08:26:16.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:16 smithi171 ceph-mon[35388]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:16.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:16 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:16.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:16 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:16.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:16 smithi171 ceph-mon[35388]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T08:26:16.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.930+0000 7f1945600700 1 -- 172.21.15.3:0/1569000788 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f19480625d0 con 0x7f19300a3030 2024-03-21T08:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 9 on host 'smithi003' 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19300a3030 msgr2=0x7f19300a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19300a3030 0x7f19300a54f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1948075ef0 tx=0x7f193400b410 comp rx=0 tx=0).stop 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1948074bf0 msgr2=0x7f194819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1948074bf0 0x7f194819a510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f193c00d9b0 tx=0x7f193c00dcc0 comp rx=0 tx=0).stop 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 shutdown_connections 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19300a3030 0x7f19300a54f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1948074bf0 0x7f194819a510 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f194810a090 0x7f19481a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 --2- 172.21.15.3:0/1569000788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19480761c0 0x7f194819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.934+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 >> 172.21.15.3:0/1569000788 conn(0x7f19480fd7a0 msgr2=0x7f1948105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:26:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.935+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 shutdown_connections 2024-03-21T08:26:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:26:16.935+0000 7f194fa00700 1 -- 172.21.15.3:0/1569000788 wait complete. 2024-03-21T08:26:17.647 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T08:26:17.647 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: osdmap e68: 10 total, 9 up, 10 in 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:17.723 DEBUG:teuthology.orchestra.run.smithi003:osd.9> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.9.service 2024-03-21T08:26:17.729 INFO:tasks.cephadm:Deploying osd.10 on smithi084 with /dev/vg_nvme/lv_1... 2024-03-21T08:26:17.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T08:26:17.818 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: osdmap e68: 10 total, 9 up, 10 in 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: osdmap e68: 10 total, 9 up, 10 in 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:17.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:18.168 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:26:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:18 smithi084 ceph-mon[35477]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:18 smithi084 ceph-mon[35477]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:26:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:18 smithi084 ceph-mon[35477]: osdmap e69: 10 total, 9 up, 10 in 2024-03-21T08:26:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:18 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:18 smithi084 ceph-mon[35477]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' 2024-03-21T08:26:18.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-mon[31675]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:18.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-mon[31675]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:26:18.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-mon[31675]: osdmap e69: 10 total, 9 up, 10 in 2024-03-21T08:26:18.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:18.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:18.637 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-mon[31675]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' 2024-03-21T08:26:18.637 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:26:18.357+0000 7fa327400700 -1 osd.9 0 waiting for initial osdmap 2024-03-21T08:26:18.637 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:26:18 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:26:18.390+0000 7fa31c800700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:26:18.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:18 smithi171 ceph-mon[35388]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:18.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:18 smithi171 ceph-mon[35388]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-03-21T08:26:18.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:18 smithi171 ceph-mon[35388]: osdmap e69: 10 total, 9 up, 10 in 2024-03-21T08:26:18.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:18.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:18 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:18.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:18 smithi171 ceph-mon[35388]: from='osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787]' entity='osd.9' 2024-03-21T08:26:19.639 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:19 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:26:19.639 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:19 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:26:19.639 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:19 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:19.639 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:19 smithi003 ceph-mon[31675]: osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] boot 2024-03-21T08:26:19.639 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:19 smithi003 ceph-mon[31675]: osdmap e70: 10 total, 10 up, 10 in 2024-03-21T08:26:19.639 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:19 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:19.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:19 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:26:19.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:19 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:26:19.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:19.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:19 smithi084 ceph-mon[35477]: osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] boot 2024-03-21T08:26:19.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:19 smithi084 ceph-mon[35477]: osdmap e70: 10 total, 10 up, 10 in 2024-03-21T08:26:19.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:19 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:19.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:19 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:26:19.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:19 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:26:19.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:19 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:19.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:19 smithi171 ceph-mon[35388]: osd.9 [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] boot 2024-03-21T08:26:19.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:19 smithi171 ceph-mon[35388]: osdmap e70: 10 total, 10 up, 10 in 2024-03-21T08:26:19.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:19 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:26:20.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:20 smithi084 ceph-mon[35477]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:20.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:20 smithi171 ceph-mon[35388]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:20.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:20 smithi003 ceph-mon[31675]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:26:21.174 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T08:26:21.174 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T08:26:21.174 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-03-21T08:26:21.174 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-03-21T08:26:21.174 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0365105 s, 287 MB/s 2024-03-21T08:26:21.174 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-03-21T08:26:21.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:21 smithi084 ceph-mon[35477]: osdmap e71: 10 total, 10 up, 10 in 2024-03-21T08:26:21.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:21 smithi171 ceph-mon[35388]: osdmap e71: 10 total, 10 up, 10 in 2024-03-21T08:26:21.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:21 smithi003 ceph-mon[31675]: osdmap e71: 10 total, 10 up, 10 in 2024-03-21T08:26:22.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi084:vg_nvme/lv_1 2024-03-21T08:26:22.457 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:26:22.808 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:22 smithi084 ceph-mon[35477]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:22.812 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:22 smithi003 ceph-mon[31675]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:22.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:22 smithi171 ceph-mon[35388]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi003 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi003 to 4273M 2024-03-21T08:26:24.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:24.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:24.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.232 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi003 2024-03-21T08:26:24.232 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.232 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.232 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi003 to 4273M 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:24.233 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi003 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi003 to 4273M 2024-03-21T08:26:24.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:24.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:24.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:24.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 -- 172.21.15.84:0/4268803652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb800761c0 msgr2=0x7fbb80074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 --2- 172.21.15.84:0/4268803652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb800761c0 0x7fbb80074620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009990 tx=0x7fbb70009ca0 comp rx=0 tx=0).stop 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 -- 172.21.15.84:0/4268803652 shutdown_connections 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 --2- 172.21.15.84:0/4268803652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb8010a090 0x7fbb8010c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 --2- 172.21.15.84:0/4268803652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb800761c0 0x7fbb80074620 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 --2- 172.21.15.84:0/4268803652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb80074bf0 0x7fbb80075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 -- 172.21.15.84:0/4268803652 >> 172.21.15.84:0/4268803652 conn(0x7fbb800fd7a0 msgr2=0x7fbb800ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 -- 172.21.15.84:0/4268803652 shutdown_connections 2024-03-21T08:26:24.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.751+0000 7fbb85400700 1 -- 172.21.15.84:0/4268803652 wait complete. 2024-03-21T08:26:24.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 Processor -- start 2024-03-21T08:26:24.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 -- start start 2024-03-21T08:26:24.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb80074bf0 0x7fbb8019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:26:24.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb800761c0 0x7fbb8019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:26:24.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8010a090 0x7fbb801a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:26:24.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8010fe80 con 0x7fbb800761c0 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8010fd00 con 0x7fbb8010a090 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.752+0000 7fbb85400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80110000 con 0x7fbb80074bf0 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8010a090 0x7fbb801a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8010a090 0x7fbb801a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37794/0 (socket says 172.21.15.84:37794) 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 -- 172.21.15.84:0/3414452902 learned_addr learned my addr 172.21.15.84:0/3414452902 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7e800700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb80074bf0 0x7fbb8019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7de00700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb800761c0 0x7fbb8019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:26:24.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 -- 172.21.15.84:0/3414452902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb80074bf0 msgr2=0x7fbb8019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb80074bf0 0x7fbb8019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 -- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb800761c0 msgr2=0x7fbb8019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb800761c0 0x7fbb8019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7f200700 1 -- 172.21.15.84:0/3414452902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb801a2180 con 0x7fbb8010a090 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.753+0000 7fbb7de00700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb800761c0 0x7fbb8019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.754+0000 7fbb7f200700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8010a090 0x7fbb801a1ae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007d70 tx=0x7fbb7800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.754+0000 7fbb77400700 1 -- 172.21.15.84:0/3414452902 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78016020 con 0x7fbb8010a090 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.754+0000 7fbb77400700 1 -- 172.21.15.84:0/3414452902 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb7800ad40 con 0x7fbb8010a090 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.754+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb801a2470 con 0x7fbb8010a090 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.754+0000 7fbb77400700 1 -- 172.21.15.84:0/3414452902 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb780115e0 con 0x7fbb8010a090 2024-03-21T08:26:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.754+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb801a2960 con 0x7fbb8010a090 2024-03-21T08:26:24.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.755+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb80068e50 con 0x7fbb8010a090 2024-03-21T08:26:24.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.757+0000 7fbb77400700 1 -- 172.21.15.84:0/3414452902 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fbb78019020 con 0x7fbb8010a090 2024-03-21T08:26:24.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.759+0000 7fbb77400700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbb640a3030 0x7fbb640a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:26:24.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.759+0000 7fbb77400700 1 -- 172.21.15.84:0/3414452902 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fbb780cec30 con 0x7fbb8010a090 2024-03-21T08:26:24.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.759+0000 7fbb7e800700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbb640a3030 0x7fbb640a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:26:24.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.761+0000 7fbb7e800700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbb640a3030 0x7fbb640a54f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000c00 tx=0x7fbb70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:26:24.861 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.763+0000 7fbb77400700 1 -- 172.21.15.84:0/3414452902 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb78020020 con 0x7fbb8010a090 2024-03-21T08:26:24.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:24.996+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb800625d0 con 0x7fbb640a3030 2024-03-21T08:26:25.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:24 smithi084 ceph-mon[35477]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:25.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:24 smithi171 ceph-mon[35388]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:25.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:24 smithi003 ceph-mon[31675]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:26.276 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:25 smithi084 ceph-mon[35477]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:26:26.276 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:26:26.276 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:26:26.276 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:26.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:25 smithi171 ceph-mon[35388]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:26:26.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:26:26.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:26:26.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:26.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:25 smithi003 ceph-mon[31675]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:26:26.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:26:26.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:26:26.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:27.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:26 smithi084 ceph-mon[35477]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:27.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:26 smithi171 ceph-mon[35388]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:27.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:26 smithi003 ceph-mon[31675]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:29 smithi084 ceph-mon[35477]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:29 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/2767661705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]: dispatch 2024-03-21T08:26:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:29 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]: dispatch 2024-03-21T08:26:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:29 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]': finished 2024-03-21T08:26:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:29 smithi084 ceph-mon[35477]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T08:26:29.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:29 smithi171 ceph-mon[35388]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:29.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:29 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/2767661705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]: dispatch 2024-03-21T08:26:29.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:29 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]: dispatch 2024-03-21T08:26:29.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:29 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]': finished 2024-03-21T08:26:29.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:29 smithi171 ceph-mon[35388]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T08:26:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:29 smithi003 ceph-mon[31675]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:29 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/2767661705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]: dispatch 2024-03-21T08:26:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:29 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]: dispatch 2024-03-21T08:26:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:29 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "960b5226-4fe7-42b0-bba6-193c2425e0de"}]': finished 2024-03-21T08:26:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:29 smithi003 ceph-mon[31675]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T08:26:30.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:30.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:30 smithi084 ceph-mon[35477]: from='client.? 172.21.15.84:0/1260605104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:26:30.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:30.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:30 smithi171 ceph-mon[35388]: from='client.? 172.21.15.84:0/1260605104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:26:30.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:30.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:30 smithi003 ceph-mon[31675]: from='client.? 172.21.15.84:0/1260605104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:26:31.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:31 smithi084 ceph-mon[35477]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:31.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:31 smithi171 ceph-mon[35388]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:31.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:31 smithi003 ceph-mon[31675]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:33.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:33 smithi084 ceph-mon[35477]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:33.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:33 smithi171 ceph-mon[35388]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:33.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:33 smithi003 ceph-mon[31675]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:35.309 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:35 smithi084 ceph-mon[35477]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:35.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:35 smithi171 ceph-mon[35388]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:35.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:35 smithi003 ceph-mon[31675]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:37.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:37 smithi084 ceph-mon[35477]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:37.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:37 smithi171 ceph-mon[35388]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:37.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:37 smithi003 ceph-mon[31675]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:39 smithi084 ceph-mon[35477]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T08:26:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:39 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:39.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:39 smithi171 ceph-mon[35388]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:39.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T08:26:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:39 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:39.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:39 smithi003 ceph-mon[31675]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:39.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T08:26:39.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:39 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:40.337 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:40 smithi084 ceph-mon[35477]: Deploying daemon osd.10 on smithi084 2024-03-21T08:26:40.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:40 smithi171 ceph-mon[35388]: Deploying daemon osd.10 on smithi084 2024-03-21T08:26:40.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:40 smithi003 ceph-mon[31675]: Deploying daemon osd.10 on smithi084 2024-03-21T08:26:41.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:41 smithi003 ceph-mon[31675]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:41.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:41 smithi084 ceph-mon[35477]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:41.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:41 smithi171 ceph-mon[35388]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:43.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:43 smithi084 ceph-mon[35477]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:43.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:43 smithi171 ceph-mon[35388]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:43.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:43 smithi003 ceph-mon[31675]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:45.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:45 smithi084 ceph-mon[35477]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:45.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:45 smithi171 ceph-mon[35388]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:45.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:45 smithi003 ceph-mon[31675]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:47.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:47 smithi084 ceph-mon[35477]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:47.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:47 smithi171 ceph-mon[35388]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:47.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:47 smithi003 ceph-mon[31675]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:48.441 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:48.441 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:48.441 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:48.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:48.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:48.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:48.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:48.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:48.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:49.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:49 smithi084 ceph-mon[35477]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:49.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:49 smithi171 ceph-mon[35388]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:49.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:49 smithi003 ceph-mon[31675]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:50.480 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:50 smithi084 ceph-mon[35477]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:50.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:50 smithi171 ceph-mon[35388]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:50.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:50 smithi003 ceph-mon[31675]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:52.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:52.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:52 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:52.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:52 smithi084 ceph-mon[35477]: from='osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:26:52.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:52 smithi084 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:26:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:52 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:52.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:52 smithi171 ceph-mon[35388]: from='osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:26:52.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:52 smithi171 ceph-mon[35388]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:26:52.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:52.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:52 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:52 smithi003 ceph-mon[31675]: from='osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:26:52.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:52 smithi003 ceph-mon[31675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:26:52.584 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.582+0000 7fbb77400700 1 -- 172.21.15.84:0/3414452902 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fbb800625d0 con 0x7fbb640a3030 2024-03-21T08:26:52.585 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 10 on host 'smithi084' 2024-03-21T08:26:52.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbb640a3030 msgr2=0x7fbb640a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:26:52.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbb640a3030 0x7fbb640a54f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000c00 tx=0x7fbb70005fb0 comp rx=0 tx=0).stop 2024-03-21T08:26:52.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8010a090 msgr2=0x7fbb801a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:26:52.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8010a090 0x7fbb801a1ae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007d70 tx=0x7fbb7800a340 comp rx=0 tx=0).stop 2024-03-21T08:26:52.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 shutdown_connections 2024-03-21T08:26:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbb640a3030 0x7fbb640a54f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb8010a090 0x7fbb801a1ae0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb80074bf0 0x7fbb8019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 --2- 172.21.15.84:0/3414452902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb800761c0 0x7fbb8019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:26:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 >> 172.21.15.84:0/3414452902 conn(0x7fbb800fd7a0 msgr2=0x7fbb80105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:26:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.586+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 shutdown_connections 2024-03-21T08:26:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:2024-03-21T08:26:52.587+0000 7fbb85400700 1 -- 172.21.15.84:0/3414452902 wait complete. 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: osdmap e73: 11 total, 10 up, 11 in 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.365 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T08:26:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: osdmap e73: 11 total, 10 up, 11 in 2024-03-21T08:26:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:26:53.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:26:53.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:53.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:53.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:53.412 DEBUG:teuthology.orchestra.run.smithi084:osd.10> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.10.service 2024-03-21T08:26:53.418 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2024-03-21T08:26:53.418 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T08:26:53.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: osdmap e73: 11 total, 10 up, 11 in 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:26:53.870 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:26:54.251 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:54 smithi084 ceph-mon[35477]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:26:54.251 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:54 smithi084 ceph-mon[35477]: osdmap e74: 11 total, 10 up, 11 in 2024-03-21T08:26:54.251 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:54.251 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:54 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:54.339 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:54 smithi171 ceph-mon[35388]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:26:54.340 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:54 smithi171 ceph-mon[35388]: osdmap e74: 11 total, 10 up, 11 in 2024-03-21T08:26:54.340 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:54.340 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:54 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:54.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:54 smithi003 ceph-mon[31675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-03-21T08:26:54.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:54 smithi003 ceph-mon[31675]: osdmap e74: 11 total, 10 up, 11 in 2024-03-21T08:26:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:54 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:55.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:55 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:26:55.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:55 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:26:55.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:55 smithi084 ceph-mon[35477]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:55.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:55 smithi084 ceph-mon[35477]: osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] boot 2024-03-21T08:26:55.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:55 smithi084 ceph-mon[35477]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T08:26:55.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:55 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:55.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:55 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:26:55.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:55 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:26:55.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:55 smithi171 ceph-mon[35388]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:55.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:55 smithi171 ceph-mon[35388]: osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] boot 2024-03-21T08:26:55.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:55 smithi171 ceph-mon[35388]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T08:26:55.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:55 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:55 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:26:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:55 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:26:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:55 smithi003 ceph-mon[31675]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:26:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:55 smithi003 ceph-mon[31675]: osd.10 [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] boot 2024-03-21T08:26:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:55 smithi003 ceph-mon[31675]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T08:26:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:55 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:26:56.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:56 smithi084 ceph-mon[35477]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T08:26:56.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:56 smithi171 ceph-mon[35388]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T08:26:56.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:56 smithi003 ceph-mon[31675]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T08:26:56.913 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T08:26:56.913 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T08:26:56.913 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-21T08:26:56.913 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-21T08:26:56.913 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0323169 s, 324 MB/s 2024-03-21T08:26:56.913 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 2024-03-21T08:26:56.913 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-21T08:26:57.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:57 smithi171 ceph-mon[35388]: pgmap v301: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:26:57.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:57 smithi171 ceph-mon[35388]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T08:26:57.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:57 smithi084 ceph-mon[35477]: pgmap v301: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:26:57.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:57 smithi084 ceph-mon[35477]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T08:26:57.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:57 smithi003 ceph-mon[31675]: pgmap v301: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:26:57.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:57 smithi003 ceph-mon[31675]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T08:26:57.770 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-03-21T08:26:58.245 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:26:58.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:58 smithi084 ceph-mon[35477]: osdmap e78: 11 total, 11 up, 11 in 2024-03-21T08:26:58.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:58 smithi171 ceph-mon[35388]: osdmap e78: 11 total, 11 up, 11 in 2024-03-21T08:26:58.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:58 smithi003 ceph-mon[31675]: osdmap e78: 11 total, 11 up, 11 in 2024-03-21T08:26:59.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: pgmap v304: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:26:59.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: osdmap e79: 11 total, 11 up, 11 in 2024-03-21T08:26:59.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:59.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:59.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:26:59 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: pgmap v304: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: osdmap e79: 11 total, 11 up, 11 in 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:59.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:59.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:26:59 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: pgmap v304: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: osdmap e79: 11 total, 11 up, 11 in 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:26:59.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:26:59.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:26:59 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:00.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:00 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi084 2024-03-21T08:27:00.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:00 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi084 to 4272M 2024-03-21T08:27:00.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:00 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi084 2024-03-21T08:27:00.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:00 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi084 to 4272M 2024-03-21T08:27:00.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.450+0000 7ff6cda00700 1 -- 172.21.15.171:0/195157084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 msgr2=0x7ff6c8108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:00.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.450+0000 7ff6cda00700 1 --2- 172.21.15.171:0/195157084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c8108c30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009990 tx=0x7ff6b8009ca0 comp rx=0 tx=0).stop 2024-03-21T08:27:00.452 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.451+0000 7ff6cda00700 1 -- 172.21.15.171:0/195157084 shutdown_connections 2024-03-21T08:27:00.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.451+0000 7ff6cda00700 1 --2- 172.21.15.171:0/195157084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6c810a3b0 0x7ff6c8110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:00.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.451+0000 7ff6cda00700 1 --2- 172.21.15.171:0/195157084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c8108c30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:00.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.451+0000 7ff6cda00700 1 --2- 172.21.15.171:0/195157084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6c8109a10 0x7ff6c8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:00.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.451+0000 7ff6cda00700 1 -- 172.21.15.171:0/195157084 >> 172.21.15.171:0/195157084 conn(0x7ff6c80777e0 msgr2=0x7ff6c8079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:00.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.451+0000 7ff6cda00700 1 -- 172.21.15.171:0/195157084 shutdown_connections 2024-03-21T08:27:00.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.451+0000 7ff6cda00700 1 -- 172.21.15.171:0/195157084 wait complete. 2024-03-21T08:27:00.455 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 Processor -- start 2024-03-21T08:27:00.455 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 -- start start 2024-03-21T08:27:00.455 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:00.455 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6c8109a10 0x7ff6c819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:00.455 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6c810a3b0 0x7ff6c81a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:00.455 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c81142d0 con 0x7ff6c810a3b0 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8114150 con 0x7ff6c8109a10 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6cda00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8114450 con 0x7ff6c8108810 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6c6a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.452+0000 7ff6c6a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:52974/0 (socket says 172.21.15.171:52974) 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c6a00700 1 -- 172.21.15.171:0/1671622624 learned_addr learned my addr 172.21.15.171:0/1671622624 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6be000700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6c8109a10 0x7ff6c819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c6a00700 1 -- 172.21.15.171:0/1671622624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6c8109a10 msgr2=0x7ff6c819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c7400700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6c810a3b0 0x7ff6c81a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c6a00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6c8109a10 0x7ff6c819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c6a00700 1 -- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6c810a3b0 msgr2=0x7ff6c81a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c6a00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6c810a3b0 0x7ff6c81a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c6a00700 1 -- 172.21.15.171:0/1671622624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c81a6480 con 0x7ff6c8108810 2024-03-21T08:27:00.456 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c7400700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6c810a3b0 0x7ff6c81a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:27:00.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.453+0000 7ff6c6a00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c819e820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8003750 tx=0x7ff6b8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:00.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.454+0000 7ff6bfe00700 1 -- 172.21.15.171:0/1671622624 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8027070 con 0x7ff6c8108810 2024-03-21T08:27:00.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.454+0000 7ff6bfe00700 1 -- 172.21.15.171:0/1671622624 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6b8004180 con 0x7ff6c8108810 2024-03-21T08:27:00.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.454+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c81a6710 con 0x7ff6c8108810 2024-03-21T08:27:00.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.454+0000 7ff6bfe00700 1 -- 172.21.15.171:0/1671622624 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8022e10 con 0x7ff6c8108810 2024-03-21T08:27:00.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.454+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c81a6bc0 con 0x7ff6c8108810 2024-03-21T08:27:00.457 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.455+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c8068e50 con 0x7ff6c8108810 2024-03-21T08:27:00.458 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.457+0000 7ff6bfe00700 1 -- 172.21.15.171:0/1671622624 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7ff6b802a070 con 0x7ff6c8108810 2024-03-21T08:27:00.461 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.459+0000 7ff6bfe00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff6ac0a3030 0x7ff6ac0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:00.461 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.459+0000 7ff6bfe00700 1 -- 172.21.15.171:0/1671622624 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6694+0+0 (secure 0 0 0) 0x7ff6b80cfcb0 con 0x7ff6c8108810 2024-03-21T08:27:00.461 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.460+0000 7ff6be000700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff6ac0a3030 0x7ff6ac0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:00.462 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.461+0000 7ff6be000700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff6ac0a3030 0x7ff6ac0a54f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000e270 tx=0x7ff6b0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:00.466 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.464+0000 7ff6bfe00700 1 -- 172.21.15.171:0/1671622624 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6b8013070 con 0x7ff6c8108810 2024-03-21T08:27:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:00 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi084 2024-03-21T08:27:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:00 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi084 to 4272M 2024-03-21T08:27:00.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:00.697+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6c80625d0 con 0x7ff6ac0a3030 2024-03-21T08:27:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:01 smithi003 ceph-mon[31675]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-21T08:27:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:01 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:27:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:01 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:27:01.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:01 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:01.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:01 smithi084 ceph-mon[35477]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-21T08:27:01.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:01 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:27:01.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:01 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:27:01.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:01 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:01.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:01 smithi171 ceph-mon[35388]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-21T08:27:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:01 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:27:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:01 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:27:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:01 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:02.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:02 smithi171 ceph-mon[35388]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:27:02.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:02 smithi003 ceph-mon[31675]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:27:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:02 smithi084 ceph-mon[35477]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:27:03.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:03 smithi171 ceph-mon[35388]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:27:03.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:03 smithi003 ceph-mon[31675]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:27:03.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:03 smithi084 ceph-mon[35477]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:27:05.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:05 smithi003 ceph-mon[31675]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T08:27:05.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:05 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/323562243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]: dispatch 2024-03-21T08:27:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:05 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]: dispatch 2024-03-21T08:27:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:05 smithi003 ceph-mon[31675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]': finished 2024-03-21T08:27:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:05 smithi003 ceph-mon[31675]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T08:27:05.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:05.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:05 smithi171 ceph-mon[35388]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T08:27:05.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:05 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/323562243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]: dispatch 2024-03-21T08:27:05.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:05 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]: dispatch 2024-03-21T08:27:05.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:05 smithi171 ceph-mon[35388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]': finished 2024-03-21T08:27:05.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:05 smithi171 ceph-mon[35388]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T08:27:05.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:05 smithi084 ceph-mon[35477]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T08:27:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:05 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/323562243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]: dispatch 2024-03-21T08:27:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:05 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]: dispatch 2024-03-21T08:27:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:05 smithi084 ceph-mon[35477]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "553edf28-7b5a-48cf-8ab4-c003e219e1b0"}]': finished 2024-03-21T08:27:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:05 smithi084 ceph-mon[35477]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T08:27:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:06.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:06 smithi003 ceph-mon[31675]: from='client.? 172.21.15.171:0/3651608100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:27:06.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:06 smithi084 ceph-mon[35477]: from='client.? 172.21.15.171:0/3651608100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:27:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:06 smithi171 ceph-mon[35388]: from='client.? 172.21.15.171:0/3651608100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:27:07.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:07 smithi003 ceph-mon[31675]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:27:07.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:07 smithi084 ceph-mon[35477]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:27:07.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:07 smithi171 ceph-mon[35388]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:27:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:09 smithi171 ceph-mon[35388]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-21T08:27:09.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:09 smithi003 ceph-mon[31675]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-21T08:27:09.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:09 smithi084 ceph-mon[35477]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-21T08:27:11.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:11 smithi171 ceph-mon[35388]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:11.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:11 smithi084 ceph-mon[35477]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:11.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:11 smithi003 ceph-mon[31675]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:13 smithi171 ceph-mon[35388]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:13.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:13 smithi084 ceph-mon[35477]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:13.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:13 smithi003 ceph-mon[31675]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:15.548 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:15 smithi171 ceph-mon[35388]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:15.548 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T08:27:15.548 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:15 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:15.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:15 smithi084 ceph-mon[35477]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:15.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T08:27:15.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:15 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:15.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:15 smithi003 ceph-mon[31675]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:15.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T08:27:15.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:15 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:16 smithi084 ceph-mon[35477]: Deploying daemon osd.11 on smithi171 2024-03-21T08:27:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:16 smithi171 ceph-mon[35388]: Deploying daemon osd.11 on smithi171 2024-03-21T08:27:16.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:16 smithi003 ceph-mon[31675]: Deploying daemon osd.11 on smithi171 2024-03-21T08:27:17.323 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:17 smithi171 ceph-mon[35388]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:17.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:17 smithi084 ceph-mon[35477]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:17.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:17 smithi003 ceph-mon[31675]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:18.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:18 smithi171 ceph-mon[35388]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:18 smithi084 ceph-mon[35477]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:18.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:18 smithi003 ceph-mon[31675]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:20.553 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:20 smithi171 ceph-mon[35388]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:20.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:20 smithi084 ceph-mon[35477]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:20.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:20 smithi003 ceph-mon[31675]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:22.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:22 smithi084 ceph-mon[35477]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:22.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:22 smithi171 ceph-mon[35388]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:22.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:22 smithi003 ceph-mon[31675]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:23.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:23.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:23.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:23 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:23.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:23.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:23.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:23 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:23.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:23.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:23.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:23 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:25.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:25 smithi084 ceph-mon[35477]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:25.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:25 smithi171 ceph-mon[35388]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:25.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:25 smithi003 ceph-mon[31675]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:27.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:27 smithi171 ceph-mon[35388]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:27.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:27 smithi171 ceph-mon[35388]: from='osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:27:27.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:27 smithi171 ceph-mon[35388]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:27:27.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:27.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:27.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:27.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:27.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:27 smithi084 ceph-mon[35477]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:27.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:27 smithi084 ceph-mon[35477]: from='osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:27:27.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:27 smithi084 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:27:27.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:27.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:27.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:27.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:27.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:27 smithi003 ceph-mon[31675]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:27.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:27 smithi003 ceph-mon[31675]: from='osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:27:27.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:27 smithi003 ceph-mon[31675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:27:27.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:27.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:27.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:27.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:28.166 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.163+0000 7ff6bfe00700 1 -- 172.21.15.171:0/1671622624 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff6c80625d0 con 0x7ff6ac0a3030 2024-03-21T08:27:28.167 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2024-03-21T08:27:28.168 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff6ac0a3030 msgr2=0x7ff6ac0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:28.168 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff6ac0a3030 0x7ff6ac0a54f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000e270 tx=0x7ff6b0009040 comp rx=0 tx=0).stop 2024-03-21T08:27:28.168 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 msgr2=0x7ff6c819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:28.168 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c819e820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8003750 tx=0x7ff6b8005e80 comp rx=0 tx=0).stop 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 shutdown_connections 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff6ac0a3030 0x7ff6ac0a54f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6c810a3b0 0x7ff6c81a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8108810 0x7ff6c819e820 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 --2- 172.21.15.171:0/1671622624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6c8109a10 0x7ff6c819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 >> 172.21.15.171:0/1671622624 conn(0x7ff6c80777e0 msgr2=0x7ff6c810ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 shutdown_connections 2024-03-21T08:27:28.169 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:28.167+0000 7ff6cda00700 1 -- 172.21.15.171:0/1671622624 wait complete. 2024-03-21T08:27:28.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:28.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:28 smithi084 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T08:27:28.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:28 smithi084 ceph-mon[35477]: osdmap e81: 12 total, 11 up, 12 in 2024-03-21T08:27:28.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:28.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:28 smithi084 ceph-mon[35477]: from='osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:27:28.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:28 smithi084 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:27:28.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:28.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:28 smithi171 ceph-mon[35388]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T08:27:28.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:28 smithi171 ceph-mon[35388]: osdmap e81: 12 total, 11 up, 12 in 2024-03-21T08:27:28.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:28.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:28 smithi171 ceph-mon[35388]: from='osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:27:28.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:28 smithi171 ceph-mon[35388]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:27:28.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:28 smithi003 ceph-mon[31675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T08:27:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:28 smithi003 ceph-mon[31675]: osdmap e81: 12 total, 11 up, 12 in 2024-03-21T08:27:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:28 smithi003 ceph-mon[31675]: from='osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:27:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:28 smithi003 ceph-mon[31675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-21T08:27:28.998 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.11.service 2024-03-21T08:27:29.004 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-21T08:27:29.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T08:27:29.198 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:29.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:27:29.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: osdmap e82: 12 total, 11 up, 12 in 2024-03-21T08:27:29.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:29.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:29.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:29.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:29.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: osdmap e82: 12 total, 11 up, 12 in 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:29.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-21T08:27:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: osdmap e82: 12 total, 11 up, 12 in 2024-03-21T08:27:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:29.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:29.476 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:27:30.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:30 smithi003 ceph-mon[31675]: purged_snaps scrub starts 2024-03-21T08:27:30.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:30 smithi003 ceph-mon[31675]: purged_snaps scrub ok 2024-03-21T08:27:30.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:30.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:30 smithi003 ceph-mon[31675]: osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] boot 2024-03-21T08:27:30.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:30 smithi003 ceph-mon[31675]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T08:27:30.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:30.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:30 smithi084 ceph-mon[35477]: purged_snaps scrub starts 2024-03-21T08:27:30.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:30 smithi084 ceph-mon[35477]: purged_snaps scrub ok 2024-03-21T08:27:30.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:30.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:30 smithi084 ceph-mon[35477]: osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] boot 2024-03-21T08:27:30.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:30 smithi084 ceph-mon[35477]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T08:27:30.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:30.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:30 smithi171 ceph-mon[35388]: purged_snaps scrub starts 2024-03-21T08:27:30.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:30 smithi171 ceph-mon[35388]: purged_snaps scrub ok 2024-03-21T08:27:30.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:30.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:30 smithi171 ceph-mon[35388]: osd.11 [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] boot 2024-03-21T08:27:30.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:30 smithi171 ceph-mon[35388]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T08:27:30.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:27:31.307 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:31 smithi171 ceph-mon[35388]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:31.307 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:31 smithi171 ceph-mon[35388]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T08:27:31.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:31 smithi003 ceph-mon[31675]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:31.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:31 smithi003 ceph-mon[31675]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T08:27:31.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:31 smithi084 ceph-mon[35477]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:27:31.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:31 smithi084 ceph-mon[35477]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T08:27:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.877+0000 7fa9dd800700 1 -- 172.21.15.3:0/2490143005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 msgr2=0x7fa9d810c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.877+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2490143005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d810c840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000b0a0 tx=0x7fa9d000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:27:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.878+0000 7fa9dd800700 1 -- 172.21.15.3:0/2490143005 shutdown_connections 2024-03-21T08:27:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.878+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2490143005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d810c840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.878+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2490143005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9d8105ad0 0x7fa9d8105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.878+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2490143005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d8104010 0x7fa9d8104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:31.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.878+0000 7fa9dd800700 1 -- 172.21.15.3:0/2490143005 >> 172.21.15.3:0/2490143005 conn(0x7fa9d80ff9b0 msgr2=0x7fa9d8101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:31.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.878+0000 7fa9dd800700 1 -- 172.21.15.3:0/2490143005 shutdown_connections 2024-03-21T08:27:31.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.878+0000 7fa9dd800700 1 -- 172.21.15.3:0/2490143005 wait complete. 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 Processor -- start 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 -- start start 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9d8104010 0x7fa9d8079f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d8105ad0 0x7fa9d8076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d810ffc0 con 0x7fa9d81049b0 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d810fe40 con 0x7fa9d8105ad0 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.879+0000 7fa9dd800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8110140 con 0x7fa9d8104010 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d807a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46594/0 (socket says 172.21.15.3:46594) 2024-03-21T08:27:31.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 -- 172.21.15.3:0/2252023193 learned_addr learned my addr 172.21.15.3:0/2252023193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9d6a00700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9d8104010 0x7fa9d8079f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9d7400700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d8105ad0 0x7fa9d8076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 -- 172.21.15.3:0/2252023193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9d8104010 msgr2=0x7fa9d8079f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9d8104010 0x7fa9d8079f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 -- 172.21.15.3:0/2252023193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d8105ad0 msgr2=0x7fa9d8076ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d8105ad0 0x7fa9d8076ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9ce000700 1 -- 172.21.15.3:0/2252023193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8077390 con 0x7fa9d81049b0 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.880+0000 7fa9d6a00700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9d8104010 0x7fa9d8079f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.881+0000 7fa9ce000700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d807a480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c9b0 tx=0x7fa9c000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.881+0000 7fa9cfe00700 1 -- 172.21.15.3:0/2252023193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007b00 con 0x7fa9d81049b0 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.881+0000 7fa9cfe00700 1 -- 172.21.15.3:0/2252023193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9c0007ca0 con 0x7fa9d81049b0 2024-03-21T08:27:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.881+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d81a68c0 con 0x7fa9d81049b0 2024-03-21T08:27:31.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.881+0000 7fa9cfe00700 1 -- 172.21.15.3:0/2252023193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0018730 con 0x7fa9d81049b0 2024-03-21T08:27:31.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.881+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d81a6d70 con 0x7fa9d81049b0 2024-03-21T08:27:31.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.882+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d8068e50 con 0x7fa9d81049b0 2024-03-21T08:27:31.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.884+0000 7fa9cfe00700 1 -- 172.21.15.3:0/2252023193 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fa9c000f450 con 0x7fa9d81049b0 2024-03-21T08:27:31.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.886+0000 7fa9cfe00700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:31.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.886+0000 7fa9cfe00700 1 -- 172.21.15.3:0/2252023193 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9c00cefd0 con 0x7fa9d81049b0 2024-03-21T08:27:31.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.886+0000 7fa9d6a00700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:31.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.887+0000 7fa9d6a00700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8105450 tx=0x7fa9c800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:31.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:31.892+0000 7fa9cfe00700 1 -- 172.21.15.3:0/2252023193 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa9c0014030 con 0x7fa9d81049b0 2024-03-21T08:27:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.118+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa9d8062da0 con 0x7fa9d81049b0 2024-03-21T08:27:32.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.119+0000 7fa9cfe00700 1 -- 172.21.15.3:0/2252023193 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa9c009c200 con 0x7fa9d81049b0 2024-03-21T08:27:32.120 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:32.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.122+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9bc0a3030 msgr2=0x7fa9bc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:32.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.122+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8105450 tx=0x7fa9c800a680 comp rx=0 tx=0).stop 2024-03-21T08:27:32.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.122+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 msgr2=0x7fa9d807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:32.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.122+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d807a480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c9b0 tx=0x7fa9c000cd70 comp rx=0 tx=0).stop 2024-03-21T08:27:32.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 shutdown_connections 2024-03-21T08:27:32.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9bc0a3030 0x7fa9bc0a54f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:32.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d8105ad0 0x7fa9d8076ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:32.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9d8104010 0x7fa9d8079f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:32.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 --2- 172.21.15.3:0/2252023193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9d81049b0 0x7fa9d807a480 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:32.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 >> 172.21.15.3:0/2252023193 conn(0x7fa9d80ff9b0 msgr2=0x7fa9d8108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:32.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 shutdown_connections 2024-03-21T08:27:32.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:32.123+0000 7fa9dd800700 1 -- 172.21.15.3:0/2252023193 wait complete. 2024-03-21T08:27:32.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:32 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2252023193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T08:27:32.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:32 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2252023193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T08:27:32.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:32 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2252023193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T08:27:33.150 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1711009649,"num_in_osds":12,"osd_in_since":1711009624,"num_remapped_pgs":0} 2024-03-21T08:27:33.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T08:27:33.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:33 smithi171 ceph-mon[35388]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:33.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:33 smithi003 ceph-mon[31675]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:27:33.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:33 smithi084 ceph-mon[35477]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:34.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:34.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: Detected new or changed devices on smithi171 2024-03-21T08:27:34.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi171 to 4272M 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:34 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: Detected new or changed devices on smithi171 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi171 to 4272M 2024-03-21T08:27:35.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:35.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:35.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:34 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: Detected new or changed devices on smithi171 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi171 to 4272M 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:35.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:35.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:35.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:34 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.044+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2219882143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0112e20 msgr2=0x7f8ce0115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:36.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.044+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2219882143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0112e20 0x7f8ce0115210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800b0a0 tx=0x7f8cd800b3b0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.045+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2219882143 shutdown_connections 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.045+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2219882143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0112e20 0x7f8ce0115210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.045+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2219882143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ce00761c0 0x7f8ce00743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.045+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2219882143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ce0074900 0x7f8ce0074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.045+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2219882143 >> 172.21.15.3:0/2219882143 conn(0x7f8ce00792c0 msgr2=0x7f8ce00796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.045+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2219882143 shutdown_connections 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.045+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2219882143 wait complete. 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 Processor -- start 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 -- start start 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0074900 0x7f8ce019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ce00761c0 0x7f8ce019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:36.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ce0112e20 0x7f8ce01a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce0118960 con 0x7f8ce0074900 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce01187e0 con 0x7f8ce00761c0 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.046+0000 7f8ce7e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce0118ae0 con 0x7f8ce0112e20 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0074900 0x7f8ce019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0074900 0x7f8ce019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46616/0 (socket says 172.21.15.3:46616) 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 -- 172.21.15.3:0/2926770879 learned_addr learned my addr 172.21.15.3:0/2926770879 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce4a00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ce00761c0 0x7f8ce019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ce0112e20 0x7f8ce01a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 -- 172.21.15.3:0/2926770879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ce0112e20 msgr2=0x7f8ce01a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ce0112e20 0x7f8ce01a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 -- 172.21.15.3:0/2926770879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ce00761c0 msgr2=0x7f8ce019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ce00761c0 0x7f8ce019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 -- 172.21.15.3:0/2926770879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce01a3e40 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce4a00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ce00761c0 0x7f8ce019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ce0112e20 0x7f8ce01a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.047+0000 7f8ce5400700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0074900 0x7f8ce019ee30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400c920 tx=0x7f8cd400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.048+0000 7f8cde000700 1 -- 172.21.15.3:0/2926770879 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4007b00 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.048+0000 7f8cde000700 1 -- 172.21.15.3:0/2926770879 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8cd4004ce0 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.048+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce01a4130 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.048+0000 7f8cde000700 1 -- 172.21.15.3:0/2926770879 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4005600 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.048+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce01ab9b0 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.049+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce0068e50 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.051+0000 7f8cde000700 1 -- 172.21.15.3:0/2926770879 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8cd4007ca0 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.054+0000 7f8cde000700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8cc80a3030 0x7f8cc80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.054+0000 7f8cde000700 1 -- 172.21.15.3:0/2926770879 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8cd40cef30 con 0x7f8ce0074900 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.054+0000 7f8ce4a00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8cc80a3030 0x7f8cc80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.054+0000 7f8ce4a00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8cc80a3030 0x7f8cc80a54f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005fd0 tx=0x7f8ccc005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:36.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.057+0000 7f8cde000700 1 -- 172.21.15.3:0/2926770879 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8cd40fd740 con 0x7f8ce0074900 2024-03-21T08:27:36.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.286+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ce01a0190 con 0x7f8ce0074900 2024-03-21T08:27:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.288+0000 7f8cde000700 1 -- 172.21.15.3:0/2926770879 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+18363 (secure 0 0 0) 0x7f8cd4014030 con 0x7f8ce0074900 2024-03-21T08:27:36.291 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:36.291 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:27:30.128868+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:27:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8cc80a3030 msgr2=0x7f8cc80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8cc80a3030 0x7f8cc80a54f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005fd0 tx=0x7f8ccc005dc0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0074900 msgr2=0x7f8ce019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0074900 0x7f8ce019ee30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400c920 tx=0x7f8cd400cce0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 shutdown_connections 2024-03-21T08:27:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8cc80a3030 0x7f8cc80a54f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce0074900 0x7f8ce019ee30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ce0112e20 0x7f8ce01a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 --2- 172.21.15.3:0/2926770879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ce00761c0 0x7f8ce019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 >> 172.21.15.3:0/2926770879 conn(0x7f8ce00792c0 msgr2=0x7f8ce0111be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 shutdown_connections 2024-03-21T08:27:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:36.291+0000 7f8ce7e00700 1 -- 172.21.15.3:0/2926770879 wait complete. 2024-03-21T08:27:37.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:36 smithi171 ceph-mon[35388]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:37.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:36 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2926770879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:27:37.136 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:36 smithi003 ceph-mon[31675]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:37.136 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:36 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2926770879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:27:37.213 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:27:37.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T08:27:37.382 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:36 smithi084 ceph-mon[35477]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:37.383 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:36 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2926770879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:27:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:27:39.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:38 smithi171 ceph-mon[35388]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:39.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:38 smithi003 ceph-mon[31675]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:39.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:38 smithi084 ceph-mon[35477]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 -- 172.21.15.3:0/3682512119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102bd0 msgr2=0x7fd97410f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 --2- 172.21.15.3:0/3682512119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102bd0 0x7fd97410f060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b0a0 tx=0x7fd96c00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 -- 172.21.15.3:0/3682512119 shutdown_connections 2024-03-21T08:27:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 --2- 172.21.15.3:0/3682512119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102bd0 0x7fd97410f060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 --2- 172.21.15.3:0/3682512119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd974108260 0x7fd974108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 --2- 172.21.15.3:0/3682512119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd974102210 0x7fd974102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 -- 172.21.15.3:0/3682512119 >> 172.21.15.3:0/3682512119 conn(0x7fd9740792e0 msgr2=0x7fd9740796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 -- 172.21.15.3:0/3682512119 shutdown_connections 2024-03-21T08:27:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.130+0000 7fd97bc00700 1 -- 172.21.15.3:0/3682512119 wait complete. 2024-03-21T08:27:40.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.131+0000 7fd97bc00700 1 Processor -- start 2024-03-21T08:27:40.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.131+0000 7fd97bc00700 1 -- start start 2024-03-21T08:27:40.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.131+0000 7fd97bc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102210 0x7fd97419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd97bc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd974102bd0 0x7fd97419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd97bc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd974108260 0x7fd974198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd97bc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741123a0 con 0x7fd974102210 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd97bc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd974112220 con 0x7fd974102bd0 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd97bc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd974112520 con 0x7fd974108260 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd979200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102210 0x7fd97419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd979200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102210 0x7fd97419e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57702/0 (socket says 172.21.15.3:57702) 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd979200700 1 -- 172.21.15.3:0/1257641659 learned_addr learned my addr 172.21.15.3:0/1257641659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd96be00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd974102bd0 0x7fd97419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd979c00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd974108260 0x7fd974198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.132+0000 7fd979200700 1 -- 172.21.15.3:0/1257641659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd974108260 msgr2=0x7fd974198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd979200700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd974108260 0x7fd974198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd979200700 1 -- 172.21.15.3:0/1257641659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd974102bd0 msgr2=0x7fd97419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd979200700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd974102bd0 0x7fd97419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd979200700 1 -- 172.21.15.3:0/1257641659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd974198f20 con 0x7fd974102210 2024-03-21T08:27:40.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd96be00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd974102bd0 0x7fd97419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:27:40.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd979200700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102210 0x7fd97419e570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c9b0 tx=0x7fd95c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd972000700 1 -- 172.21.15.3:0/1257641659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c007b00 con 0x7fd974102210 2024-03-21T08:27:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd972000700 1 -- 172.21.15.3:0/1257641659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd95c007ca0 con 0x7fd974102210 2024-03-21T08:27:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd974199210 con 0x7fd974102210 2024-03-21T08:27:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd972000700 1 -- 172.21.15.3:0/1257641659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c018730 con 0x7fd974102210 2024-03-21T08:27:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.133+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9741a54f0 con 0x7fd974102210 2024-03-21T08:27:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.135+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97410a640 con 0x7fd974102210 2024-03-21T08:27:40.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.137+0000 7fd972000700 1 -- 172.21.15.3:0/1257641659 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd95c00f450 con 0x7fd974102210 2024-03-21T08:27:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.139+0000 7fd972000700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9540a2fb0 0x7fd9540a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.139+0000 7fd972000700 1 -- 172.21.15.3:0/1257641659 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd95c0cefd0 con 0x7fd974102210 2024-03-21T08:27:40.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.139+0000 7fd96be00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9540a2fb0 0x7fd9540a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:40.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.140+0000 7fd96be00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9540a2fb0 0x7fd9540a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd974199e00 tx=0x7fd96400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:40.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.144+0000 7fd972000700 1 -- 172.21.15.3:0/1257641659 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd95c014030 con 0x7fd974102210 2024-03-21T08:27:40.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.367+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd974199b70 con 0x7fd974102210 2024-03-21T08:27:40.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.368+0000 7fd972000700 1 -- 172.21.15.3:0/1257641659 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd95c09c200 con 0x7fd974102210 2024-03-21T08:27:40.370 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:27:40.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9540a2fb0 msgr2=0x7fd9540a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:40.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9540a2fb0 0x7fd9540a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd974199e00 tx=0x7fd96400b2c0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102210 msgr2=0x7fd97419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:40.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102210 0x7fd97419e570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c9b0 tx=0x7fd95c00cd70 comp rx=0 tx=0).stop 2024-03-21T08:27:40.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 shutdown_connections 2024-03-21T08:27:40.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9540a2fb0 0x7fd9540a5470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd974102210 0x7fd97419e570 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd974108260 0x7fd974198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 --2- 172.21.15.3:0/1257641659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd974102bd0 0x7fd97419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:40.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.371+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 >> 172.21.15.3:0/1257641659 conn(0x7fd9740792e0 msgr2=0x7fd974100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:40.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.372+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 shutdown_connections 2024-03-21T08:27:40.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:40.372+0000 7fd97bc00700 1 -- 172.21.15.3:0/1257641659 wait complete. 2024-03-21T08:27:41.094 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:40 smithi003 ceph-mon[31675]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:41.094 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:40 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1257641659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.a on smithi003 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.d on smithi003 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.g on smithi003 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.j on smithi003 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.b on smithi084 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.e on smithi084 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.h on smithi084 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.k on smithi084 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.c on smithi171 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.f on smithi171 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.i on smithi171 2024-03-21T08:27:41.159 INFO:tasks.cephadm:Adding mds.l on smithi171 2024-03-21T08:27:41.160 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2024-03-21T08:27:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:40 smithi171 ceph-mon[35388]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:41.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:40 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1257641659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:27:41.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:40 smithi084 ceph-mon[35477]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:41.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:40 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1257641659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:27:41.614 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:27:43.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:42 smithi171 ceph-mon[35388]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:43.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:42 smithi003 ceph-mon[31675]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:43.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:42 smithi084 ceph-mon[35477]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 -- 172.21.15.171:0/4058252098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618107eb0 msgr2=0x7f561810a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 --2- 172.21.15.171:0/4058252098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618107eb0 0x7f561810a320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f561000b0a0 tx=0x7f561000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 -- 172.21.15.171:0/4058252098 shutdown_connections 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 --2- 172.21.15.171:0/4058252098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618107eb0 0x7f561810a320 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 --2- 172.21.15.171:0/4058252098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5618102c50 0x7f5618105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 --2- 172.21.15.171:0/4058252098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5618105580 0x7f5618107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 -- 172.21.15.171:0/4058252098 >> 172.21.15.171:0/4058252098 conn(0x7f56180fc660 msgr2=0x7f56180feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 -- 172.21.15.171:0/4058252098 shutdown_connections 2024-03-21T08:27:43.660 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.658+0000 7f5620200700 1 -- 172.21.15.171:0/4058252098 wait complete. 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 Processor -- start 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 -- start start 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5618102c50 0x7f56181025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618105580 0x7f5618100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5618107eb0 0x7f5618101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561810dd20 con 0x7f5618107eb0 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561810dba0 con 0x7f5618102c50 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.659+0000 7f5620200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561810dea0 con 0x7f5618105580 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618105580 0x7f5618100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618105580 0x7f5618100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:32872/0 (socket says 172.21.15.171:32872) 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 -- 172.21.15.171:0/3846451591 learned_addr learned my addr 172.21.15.171:0/3846451591 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-21T08:27:43.663 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561d800700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5618102c50 0x7f56181025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561e200700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5618107eb0 0x7f5618101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 -- 172.21.15.171:0/3846451591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5618102c50 msgr2=0x7f56181025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5618102c50 0x7f56181025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 -- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5618107eb0 msgr2=0x7f5618101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5618107eb0 0x7f5618101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.660+0000 7f561ce00700 1 -- 172.21.15.171:0/3846451591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56181019f0 con 0x7f5618105580 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.661+0000 7f561ce00700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618105580 0x7f5618100c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f560400b530 tx=0x7f560400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.661+0000 7f5616000700 1 -- 172.21.15.171:0/3846451591 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604015020 con 0x7f5618105580 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.661+0000 7f5616000700 1 -- 172.21.15.171:0/3846451591 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5604011ce0 con 0x7f5618105580 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.661+0000 7f5616000700 1 -- 172.21.15.171:0/3846451591 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604010d60 con 0x7f5618105580 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.661+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56181a2560 con 0x7f5618105580 2024-03-21T08:27:43.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.661+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56181a2b20 con 0x7f5618105580 2024-03-21T08:27:43.665 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.662+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5618068e50 con 0x7f5618105580 2024-03-21T08:27:43.665 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.664+0000 7f5616000700 1 -- 172.21.15.171:0/3846451591 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f5604018020 con 0x7f5618105580 2024-03-21T08:27:43.667 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.666+0000 7f5616000700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f56000a3030 0x7f56000a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:43.667 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.666+0000 7f5616000700 1 -- 172.21.15.171:0/3846451591 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f56040ce7d0 con 0x7f5618105580 2024-03-21T08:27:43.668 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.666+0000 7f561d800700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f56000a3030 0x7f56000a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:43.668 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.667+0000 7f561d800700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f56000a3030 0x7f56000a54f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f560c006fd0 tx=0x7f560c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:43.672 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.670+0000 7f5616000700 1 -- 172.21.15.171:0/3846451591 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f560401f020 con 0x7f5618105580 2024-03-21T08:27:43.906 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.904+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f56180627d0 con 0x7f56000a3030 2024-03-21T08:27:43.922 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.921+0000 7f5616000700 1 -- 172.21.15.171:0/3846451591 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f56180627d0 con 0x7f56000a3030 2024-03-21T08:27:43.922 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2024-03-21T08:27:43.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f56000a3030 msgr2=0x7f56000a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:43.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f56000a3030 0x7f56000a54f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f560c006fd0 tx=0x7f560c008040 comp rx=0 tx=0).stop 2024-03-21T08:27:43.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618105580 msgr2=0x7f5618100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:43.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618105580 0x7f5618100c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f560400b530 tx=0x7f560400b8f0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 shutdown_connections 2024-03-21T08:27:43.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f56000a3030 0x7f56000a54f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5618107eb0 0x7f5618101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5618102c50 0x7f56181025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 --2- 172.21.15.171:0/3846451591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5618105580 0x7f5618100c00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:43.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 >> 172.21.15.171:0/3846451591 conn(0x7f56180fc660 msgr2=0x7f5618103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:43.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 shutdown_connections 2024-03-21T08:27:43.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-21T08:27:43.924+0000 7f5620200700 1 -- 172.21.15.171:0/3846451591 wait complete. 2024-03-21T08:27:44.782 DEBUG:teuthology.orchestra.run.smithi003:mds.a> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.a.service 2024-03-21T08:27:44.790 DEBUG:teuthology.orchestra.run.smithi003:mds.d> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.d.service 2024-03-21T08:27:44.837 DEBUG:teuthology.orchestra.run.smithi003:mds.g> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.g.service 2024-03-21T08:27:44.883 DEBUG:teuthology.orchestra.run.smithi003:mds.j> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.j.service 2024-03-21T08:27:44.929 DEBUG:teuthology.orchestra.run.smithi084:mds.b> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.b.service 2024-03-21T08:27:44.937 DEBUG:teuthology.orchestra.run.smithi084:mds.e> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.e.service 2024-03-21T08:27:44.983 DEBUG:teuthology.orchestra.run.smithi084:mds.h> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.h.service 2024-03-21T08:27:45.030 DEBUG:teuthology.orchestra.run.smithi084:mds.k> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.k.service 2024-03-21T08:27:45.045 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:45.045 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:27:45.045 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-21T08:27:45.045 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:45.045 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:45.045 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:45.046 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:45.046 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:45.046 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:45.046 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:45.046 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:45.046 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:44 smithi171 ceph-mon[35388]: Deploying daemon mds.l on smithi171 2024-03-21T08:27:45.075 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.c.service 2024-03-21T08:27:45.083 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.f.service 2024-03-21T08:27:45.128 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.i.service 2024-03-21T08:27:45.174 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.l.service 2024-03-21T08:27:45.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:45.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:44 smithi003 ceph-mon[31675]: Deploying daemon mds.l on smithi171 2024-03-21T08:27:45.219 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-21T08:27:45.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:27:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:27:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-21T08:27:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:27:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:45.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:27:45.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:45.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:45.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:45.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:45.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:44 smithi084 ceph-mon[35477]: Deploying daemon mds.l on smithi171 2024-03-21T08:27:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 -- 172.21.15.3:0/2902034431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a0102790 msgr2=0x7f39a0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 --2- 172.21.15.3:0/2902034431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a0102790 0x7f39a0102c10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3988009900 tx=0x7f398802bc30 comp rx=0 tx=0).stop 2024-03-21T08:27:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 -- 172.21.15.3:0/2902034431 shutdown_connections 2024-03-21T08:27:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 --2- 172.21.15.3:0/2902034431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a0103150 0x7f39a01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 --2- 172.21.15.3:0/2902034431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a0102790 0x7f39a0102c10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 --2- 172.21.15.3:0/2902034431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a01015a0 0x7f39a01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 -- 172.21.15.3:0/2902034431 >> 172.21.15.3:0/2902034431 conn(0x7f39a00fcb30 msgr2=0x7f39a00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 -- 172.21.15.3:0/2902034431 shutdown_connections 2024-03-21T08:27:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.482+0000 7f399f200640 1 -- 172.21.15.3:0/2902034431 wait complete. 2024-03-21T08:27:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 Processor -- start 2024-03-21T08:27:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 -- start start 2024-03-21T08:27:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a01015a0 0x7f39a01999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a0102790 0x7f39a0199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a0103150 0x7f39a01a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a010d0e0 con 0x7f39a01015a0 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a010cf60 con 0x7f39a0102790 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a010d260 con 0x7f39a0103150 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.484+0000 7f399de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a01015a0 0x7f39a01999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a01015a0 0x7f39a01999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57716/0 (socket says 172.21.15.3:57716) 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 -- 172.21.15.3:0/4273533180 learned_addr learned my addr 172.21.15.3:0/4273533180 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f3997e00640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a0102790 0x7f39a0199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399e800640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a0103150 0x7f39a01a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 -- 172.21.15.3:0/4273533180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a0103150 msgr2=0x7f39a01a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a0103150 0x7f39a01a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 -- 172.21.15.3:0/4273533180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a0102790 msgr2=0x7f39a0199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a0102790 0x7f39a0199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 -- 172.21.15.3:0/4273533180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a01a15d0 con 0x7f39a01015a0 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399e800640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a0103150 0x7f39a01a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.485+0000 7f399de00640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a01015a0 0x7f39a01999b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f399000c970 tx=0x7f399000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.486+0000 7f3996a00640 1 -- 172.21.15.3:0/4273533180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990007c10 con 0x7f39a01015a0 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.486+0000 7f3996a00640 1 -- 172.21.15.3:0/4273533180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3990007db0 con 0x7f39a01015a0 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.486+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a01a18c0 con 0x7f39a01015a0 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.486+0000 7f3996a00640 1 -- 172.21.15.3:0/4273533180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000f870 con 0x7f39a01015a0 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.486+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a01a1de0 con 0x7f39a01015a0 2024-03-21T08:27:45.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.487+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a01a2040 con 0x7f39a01015a0 2024-03-21T08:27:45.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.490+0000 7f3996a00640 1 -- 172.21.15.3:0/4273533180 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f399000fa70 con 0x7f39a01015a0 2024-03-21T08:27:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.491+0000 7f3996a00640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f39840a2f10 0x7f39840a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.491+0000 7f3996a00640 1 -- 172.21.15.3:0/4273533180 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f39900cf3b0 con 0x7f39a01015a0 2024-03-21T08:27:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.491+0000 7f3997e00640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f39840a2f10 0x7f39840a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.492+0000 7f3997e00640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f39840a2f10 0x7f39840a53d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3988006010 tx=0x7f398802c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:45.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.496+0000 7f3996a00640 1 -- 172.21.15.3:0/4273533180 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f399009c670 con 0x7f39a01015a0 2024-03-21T08:27:45.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.700+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39a01a2210 con 0x7f39a01015a0 2024-03-21T08:27:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.702+0000 7f3996a00640 1 -- 172.21.15.3:0/4273533180 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+18363 (secure 0 0 0) 0x7f399009bf70 con 0x7f39a01015a0 2024-03-21T08:27:45.705 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:45.705 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:27:30.128868+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f39840a2f10 msgr2=0x7f39840a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f39840a2f10 0x7f39840a53d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3988006010 tx=0x7f398802c740 comp rx=0 tx=0).stop 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a01015a0 msgr2=0x7f39a01999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a01015a0 0x7f39a01999b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f399000c970 tx=0x7f399000ce30 comp rx=0 tx=0).stop 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 shutdown_connections 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f39840a2f10 0x7f39840a53d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39a0103150 0x7f39a01a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a0102790 0x7f39a0199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 --2- 172.21.15.3:0/4273533180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a01015a0 0x7f39a01999b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 >> 172.21.15.3:0/4273533180 conn(0x7f39a00fcb30 msgr2=0x7f39a00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 shutdown_connections 2024-03-21T08:27:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.705+0000 7f399f200640 1 -- 172.21.15.3:0/4273533180 wait complete. 2024-03-21T08:27:45.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:27:45.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:27:45.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.956+0000 7fe1b7200640 1 -- 172.21.15.3:0/1331264359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b01096b0 msgr2=0x7fe1b010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.956+0000 7fe1b7200640 1 --2- 172.21.15.3:0/1331264359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b01096b0 0x7fe1b010ff40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009ca0 tx=0x7fe1ac02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.956+0000 7fe1b7200640 1 -- 172.21.15.3:0/1331264359 shutdown_connections 2024-03-21T08:27:45.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.956+0000 7fe1b7200640 1 --2- 172.21.15.3:0/1331264359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b01096b0 0x7fe1b010ff40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.957+0000 7fe1b7200640 1 --2- 172.21.15.3:0/1331264359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1b0108cf0 0x7fe1b0109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.957+0000 7fe1b7200640 1 --2- 172.21.15.3:0/1331264359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b0107b00 0x7fe1b0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.957+0000 7fe1b7200640 1 -- 172.21.15.3:0/1331264359 >> 172.21.15.3:0/1331264359 conn(0x7fe1b0076a90 msgr2=0x7fe1b0078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:45.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.957+0000 7fe1b7200640 1 -- 172.21.15.3:0/1331264359 shutdown_connections 2024-03-21T08:27:45.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.957+0000 7fe1b7200640 1 -- 172.21.15.3:0/1331264359 wait complete. 2024-03-21T08:27:45.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 Processor -- start 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 -- start start 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b0107b00 0x7fe1b019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b0108cf0 0x7fe1b019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1b01096b0 0x7fe1b01a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0113630 con 0x7fe1b0108cf0 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b01134b0 con 0x7fe1b01096b0 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.958+0000 7fe1b7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b01137b0 con 0x7fe1b0107b00 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b0108cf0 0x7fe1b019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b0108cf0 0x7fe1b019e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57740/0 (socket says 172.21.15.3:57740) 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 -- 172.21.15.3:0/3382028094 learned_addr learned my addr 172.21.15.3:0/3382028094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5e00640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b0107b00 0x7fe1b019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b6800640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1b01096b0 0x7fe1b01a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 -- 172.21.15.3:0/3382028094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b0107b00 msgr2=0x7fe1b019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b0107b00 0x7fe1b019dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 -- 172.21.15.3:0/3382028094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1b01096b0 msgr2=0x7fe1b01a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1b01096b0 0x7fe1b01a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 -- 172.21.15.3:0/3382028094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b01a5900 con 0x7fe1b0108cf0 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.959+0000 7fe1b5400640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b0108cf0 0x7fe1b019e230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000d950 tx=0x7fe1a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.960+0000 7fe1aaa00640 1 -- 172.21.15.3:0/3382028094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0017070 con 0x7fe1b0108cf0 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.960+0000 7fe1aaa00640 1 -- 172.21.15.3:0/3382028094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1a0009c30 con 0x7fe1b0108cf0 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.960+0000 7fe1aaa00640 1 -- 172.21.15.3:0/3382028094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000f620 con 0x7fe1b0108cf0 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.960+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b01a5bf0 con 0x7fe1b0108cf0 2024-03-21T08:27:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.960+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b01a6190 con 0x7fe1b0108cf0 2024-03-21T08:27:45.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.961+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b010eea0 con 0x7fe1b0108cf0 2024-03-21T08:27:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.964+0000 7fe1aaa00640 1 -- 172.21.15.3:0/3382028094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fe1a00109a0 con 0x7fe1b0108cf0 2024-03-21T08:27:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.966+0000 7fe1aaa00640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1980a2fa0 0x7fe1980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.966+0000 7fe1aaa00640 1 -- 172.21.15.3:0/3382028094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1a00cec40 con 0x7fe1b0108cf0 2024-03-21T08:27:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.966+0000 7fe1b5e00640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1980a2fa0 0x7fe1980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.967+0000 7fe1b5e00640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1980a2fa0 0x7fe1980a5460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4005fd0 tx=0x7fe1a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:45.968+0000 7fe1aaa00640 1 -- 172.21.15.3:0/3382028094 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe1a0009de0 con 0x7fe1b0108cf0 2024-03-21T08:27:46.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.171+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1b002d200 con 0x7fe1b0108cf0 2024-03-21T08:27:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.172+0000 7fe1aaa00640 1 -- 172.21.15.3:0/3382028094 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1a009bf90 con 0x7fe1b0108cf0 2024-03-21T08:27:46.174 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:27:46.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:45 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4273533180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:27:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1980a2fa0 msgr2=0x7fe1980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1980a2fa0 0x7fe1980a5460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4005fd0 tx=0x7fe1a4005850 comp rx=0 tx=0).stop 2024-03-21T08:27:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b0108cf0 msgr2=0x7fe1b019e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b0108cf0 0x7fe1b019e230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000d950 tx=0x7fe1a000de10 comp rx=0 tx=0).stop 2024-03-21T08:27:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 shutdown_connections 2024-03-21T08:27:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1b01096b0 0x7fe1b01a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b0108cf0 0x7fe1b019e230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1980a2fa0 0x7fe1980a5460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 --2- 172.21.15.3:0/3382028094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b0107b00 0x7fe1b019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 >> 172.21.15.3:0/3382028094 conn(0x7fe1b0076a90 msgr2=0x7fe1b010c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:46.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 shutdown_connections 2024-03-21T08:27:46.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.175+0000 7fe1b7200640 1 -- 172.21.15.3:0/3382028094 wait complete. 2024-03-21T08:27:46.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:27:46.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:45 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4273533180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:27:46.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:45 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4273533180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:27:46.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 -- 172.21.15.3:0/2913349635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 msgr2=0x7f2a2010ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:46.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 --2- 172.21.15.3:0/2913349635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a2010ff50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009cd0 tx=0x7f2a1802c0f0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 -- 172.21.15.3:0/2913349635 shutdown_connections 2024-03-21T08:27:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 --2- 172.21.15.3:0/2913349635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a2010ff50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 --2- 172.21.15.3:0/2913349635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a20074100 0x7f2a20074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 --2- 172.21.15.3:0/2913349635 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a200756e0 0x7f2a20073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 -- 172.21.15.3:0/2913349635 >> 172.21.15.3:0/2913349635 conn(0x7f2a200fcb00 msgr2=0x7f2a200fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 -- 172.21.15.3:0/2913349635 shutdown_connections 2024-03-21T08:27:46.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.408+0000 7f2a25a00640 1 -- 172.21.15.3:0/2913349635 wait complete. 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.409+0000 7f2a25a00640 1 Processor -- start 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.409+0000 7f2a25a00640 1 -- start start 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a20074100 0x7f2a2019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a200756e0 0x7f2a2019e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a201a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a201136b0 con 0x7f2a2010db60 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20113530 con 0x7f2a20074100 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20113830 con 0x7f2a200756e0 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a201a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a201a5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57754/0 (socket says 172.21.15.3:57754) 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a25000640 1 -- 172.21.15.3:0/760513062 learned_addr learned my addr 172.21.15.3:0/760513062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a17400640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a200756e0 0x7f2a2019e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.410+0000 7f2a1fe00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a20074100 0x7f2a2019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25000640 1 -- 172.21.15.3:0/760513062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a200756e0 msgr2=0x7f2a2019e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25000640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a200756e0 0x7f2a2019e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25000640 1 -- 172.21.15.3:0/760513062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a20074100 msgr2=0x7f2a2019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25000640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a20074100 0x7f2a2019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25000640 1 -- 172.21.15.3:0/760513062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a201a5970 con 0x7f2a2010db60 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a1fe00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a20074100 0x7f2a2019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25000640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a201a5300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802c630 tx=0x7f2a180328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a1d600640 1 -- 172.21.15.3:0/760513062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18041020 con 0x7f2a2010db60 2024-03-21T08:27:46.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a201a5c00 con 0x7f2a2010db60 2024-03-21T08:27:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a1d600640 1 -- 172.21.15.3:0/760513062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a18003e20 con 0x7f2a2010db60 2024-03-21T08:27:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a1d600640 1 -- 172.21.15.3:0/760513062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18039040 con 0x7f2a2010db60 2024-03-21T08:27:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.411+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a201a60f0 con 0x7f2a2010db60 2024-03-21T08:27:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.413+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a20103c20 con 0x7f2a2010db60 2024-03-21T08:27:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.414+0000 7f2a1d600640 1 -- 172.21.15.3:0/760513062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f2a18044020 con 0x7f2a2010db60 2024-03-21T08:27:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.416+0000 7f2a1d600640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a040a2f10 0x7f2a040a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.416+0000 7f2a1d600640 1 -- 172.21.15.3:0/760513062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a180f1390 con 0x7f2a2010db60 2024-03-21T08:27:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.416+0000 7f2a1fe00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a040a2f10 0x7f2a040a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.417+0000 7f2a1fe00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a040a2f10 0x7f2a040a53d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009b50 tx=0x7f2a10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.420+0000 7f2a1d600640 1 -- 172.21.15.3:0/760513062 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a1804b020 con 0x7f2a2010db60 2024-03-21T08:27:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.663+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a200622a0 con 0x7f2a2010db60 2024-03-21T08:27:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.664+0000 7f2a1d600640 1 -- 172.21.15.3:0/760513062 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f2a1803b2f0 con 0x7f2a2010db60 2024-03-21T08:27:46.666 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:46.666 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-21T08:27:46.666 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 1 2024-03-21T08:27:46.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.666+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a040a2f10 msgr2=0x7f2a040a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.666+0000 7f2a25a00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a040a2f10 0x7f2a040a53d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009b50 tx=0x7f2a10005f90 comp rx=0 tx=0).stop 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.666+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 msgr2=0x7f2a201a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.666+0000 7f2a25a00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a201a5300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802c630 tx=0x7f2a180328a0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 shutdown_connections 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a2010db60 0x7f2a201a5300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a040a2f10 0x7f2a040a53d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a200756e0 0x7f2a2019e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 --2- 172.21.15.3:0/760513062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a20074100 0x7f2a2019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 >> 172.21.15.3:0/760513062 conn(0x7f2a200fcb00 msgr2=0x7f2a20100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 shutdown_connections 2024-03-21T08:27:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:46.667+0000 7f2a25a00640 1 -- 172.21.15.3:0/760513062 wait complete. 2024-03-21T08:27:47.059 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:46 smithi171 ceph-mon[35388]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:47.059 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:46 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3382028094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:27:47.059 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:46 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/760513062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:47.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:46 smithi084 ceph-mon[35477]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:47.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:46 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3382028094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:27:47.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:46 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/760513062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:47.384 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 systemd[1]: Starting Ceph mds.l for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:27:47.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:46 smithi003 ceph-mon[31675]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:46 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3382028094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:27:47.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:46 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/760513062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:47.771 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 podman[65947]: 2024-03-21 08:27:47.332110403 +0000 UTC m=+0.039021382 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:27:47.771 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 podman[65947]: 2024-03-21 08:27:47.505861588 +0000 UTC m=+0.212772585 container create 378e40e0af04376a4f0040091aefdb339ee0ef08acf0ed5b65a0ab6c6b882f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=) 2024-03-21T08:27:47.771 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 podman[65947]: 2024-03-21 08:27:47.768378683 +0000 UTC m=+0.475289687 container init 378e40e0af04376a4f0040091aefdb339ee0ef08acf0ed5b65a0ab6c6b882f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T08:27:48.134 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 podman[65947]: 2024-03-21 08:27:47.778681884 +0000 UTC m=+0.485592871 container start 378e40e0af04376a4f0040091aefdb339ee0ef08acf0ed5b65a0ab6c6b882f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T08:27:48.134 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 ceph-mds[65966]: -- 172.21.15.171:0/2982957597 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56307db15a40 con 0x56307dae8800 2024-03-21T08:27:48.134 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l[65962]: starting mds.l at 2024-03-21T08:27:48.134 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 bash[65947]: 378e40e0af04376a4f0040091aefdb339ee0ef08acf0ed5b65a0ab6c6b882f6e 2024-03-21T08:27:48.134 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:27:47 smithi171 systemd[1]: Started Ceph mds.l for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:27:48.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:27:48.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.923+0000 7fe427200640 1 -- 172.21.15.3:0/4071575901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 msgr2=0x7fe428104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.923+0000 7fe427200640 1 --2- 172.21.15.3:0/4071575901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428104e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe410009900 tx=0x7fe41002bc30 comp rx=0 tx=0).stop 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.923+0000 7fe427200640 1 -- 172.21.15.3:0/4071575901 shutdown_connections 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.923+0000 7fe427200640 1 --2- 172.21.15.3:0/4071575901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe428105390 0x7fe42810bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.923+0000 7fe427200640 1 --2- 172.21.15.3:0/4071575901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428104e50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.923+0000 7fe427200640 1 --2- 172.21.15.3:0/4071575901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4281037e0 0x7fe428103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.923+0000 7fe427200640 1 -- 172.21.15.3:0/4071575901 >> 172.21.15.3:0/4071575901 conn(0x7fe4280fed70 msgr2=0x7fe428101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.924+0000 7fe427200640 1 -- 172.21.15.3:0/4071575901 shutdown_connections 2024-03-21T08:27:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.924+0000 7fe427200640 1 -- 172.21.15.3:0/4071575901 wait complete. 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 Processor -- start 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 -- start start 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4281037e0 0x7fe428199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe428105390 0x7fe4281a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42810f340 con 0x7fe4281049d0 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42810f1c0 con 0x7fe428105390 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe427200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42810f4c0 con 0x7fe4281037e0 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe41d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:48.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe41d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57784/0 (socket says 172.21.15.3:57784) 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.925+0000 7fe41d400640 1 -- 172.21.15.3:0/3208805333 learned_addr learned my addr 172.21.15.3:0/3208805333 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe426800640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe428105390 0x7fe4281a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe425e00640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4281037e0 0x7fe428199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe41d400640 1 -- 172.21.15.3:0/3208805333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4281037e0 msgr2=0x7fe428199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe41d400640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4281037e0 0x7fe428199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe41d400640 1 -- 172.21.15.3:0/3208805333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe428105390 msgr2=0x7fe4281a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe41d400640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe428105390 0x7fe4281a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe41d400640 1 -- 172.21.15.3:0/3208805333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4281a15b0 con 0x7fe4281049d0 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe426800640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe428105390 0x7fe4281a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe41d400640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428199ed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe410000c00 tx=0x7fe4100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.926+0000 7fe41f400640 1 -- 172.21.15.3:0/3208805333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410043020 con 0x7fe4281049d0 2024-03-21T08:27:48.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.927+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4281a1840 con 0x7fe4281049d0 2024-03-21T08:27:48.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.927+0000 7fe41f400640 1 -- 172.21.15.3:0/3208805333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4100054c0 con 0x7fe4281049d0 2024-03-21T08:27:48.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.927+0000 7fe41f400640 1 -- 172.21.15.3:0/3208805333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410003d40 con 0x7fe4281049d0 2024-03-21T08:27:48.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.927+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4281a1de0 con 0x7fe4281049d0 2024-03-21T08:27:48.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.928+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42810aaa0 con 0x7fe4281049d0 2024-03-21T08:27:48.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.930+0000 7fe41f400640 1 -- 172.21.15.3:0/3208805333 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fe410046020 con 0x7fe4281049d0 2024-03-21T08:27:48.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.931+0000 7fe41f400640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe40c0a2f10 0x7fe40c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:48.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.931+0000 7fe41f400640 1 -- 172.21.15.3:0/3208805333 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4100f1830 con 0x7fe4281049d0 2024-03-21T08:27:48.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.932+0000 7fe425e00640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe40c0a2f10 0x7fe40c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:48.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.932+0000 7fe425e00640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe40c0a2f10 0x7fe40c0a53d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe418005fd0 tx=0x7fe418005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:48.934+0000 7fe41f400640 1 -- 172.21.15.3:0/3208805333 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe41004d020 con 0x7fe4281049d0 2024-03-21T08:27:49.096 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:49.096 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.171:6832/3477321197,v1:172.21.15.171:6833/3477321197] up:boot 2024-03-21T08:27:49.096 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: fsmap 1 up:standby 2024-03-21T08:27:49.096 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:27:49.096 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.096 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.097 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.097 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:49.097 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:49.097 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:49.097 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:48 smithi084 ceph-mon[35477]: Deploying daemon mds.k on smithi084 2024-03-21T08:27:49.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:49.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.171:6832/3477321197,v1:172.21.15.171:6833/3477321197] up:boot 2024-03-21T08:27:49.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: fsmap 1 up:standby 2024-03-21T08:27:49.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:27:49.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:49.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:49.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:49.135 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:48 smithi171 ceph-mon[35388]: Deploying daemon mds.k on smithi084 2024-03-21T08:27:49.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:49.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.171:6832/3477321197,v1:172.21.15.171:6833/3477321197] up:boot 2024-03-21T08:27:49.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: fsmap 1 up:standby 2024-03-21T08:27:49.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:27:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:49.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:48 smithi003 ceph-mon[31675]: Deploying daemon mds.k on smithi084 2024-03-21T08:27:49.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.177+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe42802d070 con 0x7fe4281049d0 2024-03-21T08:27:49.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.178+0000 7fe41f400640 1 -- 172.21.15.3:0/3208805333 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fe4100beb80 con 0x7fe4281049d0 2024-03-21T08:27:49.180 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:49.180 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T08:27:49.181 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-03-21T08:27:49.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe40c0a2f10 msgr2=0x7fe40c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:49.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe40c0a2f10 0x7fe40c0a53d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe418005fd0 tx=0x7fe418005850 comp rx=0 tx=0).stop 2024-03-21T08:27:49.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 msgr2=0x7fe428199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:49.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428199ed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe410000c00 tx=0x7fe4100042f0 comp rx=0 tx=0).stop 2024-03-21T08:27:49.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 shutdown_connections 2024-03-21T08:27:49.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe428105390 0x7fe4281a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:49.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4281049d0 0x7fe428199ed0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:49.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe40c0a2f10 0x7fe40c0a53d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:49.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 --2- 172.21.15.3:0/3208805333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4281037e0 0x7fe428199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:49.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 >> 172.21.15.3:0/3208805333 conn(0x7fe4280fed70 msgr2=0x7fe428107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:49.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.181+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 shutdown_connections 2024-03-21T08:27:49.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:49.182+0000 7fe427200640 1 -- 172.21.15.3:0/3208805333 wait complete. 2024-03-21T08:27:50.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:49 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3208805333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:50.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:49 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3208805333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:50.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:49 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3208805333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:51.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:50 smithi084 ceph-mon[35477]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:51.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:50 smithi171 ceph-mon[35388]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:51.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:50 smithi003 ceph-mon[31675]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:51.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:27:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 -- 172.21.15.3:0/713441156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 msgr2=0x7f8ee0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/713441156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0103be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009ca0 tx=0x7f8ed002c100 comp rx=0 tx=0).stop 2024-03-21T08:27:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 -- 172.21.15.3:0/713441156 shutdown_connections 2024-03-21T08:27:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/713441156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0105370 0x7f8ee010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/713441156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee01049b0 0x7f8ee0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/713441156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0103be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 -- 172.21.15.3:0/713441156 >> 172.21.15.3:0/713441156 conn(0x7f8ee00fed50 msgr2=0x7f8ee0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 -- 172.21.15.3:0/713441156 shutdown_connections 2024-03-21T08:27:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.440+0000 7f8ee4a00640 1 -- 172.21.15.3:0/713441156 wait complete. 2024-03-21T08:27:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 Processor -- start 2024-03-21T08:27:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 -- start start 2024-03-21T08:27:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee01049b0 0x7f8ee0199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0105370 0x7f8ee01a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee010f2c0 con 0x7f8ee01037c0 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee010f140 con 0x7f8ee01049b0 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8ee4a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee010f440 con 0x7f8ee0105370 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.441+0000 7f8edf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60328/0 (socket says 172.21.15.3:60328) 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 -- 172.21.15.3:0/552632827 learned_addr learned my addr 172.21.15.3:0/552632827 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edea00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee01049b0 0x7f8ee0199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edfe00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0105370 0x7f8ee01a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:51.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 -- 172.21.15.3:0/552632827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0105370 msgr2=0x7f8ee01a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0105370 0x7f8ee01a0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 -- 172.21.15.3:0/552632827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee01049b0 msgr2=0x7f8ee0199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee01049b0 0x7f8ee0199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 -- 172.21.15.3:0/552632827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee01a15a0 con 0x7f8ee01037c0 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8edf400640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0199980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0033000 tx=0x7f8ed0035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.442+0000 7f8ed7e00640 1 -- 172.21.15.3:0/552632827 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0039040 con 0x7f8ee01037c0 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.443+0000 7f8ed7e00640 1 -- 172.21.15.3:0/552632827 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ed0044070 con 0x7f8ee01037c0 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.443+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee01a1830 con 0x7f8ee01037c0 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.443+0000 7f8ed7e00640 1 -- 172.21.15.3:0/552632827 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed00044d0 con 0x7f8ee01037c0 2024-03-21T08:27:51.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.443+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee01a1d70 con 0x7f8ee01037c0 2024-03-21T08:27:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.444+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee010abb0 con 0x7f8ee01037c0 2024-03-21T08:27:51.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.446+0000 7f8ed7e00640 1 -- 172.21.15.3:0/552632827 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8ed00327c0 con 0x7f8ee01037c0 2024-03-21T08:27:51.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.448+0000 7f8ed7e00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8ec40a2f90 0x7f8ec40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:51.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.448+0000 7f8ed7e00640 1 -- 172.21.15.3:0/552632827 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8ed00f1630 con 0x7f8ee01037c0 2024-03-21T08:27:51.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.449+0000 7f8edea00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8ec40a2f90 0x7f8ec40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:51.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.449+0000 7f8edea00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8ec40a2f90 0x7f8ec40a5450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ee019ae90 tx=0x7f8ec800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:51.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.453+0000 7f8ed7e00640 1 -- 172.21.15.3:0/552632827 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8ed00be860 con 0x7f8ee01037c0 2024-03-21T08:27:51.626 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:51 smithi084 systemd[1]: Starting Ceph mds.k for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:27:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.701+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ee002d070 con 0x7f8ee01037c0 2024-03-21T08:27:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.702+0000 7f8ed7e00640 1 -- 172.21.15.3:0/552632827 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f8ed00be160 con 0x7f8ee01037c0 2024-03-21T08:27:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:51.704 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T08:27:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-03-21T08:27:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8ec40a2f90 msgr2=0x7f8ec40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8ec40a2f90 0x7f8ec40a5450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ee019ae90 tx=0x7f8ec800b410 comp rx=0 tx=0).stop 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 msgr2=0x7f8ee0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0199980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0033000 tx=0x7f8ed0035d60 comp rx=0 tx=0).stop 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 shutdown_connections 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8ec40a2f90 0x7f8ec40a5450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0105370 0x7f8ee01a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee01049b0 0x7f8ee0199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.705+0000 7f8ee4a00640 1 --2- 172.21.15.3:0/552632827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ee01037c0 0x7f8ee0199980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.706+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 >> 172.21.15.3:0/552632827 conn(0x7f8ee00fed50 msgr2=0x7f8ee0107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.706+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 shutdown_connections 2024-03-21T08:27:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:51.706+0000 7f8ee4a00640 1 -- 172.21.15.3:0/552632827 wait complete. 2024-03-21T08:27:51.972 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:51 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/552632827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:51.973 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:51 smithi084 podman[64493]: 2024-03-21 08:27:51.553293533 +0000 UTC m=+0.034133175 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:27:51.973 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:51 smithi084 podman[64493]: 2024-03-21 08:27:51.675355508 +0000 UTC m=+0.156195179 container create 0498498dd34531742d8d31142a39f1c1a9784edcd274f6d54b92f0e61f3b8ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-21T08:27:51.973 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:51 smithi084 podman[64493]: 2024-03-21 08:27:51.971247403 +0000 UTC m=+0.452087080 container init 0498498dd34531742d8d31142a39f1c1a9784edcd274f6d54b92f0e61f3b8ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T08:27:52.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:51 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/552632827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:52.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:51 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/552632827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:52.376 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:51 smithi084 podman[64493]: 2024-03-21 08:27:51.981532465 +0000 UTC m=+0.462372141 container start 0498498dd34531742d8d31142a39f1c1a9784edcd274f6d54b92f0e61f3b8ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T08:27:52.376 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:52 smithi084 ceph-mds[64512]: -- 172.21.15.84:0/735036243 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d257a29a40 con 0x55d2579fcc00 2024-03-21T08:27:52.376 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:52 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k[64508]: starting mds.k at 2024-03-21T08:27:52.376 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:52 smithi084 bash[64493]: 0498498dd34531742d8d31142a39f1c1a9784edcd274f6d54b92f0e61f3b8ca9 2024-03-21T08:27:52.377 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:27:52 smithi084 systemd[1]: Started Ceph mds.k for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:27:53.421 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: Deploying daemon mds.j on smithi003 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:boot 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: fsmap 2 up:standby 2024-03-21T08:27:53.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:53 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: Deploying daemon mds.j on smithi003 2024-03-21T08:27:53.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:boot 2024-03-21T08:27:53.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: fsmap 2 up:standby 2024-03-21T08:27:53.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:53 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: Deploying daemon mds.j on smithi003 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:boot 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: fsmap 2 up:standby 2024-03-21T08:27:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:53 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:27:53.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.951+0000 7fdb72600640 1 -- 172.21.15.3:0/3410847636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c108db0 msgr2=0x7fdb6c109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.951+0000 7fdb72600640 1 --2- 172.21.15.3:0/3410847636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c108db0 0x7fdb6c109230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009900 tx=0x7fdb5802bc40 comp rx=0 tx=0).stop 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.952+0000 7fdb72600640 1 -- 172.21.15.3:0/3410847636 shutdown_connections 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.952+0000 7fdb72600640 1 --2- 172.21.15.3:0/3410847636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb6c109770 0x7fdb6c110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.952+0000 7fdb72600640 1 --2- 172.21.15.3:0/3410847636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c108db0 0x7fdb6c109230 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.952+0000 7fdb72600640 1 --2- 172.21.15.3:0/3410847636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb6c107bc0 0x7fdb6c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.952+0000 7fdb72600640 1 -- 172.21.15.3:0/3410847636 >> 172.21.15.3:0/3410847636 conn(0x7fdb6c076b60 msgr2=0x7fdb6c078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.952+0000 7fdb72600640 1 -- 172.21.15.3:0/3410847636 shutdown_connections 2024-03-21T08:27:53.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.952+0000 7fdb72600640 1 -- 172.21.15.3:0/3410847636 wait complete. 2024-03-21T08:27:53.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.953+0000 7fdb72600640 1 Processor -- start 2024-03-21T08:27:53.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.953+0000 7fdb72600640 1 -- start start 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb72600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c107bc0 0x7fdb6c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb72600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb6c108db0 0x7fdb6c19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb72600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb6c109770 0x7fdb6c1a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb72600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c113720 con 0x7fdb6c107bc0 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb72600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c1135a0 con 0x7fdb6c109770 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb72600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c1138a0 con 0x7fdb6c108db0 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c107bc0 0x7fdb6c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c107bc0 0x7fdb6c19dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60346/0 (socket says 172.21.15.3:60346) 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 -- 172.21.15.3:0/3106843557 learned_addr learned my addr 172.21.15.3:0/3106843557 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:53.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71c00640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb6c109770 0x7fdb6c1a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 -- 172.21.15.3:0/3106843557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb6c108db0 msgr2=0x7fdb6c19e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb6be00640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb6c108db0 0x7fdb6c19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb6c108db0 0x7fdb6c19e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 -- 172.21.15.3:0/3106843557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb6c109770 msgr2=0x7fdb6c1a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb6c109770 0x7fdb6c1a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71200640 1 -- 172.21.15.3:0/3106843557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c1a58f0 con 0x7fdb6c107bc0 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.954+0000 7fdb71c00640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb6c109770 0x7fdb6c1a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.955+0000 7fdb71200640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c107bc0 0x7fdb6c19dcb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000c970 tx=0x7fdb6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.955+0000 7fdb69600640 1 -- 172.21.15.3:0/3106843557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60007c10 con 0x7fdb6c107bc0 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.955+0000 7fdb69600640 1 -- 172.21.15.3:0/3106843557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb60007db0 con 0x7fdb6c107bc0 2024-03-21T08:27:53.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.955+0000 7fdb69600640 1 -- 172.21.15.3:0/3106843557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000f870 con 0x7fdb6c107bc0 2024-03-21T08:27:53.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.955+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6c1a5be0 con 0x7fdb6c107bc0 2024-03-21T08:27:53.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.955+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c1a6130 con 0x7fdb6c107bc0 2024-03-21T08:27:53.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.957+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6c10ee10 con 0x7fdb6c107bc0 2024-03-21T08:27:53.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.959+0000 7fdb69600640 1 -- 172.21.15.3:0/3106843557 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fdb6000fa10 con 0x7fdb6c107bc0 2024-03-21T08:27:53.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.960+0000 7fdb69600640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdb540a2f10 0x7fdb540a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:53.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.960+0000 7fdb69600640 1 -- 172.21.15.3:0/3106843557 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdb600cf3e0 con 0x7fdb6c107bc0 2024-03-21T08:27:53.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.960+0000 7fdb6be00640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdb540a2f10 0x7fdb540a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:53.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.961+0000 7fdb6be00640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdb540a2f10 0x7fdb540a53d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb580096c0 tx=0x7fdb58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:53.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:53.965+0000 7fdb69600640 1 -- 172.21.15.3:0/3106843557 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb6009c6a0 con 0x7fdb6c107bc0 2024-03-21T08:27:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.210+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb6c02d070 con 0x7fdb6c107bc0 2024-03-21T08:27:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.211+0000 7fdb69600640 1 -- 172.21.15.3:0/3106843557 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fdb6009bfa0 con 0x7fdb6c107bc0 2024-03-21T08:27:54.213 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:54.213 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T08:27:54.213 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdb540a2f10 msgr2=0x7fdb540a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdb540a2f10 0x7fdb540a53d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb580096c0 tx=0x7fdb58005fd0 comp rx=0 tx=0).stop 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c107bc0 msgr2=0x7fdb6c19dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c107bc0 0x7fdb6c19dcb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000c970 tx=0x7fdb6000ce30 comp rx=0 tx=0).stop 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 shutdown_connections 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb6c109770 0x7fdb6c1a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fdb540a2f10 0x7fdb540a53d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb6c108db0 0x7fdb6c19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 --2- 172.21.15.3:0/3106843557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c107bc0 0x7fdb6c19dcb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 >> 172.21.15.3:0/3106843557 conn(0x7fdb6c076b60 msgr2=0x7fdb6c10bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 shutdown_connections 2024-03-21T08:27:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:54.214+0000 7fdb72600640 1 -- 172.21.15.3:0/3106843557 wait complete. 2024-03-21T08:27:55.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:55 smithi003 ceph-mon[31675]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:55.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:55 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3106843557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:55.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:55 smithi084 ceph-mon[35477]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:55.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:55 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3106843557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:55.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:55 smithi171 ceph-mon[35388]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:55.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:55 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3106843557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:55.857 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:55 smithi003 systemd[1]: Starting Ceph mds.j for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:27:56.142 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:55 smithi003 podman[71866]: 2024-03-21 08:27:55.760869117 +0000 UTC m=+0.036596205 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:27:56.142 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:55 smithi003 podman[71866]: 2024-03-21 08:27:55.89371713 +0000 UTC m=+0.169444258 container create a0523c19653a5c796648ceed99ac5ddcadcd97da980e05da43021102e74e2ec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T08:27:56.142 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:56 smithi003 podman[71866]: 2024-03-21 08:27:56.139408046 +0000 UTC m=+0.415135161 container init a0523c19653a5c796648ceed99ac5ddcadcd97da980e05da43021102e74e2ec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T08:27:56.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:27:56.426 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:56 smithi003 podman[71866]: 2024-03-21 08:27:56.149905771 +0000 UTC m=+0.425632886 container start a0523c19653a5c796648ceed99ac5ddcadcd97da980e05da43021102e74e2ec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:27:56.427 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:56 smithi003 ceph-mds[71885]: -- 172.21.15.3:0/1217951436 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f5fb825a40 con 0x55f5fb7f8800 2024-03-21T08:27:56.427 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j[71881]: starting mds.j at 2024-03-21T08:27:56.427 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:56 smithi003 bash[71866]: a0523c19653a5c796648ceed99ac5ddcadcd97da980e05da43021102e74e2ec9 2024-03-21T08:27:56.427 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:27:56 smithi003 systemd[1]: Started Ceph mds.j for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:27:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 -- 172.21.15.3:0/1231836713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701027b0 msgr2=0x7f0370102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 --2- 172.21.15.3:0/1231836713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701027b0 0x7f0370102c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0358009900 tx=0x7f035802bc30 comp rx=0 tx=0).stop 2024-03-21T08:27:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 -- 172.21.15.3:0/1231836713 shutdown_connections 2024-03-21T08:27:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 --2- 172.21.15.3:0/1231836713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0370103170 0x7f0370109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 --2- 172.21.15.3:0/1231836713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701027b0 0x7f0370102c30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 --2- 172.21.15.3:0/1231836713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03701015c0 0x7f03701019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 -- 172.21.15.3:0/1231836713 >> 172.21.15.3:0/1231836713 conn(0x7f03700fcb30 msgr2=0x7f03700fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 -- 172.21.15.3:0/1231836713 shutdown_connections 2024-03-21T08:27:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.436+0000 7f036f200640 1 -- 172.21.15.3:0/1231836713 wait complete. 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.437+0000 7f036f200640 1 Processor -- start 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.437+0000 7f036f200640 1 -- start start 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701015c0 0x7f0370075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701027b0 0x7f0370073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0370103170 0x7f0370073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037010d1a0 con 0x7f03701015c0 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037010d020 con 0x7f0370103170 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037010d320 con 0x7f03701027b0 2024-03-21T08:27:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701015c0 0x7f0370075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701015c0 0x7f0370075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60368/0 (socket says 172.21.15.3:60368) 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036de00640 1 -- 172.21.15.3:0/1495356974 learned_addr learned my addr 172.21.15.3:0/1495356974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f0367e00640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701027b0 0x7f0370073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.438+0000 7f036e800640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0370103170 0x7f0370073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.439+0000 7f036de00640 1 -- 172.21.15.3:0/1495356974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701027b0 msgr2=0x7f0370073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.439+0000 7f036de00640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701027b0 0x7f0370073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.439+0000 7f036de00640 1 -- 172.21.15.3:0/1495356974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0370103170 msgr2=0x7f0370073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.439+0000 7f036de00640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0370103170 0x7f0370073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.439+0000 7f036de00640 1 -- 172.21.15.3:0/1495356974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0370074860 con 0x7f03701015c0 2024-03-21T08:27:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.439+0000 7f036de00640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701015c0 0x7f0370075450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f036000d770 tx=0x7f036000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.440+0000 7f0366a00640 1 -- 172.21.15.3:0/1495356974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360017070 con 0x7f03701015c0 2024-03-21T08:27:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.440+0000 7f0366a00640 1 -- 172.21.15.3:0/1495356974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0360009c30 con 0x7f03701015c0 2024-03-21T08:27:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.440+0000 7f0366a00640 1 -- 172.21.15.3:0/1495356974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360003d00 con 0x7f03701015c0 2024-03-21T08:27:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.440+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03701a1930 con 0x7f03701015c0 2024-03-21T08:27:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.440+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03701a1e80 con 0x7f03701015c0 2024-03-21T08:27:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.441+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03701088e0 con 0x7f03701015c0 2024-03-21T08:27:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.443+0000 7f0366a00640 1 -- 172.21.15.3:0/1495356974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f0360003ea0 con 0x7f03701015c0 2024-03-21T08:27:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.444+0000 7f0366a00640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03540a2f90 0x7f03540a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.444+0000 7f0366a00640 1 -- 172.21.15.3:0/1495356974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f03600cebe0 con 0x7f03701015c0 2024-03-21T08:27:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.445+0000 7f0367e00640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03540a2f90 0x7f03540a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:56.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.445+0000 7f0367e00640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03540a2f90 0x7f03540a5450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0358000c00 tx=0x7f035802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:56.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.449+0000 7f0366a00640 1 -- 172.21.15.3:0/1495356974 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0360009de0 con 0x7f03701015c0 2024-03-21T08:27:56.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.696+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f037002d320 con 0x7f03701015c0 2024-03-21T08:27:56.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.697+0000 7f0366a00640 1 -- 172.21.15.3:0/1495356974 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f036009b7c0 con 0x7f03701015c0 2024-03-21T08:27:56.699 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T08:27:56.699 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03540a2f90 msgr2=0x7f03540a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03540a2f90 0x7f03540a5450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0358000c00 tx=0x7f035802c7e0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701015c0 msgr2=0x7f0370075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701015c0 0x7f0370075450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f036000d770 tx=0x7f036000dc30 comp rx=0 tx=0).stop 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 shutdown_connections 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0370103170 0x7f0370073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03540a2f90 0x7f03540a5450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701027b0 0x7f0370073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 --2- 172.21.15.3:0/1495356974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03701015c0 0x7f0370075450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.699+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 >> 172.21.15.3:0/1495356974 conn(0x7f03700fcb30 msgr2=0x7f0370105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.700+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 shutdown_connections 2024-03-21T08:27:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:56.700+0000 7f036f200640 1 -- 172.21.15.3:0/1495356974 wait complete. 2024-03-21T08:27:57.518 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:57.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:57.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:57.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:57.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:57 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1495356974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:57 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1495356974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:27:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:27:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:27:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:27:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:57 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1495356974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:27:58.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:58 smithi003 ceph-mon[31675]: Deploying daemon mds.d on smithi003 2024-03-21T08:27:58.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:58 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.3:6834/3089491924,v1:172.21.15.3:6835/3089491924] up:boot 2024-03-21T08:27:58.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:58 smithi003 ceph-mon[31675]: fsmap 3 up:standby 2024-03-21T08:27:58.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:58 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:27:58.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:58 smithi084 ceph-mon[35477]: Deploying daemon mds.d on smithi003 2024-03-21T08:27:58.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:58 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.3:6834/3089491924,v1:172.21.15.3:6835/3089491924] up:boot 2024-03-21T08:27:58.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:58 smithi084 ceph-mon[35477]: fsmap 3 up:standby 2024-03-21T08:27:58.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:58 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:27:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:58 smithi171 ceph-mon[35388]: Deploying daemon mds.d on smithi003 2024-03-21T08:27:58.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:58 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.3:6834/3089491924,v1:172.21.15.3:6835/3089491924] up:boot 2024-03-21T08:27:58.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:58 smithi171 ceph-mon[35388]: fsmap 3 up:standby 2024-03-21T08:27:58.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:58 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:27:58.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:27:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.047+0000 7fe300400640 1 -- 172.21.15.3:0/1593918332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f81086a0 msgr2=0x7fe2f810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.047+0000 7fe300400640 1 --2- 172.21.15.3:0/1593918332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f81086a0 0x7fe2f810aad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009900 tx=0x7fe2e402bcd0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.048+0000 7fe300400640 1 -- 172.21.15.3:0/1593918332 shutdown_connections 2024-03-21T08:27:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.048+0000 7fe300400640 1 --2- 172.21.15.3:0/1593918332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f810b490 0x7fe2f8114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.048+0000 7fe300400640 1 --2- 172.21.15.3:0/1593918332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f810b010 0x7fe2f8111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.048+0000 7fe300400640 1 --2- 172.21.15.3:0/1593918332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f81086a0 0x7fe2f810aad0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.048+0000 7fe300400640 1 -- 172.21.15.3:0/1593918332 >> 172.21.15.3:0/1593918332 conn(0x7fe2f806e8f0 msgr2=0x7fe2f8070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.049+0000 7fe300400640 1 -- 172.21.15.3:0/1593918332 shutdown_connections 2024-03-21T08:27:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.049+0000 7fe300400640 1 -- 172.21.15.3:0/1593918332 wait complete. 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.050+0000 7fe300400640 1 Processor -- start 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.051+0000 7fe300400640 1 -- start start 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.051+0000 7fe300400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f810b010 0x7fe2f81a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.051+0000 7fe300400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f810b490 0x7fe2f81a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.051+0000 7fe300400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f81a7af0 0x7fe2f81a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.051+0000 7fe300400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f8117ca0 con 0x7fe2f810b490 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.051+0000 7fe300400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f8117b20 con 0x7fe2f81a7af0 2024-03-21T08:27:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.051+0000 7fe300400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f8117e20 con 0x7fe2f810b010 2024-03-21T08:27:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f810b490 0x7fe2f81a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fe400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f81a7af0 0x7fe2f81a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f810b490 0x7fe2f81a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60388/0 (socket says 172.21.15.3:60388) 2024-03-21T08:27:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 -- 172.21.15.3:0/627536172 learned_addr learned my addr 172.21.15.3:0/627536172 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:27:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fda00640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f810b010 0x7fe2f81a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 -- 172.21.15.3:0/627536172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f810b010 msgr2=0x7fe2f81a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f810b010 0x7fe2f81a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 -- 172.21.15.3:0/627536172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f81a7af0 msgr2=0x7fe2f81a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f81a7af0 0x7fe2f81a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.052+0000 7fe2fd000640 1 -- 172.21.15.3:0/627536172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f81aa550 con 0x7fe2f810b490 2024-03-21T08:27:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.053+0000 7fe2fd000640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f810b490 0x7fe2f81a2a90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000e9c0 tx=0x7fe2f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.053+0000 7fe2f6000640 1 -- 172.21.15.3:0/627536172 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000cd80 con 0x7fe2f810b490 2024-03-21T08:27:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.053+0000 7fe2f6000640 1 -- 172.21.15.3:0/627536172 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2f00035f0 con 0x7fe2f810b490 2024-03-21T08:27:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.053+0000 7fe2f6000640 1 -- 172.21.15.3:0/627536172 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0010690 con 0x7fe2f810b490 2024-03-21T08:27:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.053+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f81aa840 con 0x7fe2f810b490 2024-03-21T08:27:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.053+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f81aadc0 con 0x7fe2f810b490 2024-03-21T08:27:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.055+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f81134f0 con 0x7fe2f810b490 2024-03-21T08:27:59.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.057+0000 7fe2f6000640 1 -- 172.21.15.3:0/627536172 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fe2f00037a0 con 0x7fe2f810b490 2024-03-21T08:27:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.058+0000 7fe2f6000640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe2dc0a2fa0 0x7fe2dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:27:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.058+0000 7fe2f6000640 1 -- 172.21.15.3:0/627536172 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2f00cf2e0 con 0x7fe2f810b490 2024-03-21T08:27:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.059+0000 7fe2fda00640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe2dc0a2fa0 0x7fe2dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:27:59.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.059+0000 7fe2fda00640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe2dc0a2fa0 0x7fe2dc0a5460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4036040 tx=0x7fe2e402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:27:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.066+0000 7fe2f6000640 1 -- 172.21.15.3:0/627536172 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2f009c510 con 0x7fe2f810b490 2024-03-21T08:27:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.312+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2f802d070 con 0x7fe2f810b490 2024-03-21T08:27:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.313+0000 7fe2f6000640 1 -- 172.21.15.3:0/627536172 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7fe2f009be10 con 0x7fe2f810b490 2024-03-21T08:27:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:27:59.315 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T08:27:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 4 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe2dc0a2fa0 msgr2=0x7fe2dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe2dc0a2fa0 0x7fe2dc0a5460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4036040 tx=0x7fe2e402c880 comp rx=0 tx=0).stop 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f810b490 msgr2=0x7fe2f81a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f810b490 0x7fe2f81a2a90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000e9c0 tx=0x7fe2f000ee80 comp rx=0 tx=0).stop 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 shutdown_connections 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f81a7af0 0x7fe2f81a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f810b490 0x7fe2f81a2a90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe2dc0a2fa0 0x7fe2dc0a5460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 --2- 172.21.15.3:0/627536172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f810b010 0x7fe2f81a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:27:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 >> 172.21.15.3:0/627536172 conn(0x7fe2f806e8f0 msgr2=0x7fe2f806f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:27:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 shutdown_connections 2024-03-21T08:27:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:27:59.316+0000 7fe300400640 1 -- 172.21.15.3:0/627536172 wait complete. 2024-03-21T08:27:59.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:27:59 smithi084 ceph-mon[35477]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:59.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:27:59 smithi003 ceph-mon[31675]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:59.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:27:59 smithi171 ceph-mon[35388]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:27:59.911 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:27:59 smithi003 podman[72251]: 2024-03-21 08:27:59.631249404 +0000 UTC m=+0.132444542 container create 217b9b8b110b09647c7dbfd3a073c0b8a1a9687964ac030672b161d580bce282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:28:00.177 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:27:59 smithi003 podman[72251]: 2024-03-21 08:27:59.910039164 +0000 UTC m=+0.411234309 container init 217b9b8b110b09647c7dbfd3a073c0b8a1a9687964ac030672b161d580bce282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T08:28:00.177 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:27:59 smithi003 podman[72251]: 2024-03-21 08:27:59.921782553 +0000 UTC m=+0.422977671 container start 217b9b8b110b09647c7dbfd3a073c0b8a1a9687964ac030672b161d580bce282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T08:28:00.177 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:27:59 smithi003 ceph-mds[72270]: -- 172.21.15.3:0/3659973952 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d319b6fa40 con 0x55d319b42c00 2024-03-21T08:28:00.177 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:27:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d[72266]: starting mds.d at 2024-03-21T08:28:00.178 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:28:00 smithi003 bash[72251]: 217b9b8b110b09647c7dbfd3a073c0b8a1a9687964ac030672b161d580bce282 2024-03-21T08:28:00.178 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:28:00 smithi003 systemd[1]: Started Ceph mds.d for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:00.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:00 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/627536172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:00 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:00 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:00 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:00 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:00 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:00.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:00 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/627536172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:00 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:00 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:00 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:00 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:00 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:00 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/627536172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:00.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:00 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:01.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:01.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:01 smithi084 ceph-mon[35477]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:01.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:01 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:01.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:01 smithi084 ceph-mon[35477]: Deploying daemon mds.e on smithi084 2024-03-21T08:28:01.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:01 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.3:6836/4111852519,v1:172.21.15.3:6837/4111852519] up:boot 2024-03-21T08:28:01.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:01 smithi084 ceph-mon[35477]: fsmap 4 up:standby 2024-03-21T08:28:01.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:01 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:28:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 -- 172.21.15.3:0/838664247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc0ffe20 msgr2=0x7fc1cc102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 --2- 172.21.15.3:0/838664247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc0ffe20 0x7fc1cc102250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009900 tx=0x7fc1bc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 -- 172.21.15.3:0/838664247 shutdown_connections 2024-03-21T08:28:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 --2- 172.21.15.3:0/838664247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1cc105140 0x7fc1cc1055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 --2- 172.21.15.3:0/838664247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1cc102790 0x7fc1cc104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 --2- 172.21.15.3:0/838664247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc0ffe20 0x7fc1cc102250 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 -- 172.21.15.3:0/838664247 >> 172.21.15.3:0/838664247 conn(0x7fc1cc0fba20 msgr2=0x7fc1cc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 -- 172.21.15.3:0/838664247 shutdown_connections 2024-03-21T08:28:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.576+0000 7fc1d1600640 1 -- 172.21.15.3:0/838664247 wait complete. 2024-03-21T08:28:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.577+0000 7fc1d1600640 1 Processor -- start 2024-03-21T08:28:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.577+0000 7fc1d1600640 1 -- start start 2024-03-21T08:28:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1d1600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1cc0ffe20 0x7fc1cc19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1d1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc102790 0x7fc1cc19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1d1600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1cc105140 0x7fc1cc1a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1d1600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc113720 con 0x7fc1cc102790 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1d1600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1135a0 con 0x7fc1cc105140 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1d1600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1138a0 con 0x7fc1cc0ffe20 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc102790 0x7fc1cc19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc102790 0x7fc1cc19e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55432/0 (socket says 172.21.15.3:55432) 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1cb400640 1 -- 172.21.15.3:0/646551416 learned_addr learned my addr 172.21.15.3:0/646551416 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1d0c00640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1cc105140 0x7fc1cc1a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.578+0000 7fc1cbe00640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1cc0ffe20 0x7fc1cc19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1cb400640 1 -- 172.21.15.3:0/646551416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1cc0ffe20 msgr2=0x7fc1cc19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1cb400640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1cc0ffe20 0x7fc1cc19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1cb400640 1 -- 172.21.15.3:0/646551416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1cc105140 msgr2=0x7fc1cc1a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1cb400640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1cc105140 0x7fc1cc1a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1cb400640 1 -- 172.21.15.3:0/646551416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc1a59e0 con 0x7fc1cc102790 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1d0c00640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1cc105140 0x7fc1cc1a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1cbe00640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1cc0ffe20 0x7fc1cc19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1cb400640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc102790 0x7fc1cc19e300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c970 tx=0x7fc1b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1c8c00640 1 -- 172.21.15.3:0/646551416 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8007bb0 con 0x7fc1cc102790 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc1a5cd0 con 0x7fc1cc102790 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1c8c00640 1 -- 172.21.15.3:0/646551416 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1b8007d50 con 0x7fc1cc102790 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.579+0000 7fc1c8c00640 1 -- 172.21.15.3:0/646551416 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800f450 con 0x7fc1cc102790 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.580+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc1a61d0 con 0x7fc1cc102790 2024-03-21T08:28:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.581+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc0fd510 con 0x7fc1cc102790 2024-03-21T08:28:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.583+0000 7fc1c8c00640 1 -- 172.21.15.3:0/646551416 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fc1b8004f20 con 0x7fc1cc102790 2024-03-21T08:28:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.585+0000 7fc1c8c00640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1b00a2f10 0x7fc1b00a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.585+0000 7fc1c8c00640 1 -- 172.21.15.3:0/646551416 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1b80cedf0 con 0x7fc1cc102790 2024-03-21T08:28:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.585+0000 7fc1cbe00640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1b00a2f10 0x7fc1b00a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.586+0000 7fc1cbe00640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1b00a2f10 0x7fc1b00a53d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc006010 tx=0x7fc1bc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.589+0000 7fc1c8c00640 1 -- 172.21.15.3:0/646551416 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc1b8014030 con 0x7fc1cc102790 2024-03-21T08:28:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:01 smithi171 ceph-mon[35388]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:01 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:01 smithi171 ceph-mon[35388]: Deploying daemon mds.e on smithi084 2024-03-21T08:28:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:01 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.3:6836/4111852519,v1:172.21.15.3:6837/4111852519] up:boot 2024-03-21T08:28:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:01 smithi171 ceph-mon[35388]: fsmap 4 up:standby 2024-03-21T08:28:01.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:01 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:28:01.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:01 smithi003 ceph-mon[31675]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:01.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:01 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:01.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:01 smithi003 ceph-mon[31675]: Deploying daemon mds.e on smithi084 2024-03-21T08:28:01.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:01 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.3:6836/4111852519,v1:172.21.15.3:6837/4111852519] up:boot 2024-03-21T08:28:01.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:01 smithi003 ceph-mon[31675]: fsmap 4 up:standby 2024-03-21T08:28:01.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:01 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:28:01.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.833+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1cc02d070 con 0x7fc1cc102790 2024-03-21T08:28:01.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.834+0000 7fc1c8c00640 1 -- 172.21.15.3:0/646551416 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3695 (secure 0 0 0) 0x7fc1b809c020 con 0x7fc1cc102790 2024-03-21T08:28:01.836 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:01.836 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T08:28:01.837 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 5 2024-03-21T08:28:01.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1b00a2f10 msgr2=0x7fc1b00a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1b00a2f10 0x7fc1b00a53d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc006010 tx=0x7fc1bc02c7e0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc102790 msgr2=0x7fc1cc19e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc102790 0x7fc1cc19e300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c970 tx=0x7fc1b800ce30 comp rx=0 tx=0).stop 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 shutdown_connections 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1cc105140 0x7fc1cc1a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1cc102790 0x7fc1cc19e300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1b00a2f10 0x7fc1b00a53d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 --2- 172.21.15.3:0/646551416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1cc0ffe20 0x7fc1cc19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 >> 172.21.15.3:0/646551416 conn(0x7fc1cc0fba20 msgr2=0x7fc1cc103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 shutdown_connections 2024-03-21T08:28:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:01.837+0000 7fc1d1600640 1 -- 172.21.15.3:0/646551416 wait complete. 2024-03-21T08:28:02.534 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:02 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/646551416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:02.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:02 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/646551416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:02.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:02 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/646551416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:03.607 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:03 smithi084 ceph-mon[35477]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:03.607 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 systemd[1]: Starting Ceph mds.e for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:28:03.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:03 smithi171 ceph-mon[35388]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:03.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:03 smithi003 ceph-mon[31675]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:03.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:03.876 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 podman[64811]: 2024-03-21 08:28:03.514229762 +0000 UTC m=+0.041636290 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:28:03.876 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 podman[64811]: 2024-03-21 08:28:03.629618453 +0000 UTC m=+0.157024995 container create 88eb30fefa9eb0385edd9a1d1680bacf0330683af1f0e142c086c9839201769e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T08:28:04.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 -- 172.21.15.3:0/1049775749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 msgr2=0x7ff93410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:04.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 --2- 172.21.15.3:0/1049775749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff93410bd20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009900 tx=0x7ff92c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 -- 172.21.15.3:0/1049775749 shutdown_connections 2024-03-21T08:28:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 --2- 172.21.15.3:0/1049775749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff93410bd20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 --2- 172.21.15.3:0/1049775749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff934104ad0 0x7ff934104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 --2- 172.21.15.3:0/1049775749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9341038e0 0x7ff934103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 -- 172.21.15.3:0/1049775749 >> 172.21.15.3:0/1049775749 conn(0x7ff9340fee90 msgr2=0x7ff9341012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 -- 172.21.15.3:0/1049775749 shutdown_connections 2024-03-21T08:28:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.092+0000 7ff939200640 1 -- 172.21.15.3:0/1049775749 wait complete. 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.093+0000 7ff939200640 1 Processor -- start 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.093+0000 7ff939200640 1 -- start start 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff939200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9341038e0 0x7ff934199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff939200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff934104ad0 0x7ff934199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff939200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff9341a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff939200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93410f460 con 0x7ff934105490 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff939200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93410f2e0 con 0x7ff934104ad0 2024-03-21T08:28:04.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff939200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93410f5e0 con 0x7ff9341038e0 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff932800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff9341a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff932800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff9341a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55450/0 (socket says 172.21.15.3:55450) 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff932800640 1 -- 172.21.15.3:0/3220418261 learned_addr learned my addr 172.21.15.3:0/3220418261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff929400640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff934104ad0 0x7ff934199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.094+0000 7ff931e00640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9341038e0 0x7ff934199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff932800640 1 -- 172.21.15.3:0/3220418261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9341038e0 msgr2=0x7ff934199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff932800640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9341038e0 0x7ff934199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff932800640 1 -- 172.21.15.3:0/3220418261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff934104ad0 msgr2=0x7ff934199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff932800640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff934104ad0 0x7ff934199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff932800640 1 -- 172.21.15.3:0/3220418261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9341a16c0 con 0x7ff934105490 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff932800640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff9341a1050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff92c036040 tx=0x7ff92c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff92b400640 1 -- 172.21.15.3:0/3220418261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c043020 con 0x7ff934105490 2024-03-21T08:28:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff92b400640 1 -- 172.21.15.3:0/3220418261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff92c004490 con 0x7ff934105490 2024-03-21T08:28:04.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9341a1950 con 0x7ff934105490 2024-03-21T08:28:04.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff92b400640 1 -- 172.21.15.3:0/3220418261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c003d10 con 0x7ff934105490 2024-03-21T08:28:04.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.095+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9341a1ef0 con 0x7ff934105490 2024-03-21T08:28:04.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.097+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93410ac00 con 0x7ff934105490 2024-03-21T08:28:04.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.099+0000 7ff92b400640 1 -- 172.21.15.3:0/3220418261 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7ff92c046020 con 0x7ff934105490 2024-03-21T08:28:04.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.100+0000 7ff92b400640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff9180a2fa0 0x7ff9180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:04.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.100+0000 7ff92b400640 1 -- 172.21.15.3:0/3220418261 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff92c0f13c0 con 0x7ff934105490 2024-03-21T08:28:04.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.100+0000 7ff931e00640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff9180a2fa0 0x7ff9180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:04.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.101+0000 7ff931e00640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff9180a2fa0 0x7ff9180a5460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff934104930 tx=0x7ff9240098f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:04.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.104+0000 7ff92b400640 1 -- 172.21.15.3:0/3220418261 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff92c04d020 con 0x7ff934105490 2024-03-21T08:28:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.356+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9340622a0 con 0x7ff934105490 2024-03-21T08:28:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.357+0000 7ff92b400640 1 -- 172.21.15.3:0/3220418261 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4486 (secure 0 0 0) 0x7ff92c0be5f0 con 0x7ff934105490 2024-03-21T08:28:04.359 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:04.359 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T08:28:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2024-03-21T08:28:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff9180a2fa0 msgr2=0x7ff9180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff9180a2fa0 0x7ff9180a5460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff934104930 tx=0x7ff9240098f0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 msgr2=0x7ff9341a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff9341a1050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff92c036040 tx=0x7ff92c004270 comp rx=0 tx=0).stop 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 shutdown_connections 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff934105490 0x7ff9341a1050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff934104ad0 0x7ff934199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff9180a2fa0 0x7ff9180a5460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 --2- 172.21.15.3:0/3220418261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9341038e0 0x7ff934199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.360+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 >> 172.21.15.3:0/3220418261 conn(0x7ff9340fee90 msgr2=0x7ff934107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.361+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 shutdown_connections 2024-03-21T08:28:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:04.361+0000 7ff939200640 1 -- 172.21.15.3:0/3220418261 wait complete. 2024-03-21T08:28:04.376 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 podman[64811]: 2024-03-21 08:28:03.900210661 +0000 UTC m=+0.427617213 container init 88eb30fefa9eb0385edd9a1d1680bacf0330683af1f0e142c086c9839201769e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T08:28:04.377 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 podman[64811]: 2024-03-21 08:28:03.909615267 +0000 UTC m=+0.437021816 container start 88eb30fefa9eb0385edd9a1d1680bacf0330683af1f0e142c086c9839201769e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T08:28:04.377 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 ceph-mds[64832]: -- 172.21.15.84:0/366905310 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5555ce18e4e0 con 0x5555ce165000 2024-03-21T08:28:04.377 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 ceph-mds[64832]: -- 172.21.15.84:0/366905310 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555ce191a40 con 0x5555ce165000 2024-03-21T08:28:04.377 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:03 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e[64828]: starting mds.e at 2024-03-21T08:28:04.377 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:04 smithi084 bash[64811]: 88eb30fefa9eb0385edd9a1d1680bacf0330683af1f0e142c086c9839201769e 2024-03-21T08:28:04.377 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:28:04 smithi084 systemd[1]: Started Ceph mds.e for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:05.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:05.394 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.394 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.394 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.394 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:05.394 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:05.394 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:05.395 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: Deploying daemon mds.f on smithi171 2024-03-21T08:28:05.395 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] up:boot 2024-03-21T08:28:05.395 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: fsmap 5 up:standby 2024-03-21T08:28:05.395 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:28:05.395 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:05 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3220418261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:05.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:05.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:05.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:05.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:05.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: Deploying daemon mds.f on smithi171 2024-03-21T08:28:05.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] up:boot 2024-03-21T08:28:05.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: fsmap 5 up:standby 2024-03-21T08:28:05.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:28:05.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:05 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3220418261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:05.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:05.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: Deploying daemon mds.f on smithi171 2024-03-21T08:28:05.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] up:boot 2024-03-21T08:28:05.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: fsmap 5 up:standby 2024-03-21T08:28:05.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:28:05.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:05 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3220418261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:06.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.613+0000 7f0809800640 1 -- 172.21.15.3:0/3038916044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804105510 msgr2=0x7f080410bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:06.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.613+0000 7f0809800640 1 --2- 172.21.15.3:0/3038916044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804105510 0x7f080410bda0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009900 tx=0x7f07fc02bc40 comp rx=0 tx=0).stop 2024-03-21T08:28:06.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.614+0000 7f0809800640 1 -- 172.21.15.3:0/3038916044 shutdown_connections 2024-03-21T08:28:06.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.614+0000 7f0809800640 1 --2- 172.21.15.3:0/3038916044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804105510 0x7f080410bda0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.614+0000 7f0809800640 1 --2- 172.21.15.3:0/3038916044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0804104b50 0x7f0804104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.614+0000 7f0809800640 1 --2- 172.21.15.3:0/3038916044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0804103960 0x7f0804103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.614+0000 7f0809800640 1 -- 172.21.15.3:0/3038916044 >> 172.21.15.3:0/3038916044 conn(0x7f08040feeb0 msgr2=0x7f0804101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.614+0000 7f0809800640 1 -- 172.21.15.3:0/3038916044 shutdown_connections 2024-03-21T08:28:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.614+0000 7f0809800640 1 -- 172.21.15.3:0/3038916044 wait complete. 2024-03-21T08:28:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 Processor -- start 2024-03-21T08:28:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 -- start start 2024-03-21T08:28:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804103960 0x7f0804199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0804104b50 0x7f0804199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0804105510 0x7f08041a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080410f420 con 0x7f0804103960 2024-03-21T08:28:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080410f2a0 con 0x7f0804105510 2024-03-21T08:28:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.615+0000 7f0809800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080410f5a0 con 0x7f0804104b50 2024-03-21T08:28:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804103960 0x7f0804199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804103960 0x7f0804199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55458/0 (socket says 172.21.15.3:55458) 2024-03-21T08:28:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 -- 172.21.15.3:0/2086927326 learned_addr learned my addr 172.21.15.3:0/2086927326 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f07fa000640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0804104b50 0x7f0804199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 -- 172.21.15.3:0/2086927326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0804104b50 msgr2=0x7f0804199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0803400640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0804105510 0x7f08041a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0804104b50 0x7f0804199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 -- 172.21.15.3:0/2086927326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0804105510 msgr2=0x7f08041a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0804105510 0x7f08041a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 -- 172.21.15.3:0/2086927326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08041a1620 con 0x7f0804103960 2024-03-21T08:28:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.616+0000 7f0802a00640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804103960 0x7f0804199a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07f400ccb0 tx=0x7f07f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.617+0000 7f07fbe00640 1 -- 172.21.15.3:0/2086927326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4004b20 con 0x7f0804103960 2024-03-21T08:28:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.617+0000 7f07fbe00640 1 -- 172.21.15.3:0/2086927326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f07f4003e40 con 0x7f0804103960 2024-03-21T08:28:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.617+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08041a1910 con 0x7f0804103960 2024-03-21T08:28:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.617+0000 7f07fbe00640 1 -- 172.21.15.3:0/2086927326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f40047e0 con 0x7f0804103960 2024-03-21T08:28:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.617+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08041a1e60 con 0x7f0804103960 2024-03-21T08:28:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.618+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080410ac80 con 0x7f0804103960 2024-03-21T08:28:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.620+0000 7f07fbe00640 1 -- 172.21.15.3:0/2086927326 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f07f4003990 con 0x7f0804103960 2024-03-21T08:28:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.622+0000 7f07fbe00640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07e80a2fa0 0x7f07e80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.622+0000 7f07fbe00640 1 -- 172.21.15.3:0/2086927326 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f07f40cf0b0 con 0x7f0804103960 2024-03-21T08:28:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.623+0000 7f07fa000640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07e80a2fa0 0x7f07e80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.623+0000 7f07fa000640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07e80a2fa0 0x7f07e80a5460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f080419af10 tx=0x7f07ec005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.627+0000 7f07fbe00640 1 -- 172.21.15.3:0/2086927326 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f07f4014030 con 0x7f0804103960 2024-03-21T08:28:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.874+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f080402d070 con 0x7f0804103960 2024-03-21T08:28:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.875+0000 7f07fbe00640 1 -- 172.21.15.3:0/2086927326 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4486 (secure 0 0 0) 0x7f07f409c380 con 0x7f0804103960 2024-03-21T08:28:06.877 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:06.878 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T08:28:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2024-03-21T08:28:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07e80a2fa0 msgr2=0x7f07e80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07e80a2fa0 0x7f07e80a5460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f080419af10 tx=0x7f07ec005cc0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804103960 msgr2=0x7f0804199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804103960 0x7f0804199a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07f400ccb0 tx=0x7f07f400a620 comp rx=0 tx=0).stop 2024-03-21T08:28:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 shutdown_connections 2024-03-21T08:28:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0804105510 0x7f08041a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07e80a2fa0 0x7f07e80a5460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0804104b50 0x7f0804199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 --2- 172.21.15.3:0/2086927326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0804103960 0x7f0804199a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 >> 172.21.15.3:0/2086927326 conn(0x7f08040feeb0 msgr2=0x7f0804107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.877+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 shutdown_connections 2024-03-21T08:28:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:06.878+0000 7f0809800640 1 -- 172.21.15.3:0/2086927326 wait complete. 2024-03-21T08:28:07.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:07 smithi003 ceph-mon[31675]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:07.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:07 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2086927326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:07.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:07 smithi171 ceph-mon[35388]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:07.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:07 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2086927326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:07.586 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 systemd[1]: Starting Ceph mds.f for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:28:07.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:07 smithi084 ceph-mon[35477]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:07.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:07 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2086927326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:07.842 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 podman[66262]: 2024-03-21 08:28:07.490230524 +0000 UTC m=+0.033609466 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:28:07.842 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 podman[66262]: 2024-03-21 08:28:07.609968514 +0000 UTC m=+0.153347463 container create c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T08:28:07.842 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 podman[66262]: 2024-03-21 08:28:07.841063156 +0000 UTC m=+0.384442108 container init c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T08:28:08.134 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 podman[66262]: 2024-03-21 08:28:07.852639858 +0000 UTC m=+0.396018793 container start c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T08:28:08.134 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 ceph-mds[66284]: -- 172.21.15.171:0/2048737431 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a3e856ba40 con 0x55a3e853ec00 2024-03-21T08:28:08.134 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f[66279]: starting mds.f at 2024-03-21T08:28:08.134 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 bash[66262]: c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db 2024-03-21T08:28:08.134 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:28:07 smithi171 systemd[1]: Started Ceph mds.f for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:08.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.137+0000 7f05b3200640 1 -- 172.21.15.3:0/2054563848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 msgr2=0x7f05b4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.137+0000 7f05b3200640 1 --2- 172.21.15.3:0/2054563848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4103be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009900 tx=0x7f05a402bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.138+0000 7f05b3200640 1 -- 172.21.15.3:0/2054563848 shutdown_connections 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.138+0000 7f05b3200640 1 --2- 172.21.15.3:0/2054563848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05b4105370 0x7f05b410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.138+0000 7f05b3200640 1 --2- 172.21.15.3:0/2054563848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05b41049b0 0x7f05b4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.138+0000 7f05b3200640 1 --2- 172.21.15.3:0/2054563848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4103be0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.138+0000 7f05b3200640 1 -- 172.21.15.3:0/2054563848 >> 172.21.15.3:0/2054563848 conn(0x7f05b40fed50 msgr2=0x7f05b4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.138+0000 7f05b3200640 1 -- 172.21.15.3:0/2054563848 shutdown_connections 2024-03-21T08:28:09.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.138+0000 7f05b3200640 1 -- 172.21.15.3:0/2054563848 wait complete. 2024-03-21T08:28:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 Processor -- start 2024-03-21T08:28:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 -- start start 2024-03-21T08:28:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05b41049b0 0x7f05b4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05b4105370 0x7f05b41a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b410f330 con 0x7f05b41037c0 2024-03-21T08:28:09.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b410f1b0 con 0x7f05b4105370 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.139+0000 7f05b3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b410f4b0 con 0x7f05b41049b0 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55468/0 (socket says 172.21.15.3:55468) 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 -- 172.21.15.3:0/3275849611 learned_addr learned my addr 172.21.15.3:0/3275849611 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b2800640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05b4105370 0x7f05b41a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05abe00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05b41049b0 0x7f05b4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 -- 172.21.15.3:0/3275849611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05b41049b0 msgr2=0x7f05b4199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05b41049b0 0x7f05b4199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 -- 172.21.15.3:0/3275849611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05b4105370 msgr2=0x7f05b41a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05b4105370 0x7f05b41a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 -- 172.21.15.3:0/3275849611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b41a1620 con 0x7f05b41037c0 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05abe00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05b41049b0 0x7f05b4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b2800640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05b4105370 0x7f05b41a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:09.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.140+0000 7f05b1e00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4199a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f05a4000c00 tx=0x7f05a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:09.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.141+0000 7f05aaa00640 1 -- 172.21.15.3:0/3275849611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4043020 con 0x7f05b41037c0 2024-03-21T08:28:09.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.141+0000 7f05aaa00640 1 -- 172.21.15.3:0/3275849611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f05a40044c0 con 0x7f05b41037c0 2024-03-21T08:28:09.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.141+0000 7f05aaa00640 1 -- 172.21.15.3:0/3275849611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4003c90 con 0x7f05b41037c0 2024-03-21T08:28:09.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.141+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b41a18b0 con 0x7f05b41037c0 2024-03-21T08:28:09.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.141+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b41a1e50 con 0x7f05b41037c0 2024-03-21T08:28:09.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.142+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b410ab80 con 0x7f05b41037c0 2024-03-21T08:28:09.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.146+0000 7f05aaa00640 1 -- 172.21.15.3:0/3275849611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f05a4046020 con 0x7f05b41037c0 2024-03-21T08:28:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.147+0000 7f05aaa00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05980a2f90 0x7f05980a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.147+0000 7f05aaa00640 1 -- 172.21.15.3:0/3275849611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05a40f1690 con 0x7f05b41037c0 2024-03-21T08:28:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.147+0000 7f05abe00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05980a2f90 0x7f05980a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:09.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.148+0000 7f05abe00640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05980a2f90 0x7f05980a5450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05b419af10 tx=0x7f059c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:09.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.150+0000 7f05aaa00640 1 -- 172.21.15.3:0/3275849611 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f05a404d020 con 0x7f05b41037c0 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: Deploying daemon mds.c on smithi171 2024-03-21T08:28:09.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.171:6834/1859190507,v1:172.21.15.171:6835/1859190507] up:boot 2024-03-21T08:28:09.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: fsmap 6 up:standby 2024-03-21T08:28:09.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:09 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: Deploying daemon mds.c on smithi171 2024-03-21T08:28:09.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.171:6834/1859190507,v1:172.21.15.171:6835/1859190507] up:boot 2024-03-21T08:28:09.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: fsmap 6 up:standby 2024-03-21T08:28:09.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:09 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:28:09.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.392+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05b402d070 con 0x7f05b41037c0 2024-03-21T08:28:09.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.394+0000 7f05aaa00640 1 -- 172.21.15.3:0/3275849611 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5280 (secure 0 0 0) 0x7f05a40be8c0 con 0x7f05b41037c0 2024-03-21T08:28:09.396 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:09.396 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T08:28:09.397 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-03-21T08:28:09.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05980a2f90 msgr2=0x7f05980a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:09.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05980a2f90 0x7f05980a5450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05b419af10 tx=0x7f059c005f90 comp rx=0 tx=0).stop 2024-03-21T08:28:09.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 msgr2=0x7f05b4199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:09.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4199a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f05a4000c00 tx=0x7f05a40040c0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 shutdown_connections 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05b4105370 0x7f05b41a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05980a2f90 0x7f05980a5450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05b41049b0 0x7f05b4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 --2- 172.21.15.3:0/3275849611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05b41037c0 0x7f05b4199a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.397+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 >> 172.21.15.3:0/3275849611 conn(0x7f05b40fed50 msgr2=0x7f05b4107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.398+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 shutdown_connections 2024-03-21T08:28:09.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:09.398+0000 7f05b3200640 1 -- 172.21.15.3:0/3275849611 wait complete. 2024-03-21T08:28:09.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:09.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:09.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:09.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:09.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:09.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: Deploying daemon mds.c on smithi171 2024-03-21T08:28:09.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.171:6834/1859190507,v1:172.21.15.171:6835/1859190507] up:boot 2024-03-21T08:28:09.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: fsmap 6 up:standby 2024-03-21T08:28:09.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:09 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:28:10.145 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:10 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3275849611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:10.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:10 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3275849611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:10.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:10 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3275849611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:11.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:11 smithi084 ceph-mon[35477]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:11.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:11 smithi171 ceph-mon[35388]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:11.384 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 systemd[1]: Starting Ceph mds.c for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:28:11.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:11.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:11 smithi003 ceph-mon[31675]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 -- 172.21.15.3:0/3877431015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 msgr2=0x7f8844109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 --2- 172.21.15.3:0/3877431015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f8844109230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f882c009ca0 tx=0x7f882c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 -- 172.21.15.3:0/3877431015 shutdown_connections 2024-03-21T08:28:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 --2- 172.21.15.3:0/3877431015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8844109770 0x7f8844110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 --2- 172.21.15.3:0/3877431015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f8844109230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 --2- 172.21.15.3:0/3877431015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8844107bc0 0x7f8844107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 -- 172.21.15.3:0/3877431015 >> 172.21.15.3:0/3877431015 conn(0x7f8844076b60 msgr2=0x7f8844078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 -- 172.21.15.3:0/3877431015 shutdown_connections 2024-03-21T08:28:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.650+0000 7f8849c00640 1 -- 172.21.15.3:0/3877431015 wait complete. 2024-03-21T08:28:11.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.651+0000 7f8849c00640 1 Processor -- start 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.651+0000 7f8849c00640 1 -- start start 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.651+0000 7f8849c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8844107bc0 0x7f884419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.651+0000 7f8849c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f884419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.651+0000 7f8849c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8844109770 0x7f88441a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8849c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88441136f0 con 0x7f8844108db0 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8849c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8844113570 con 0x7f8844107bc0 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8849c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8844113870 con 0x7f8844109770 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f884419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f884419e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55360/0 (socket says 172.21.15.3:55360) 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 -- 172.21.15.3:0/2863126426 learned_addr learned my addr 172.21.15.3:0/2863126426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8849200640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8844109770 0x7f88441a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:11.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843e00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8844107bc0 0x7f884419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 -- 172.21.15.3:0/2863126426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8844109770 msgr2=0x7f88441a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8844109770 0x7f88441a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 -- 172.21.15.3:0/2863126426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8844107bc0 msgr2=0x7f884419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8844107bc0 0x7f884419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.652+0000 7f8843400640 1 -- 172.21.15.3:0/2863126426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88441a59f0 con 0x7f8844108db0 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.653+0000 7f8843400640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f884419e310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f882c009dc0 tx=0x7f882c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.653+0000 7f8840c00640 1 -- 172.21.15.3:0/2863126426 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c041020 con 0x7f8844108db0 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.653+0000 7f8840c00640 1 -- 172.21.15.3:0/2863126426 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f882c003e20 con 0x7f8844108db0 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.653+0000 7f8840c00640 1 -- 172.21.15.3:0/2863126426 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c039040 con 0x7f8844108db0 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.653+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88441a5c80 con 0x7f8844108db0 2024-03-21T08:28:11.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.653+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88441a61c0 con 0x7f8844108db0 2024-03-21T08:28:11.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.654+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884410ef60 con 0x7f8844108db0 2024-03-21T08:28:11.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.657+0000 7f8840c00640 1 -- 172.21.15.3:0/2863126426 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f882c044020 con 0x7f8844108db0 2024-03-21T08:28:11.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.659+0000 7f8840c00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88280a2f90 0x7f88280a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:11.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.659+0000 7f8840c00640 1 -- 172.21.15.3:0/2863126426 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f882c0f11d0 con 0x7f8844108db0 2024-03-21T08:28:11.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.659+0000 7f8843e00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88280a2f90 0x7f88280a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:11.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.659+0000 7f8843e00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88280a2f90 0x7f88280a5450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8834005fd0 tx=0x7f8834005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:11.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.662+0000 7f8840c00640 1 -- 172.21.15.3:0/2863126426 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f882c04b020 con 0x7f8844108db0 2024-03-21T08:28:11.797 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 podman[66577]: 2024-03-21 08:28:11.365526649 +0000 UTC m=+0.032467959 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:28:11.797 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 podman[66577]: 2024-03-21 08:28:11.490938808 +0000 UTC m=+0.157880149 container create a817d127d543d4a3b559fa60c449930c42dc043b451f5093389b7efa78058813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:28:11.797 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 podman[66577]: 2024-03-21 08:28:11.794892296 +0000 UTC m=+0.461833647 container init a817d127d543d4a3b559fa60c449930c42dc043b451f5093389b7efa78058813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T08:28:11.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.905+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88440622a0 con 0x7f8844108db0 2024-03-21T08:28:11.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.907+0000 7f8840c00640 1 -- 172.21.15.3:0/2863126426 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5280 (secure 0 0 0) 0x7f882c0be400 con 0x7f8844108db0 2024-03-21T08:28:11.909 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:11.909 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T08:28:11.909 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-03-21T08:28:11.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88280a2f90 msgr2=0x7f88280a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:11.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88280a2f90 0x7f88280a5450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8834005fd0 tx=0x7f8834005850 comp rx=0 tx=0).stop 2024-03-21T08:28:11.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 msgr2=0x7f884419e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:11.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f884419e310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f882c009dc0 tx=0x7f882c0328a0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 shutdown_connections 2024-03-21T08:28:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88280a2f90 0x7f88280a5450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8844109770 0x7f88441a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8844108db0 0x7f884419e310 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 --2- 172.21.15.3:0/2863126426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8844107bc0 0x7f884419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 >> 172.21.15.3:0/2863126426 conn(0x7f8844076b60 msgr2=0x7f884410c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 shutdown_connections 2024-03-21T08:28:11.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:11.910+0000 7f8849c00640 1 -- 172.21.15.3:0/2863126426 wait complete. 2024-03-21T08:28:12.108 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 podman[66577]: 2024-03-21 08:28:11.804737327 +0000 UTC m=+0.471678640 container start a817d127d543d4a3b559fa60c449930c42dc043b451f5093389b7efa78058813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-21T08:28:12.108 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 ceph-mds[66595]: -- 172.21.15.171:0/3592255963 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55da80a17a40 con 0x55da809eac00 2024-03-21T08:28:12.108 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c[66591]: starting mds.c at 2024-03-21T08:28:12.108 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 bash[66577]: a817d127d543d4a3b559fa60c449930c42dc043b451f5093389b7efa78058813 2024-03-21T08:28:12.108 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:28:11 smithi171 systemd[1]: Started Ceph mds.c for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:12 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2863126426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:12 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:12.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:12 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2863126426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:12.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:12 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:12.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:12 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2863126426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:12.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:12.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:12 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:13.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:13 smithi084 ceph-mon[35477]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:13.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:13.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:13.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:13 smithi084 ceph-mon[35477]: Deploying daemon mds.h on smithi084 2024-03-21T08:28:13.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:13 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.171:6836/2662792128,v1:172.21.15.171:6837/2662792128] up:boot 2024-03-21T08:28:13.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:13 smithi084 ceph-mon[35477]: fsmap 7 up:standby 2024-03-21T08:28:13.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:13 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:28:13.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:13 smithi171 ceph-mon[35388]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:13 smithi171 ceph-mon[35388]: Deploying daemon mds.h on smithi084 2024-03-21T08:28:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:13 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.171:6836/2662792128,v1:172.21.15.171:6837/2662792128] up:boot 2024-03-21T08:28:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:13 smithi171 ceph-mon[35388]: fsmap 7 up:standby 2024-03-21T08:28:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:13 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:28:13.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:13 smithi003 ceph-mon[31675]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:13 smithi003 ceph-mon[31675]: Deploying daemon mds.h on smithi084 2024-03-21T08:28:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:13 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.171:6836/2662792128,v1:172.21.15.171:6837/2662792128] up:boot 2024-03-21T08:28:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:13 smithi003 ceph-mon[31675]: fsmap 7 up:standby 2024-03-21T08:28:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:13 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:28:13.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:14.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 -- 172.21.15.3:0/3245588722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908108db0 msgr2=0x7ff908109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:14.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 --2- 172.21.15.3:0/3245588722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908108db0 0x7ff908109230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f002bc50 comp rx=0 tx=0).stop 2024-03-21T08:28:14.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 -- 172.21.15.3:0/3245588722 shutdown_connections 2024-03-21T08:28:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 --2- 172.21.15.3:0/3245588722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff908109770 0x7ff908110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 --2- 172.21.15.3:0/3245588722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908108db0 0x7ff908109230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 --2- 172.21.15.3:0/3245588722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff908107bc0 0x7ff908107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 -- 172.21.15.3:0/3245588722 >> 172.21.15.3:0/3245588722 conn(0x7ff908076b60 msgr2=0x7ff908078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 -- 172.21.15.3:0/3245588722 shutdown_connections 2024-03-21T08:28:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.166+0000 7ff907200640 1 -- 172.21.15.3:0/3245588722 wait complete. 2024-03-21T08:28:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 Processor -- start 2024-03-21T08:28:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 -- start start 2024-03-21T08:28:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908107bc0 0x7ff90819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff908108db0 0x7ff90819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:14.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff908109770 0x7ff9081a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9081136f0 con 0x7ff908107bc0 2024-03-21T08:28:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908113570 con 0x7ff908109770 2024-03-21T08:28:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.167+0000 7ff907200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908113870 con 0x7ff908108db0 2024-03-21T08:28:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908107bc0 0x7ff90819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908107bc0 0x7ff90819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55372/0 (socket says 172.21.15.3:55372) 2024-03-21T08:28:14.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 -- 172.21.15.3:0/1428712922 learned_addr learned my addr 172.21.15.3:0/1428712922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff906800640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff908109770 0x7ff9081a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff8fd400640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff908108db0 0x7ff90819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 -- 172.21.15.3:0/1428712922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff908108db0 msgr2=0x7ff90819e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff908108db0 0x7ff90819e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 -- 172.21.15.3:0/1428712922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff908109770 msgr2=0x7ff9081a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:14.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff908109770 0x7ff9081a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff905e00640 1 -- 172.21.15.3:0/1428712922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9081a59c0 con 0x7ff908107bc0 2024-03-21T08:28:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.168+0000 7ff8fd400640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff908108db0 0x7ff90819e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.169+0000 7ff905e00640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908107bc0 0x7ff90819ddb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800ea80 tx=0x7ff8f800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.169+0000 7ff8ff400640 1 -- 172.21.15.3:0/1428712922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800ce40 con 0x7ff908107bc0 2024-03-21T08:28:14.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.169+0000 7ff8ff400640 1 -- 172.21.15.3:0/1428712922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8f8011920 con 0x7ff908107bc0 2024-03-21T08:28:14.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.169+0000 7ff8ff400640 1 -- 172.21.15.3:0/1428712922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f801aaa0 con 0x7ff908107bc0 2024-03-21T08:28:14.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.169+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9081a5cb0 con 0x7ff908107bc0 2024-03-21T08:28:14.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.169+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9081a61b0 con 0x7ff908107bc0 2024-03-21T08:28:14.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.170+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90810ef60 con 0x7ff908107bc0 2024-03-21T08:28:14.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.174+0000 7ff8ff400640 1 -- 172.21.15.3:0/1428712922 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7ff8f8011ad0 con 0x7ff908107bc0 2024-03-21T08:28:14.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.175+0000 7ff8ff400640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8ec0a2f90 0x7ff8ec0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:14.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.175+0000 7ff8ff400640 1 -- 172.21.15.3:0/1428712922 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8f80cf850 con 0x7ff908107bc0 2024-03-21T08:28:14.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.176+0000 7ff8fd400640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8ec0a2f90 0x7ff8ec0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:14.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.176+0000 7ff8fd400640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8ec0a2f90 0x7ff8ec0a5450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00096c0 tx=0x7ff8f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:14.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.178+0000 7ff8ff400640 1 -- 172.21.15.3:0/1428712922 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8f8016030 con 0x7ff908107bc0 2024-03-21T08:28:14.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.423+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff90802d320 con 0x7ff908107bc0 2024-03-21T08:28:14.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.424+0000 7ff8ff400640 1 -- 172.21.15.3:0/1428712922 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6074 (secure 0 0 0) 0x7ff8f809cb00 con 0x7ff908107bc0 2024-03-21T08:28:14.426 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:14.426 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T08:28:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-03-21T08:28:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8ec0a2f90 msgr2=0x7ff8ec0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8ec0a2f90 0x7ff8ec0a5450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00096c0 tx=0x7ff8f0000bc0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908107bc0 msgr2=0x7ff90819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908107bc0 0x7ff90819ddb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800ea80 tx=0x7ff8f800ef40 comp rx=0 tx=0).stop 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 shutdown_connections 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff908109770 0x7ff9081a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8ec0a2f90 0x7ff8ec0a5450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff908108db0 0x7ff90819e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 --2- 172.21.15.3:0/1428712922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff908107bc0 0x7ff90819ddb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.427+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 >> 172.21.15.3:0/1428712922 conn(0x7ff908076b60 msgr2=0x7ff90810c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.428+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 shutdown_connections 2024-03-21T08:28:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:14.428+0000 7ff907200640 1 -- 172.21.15.3:0/1428712922 wait complete. 2024-03-21T08:28:15.188 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:15 smithi084 ceph-mon[35477]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:15.188 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:15 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1428712922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:15.189 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 systemd[1]: Starting Ceph mds.h for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:28:15.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:15 smithi171 ceph-mon[35388]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:15.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:15 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1428712922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:15.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:15 smithi003 ceph-mon[31675]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:15.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:15 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1428712922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:15.807 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 podman[65114]: 2024-03-21 08:28:15.436222202 +0000 UTC m=+0.031744179 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:28:15.807 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 podman[65114]: 2024-03-21 08:28:15.542596581 +0000 UTC m=+0.138118597 container create 83c96345e0c4e79986d1c530ed4cae21dcd07c1e3d06d1e5c9d657fcf39f780b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T08:28:15.807 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 podman[65114]: 2024-03-21 08:28:15.805165108 +0000 UTC m=+0.400687143 container init 83c96345e0c4e79986d1c530ed4cae21dcd07c1e3d06d1e5c9d657fcf39f780b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T08:28:16.126 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 podman[65114]: 2024-03-21 08:28:15.815212407 +0000 UTC m=+0.410734442 container start 83c96345e0c4e79986d1c530ed4cae21dcd07c1e3d06d1e5c9d657fcf39f780b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T08:28:16.126 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 ceph-mds[65133]: -- 172.21.15.84:0/140144242 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5631714b1a40 con 0x563171484c00 2024-03-21T08:28:16.126 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h[65129]: starting mds.h at 2024-03-21T08:28:16.126 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 bash[65114]: 83c96345e0c4e79986d1c530ed4cae21dcd07c1e3d06d1e5c9d657fcf39f780b 2024-03-21T08:28:16.126 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:15 smithi084 systemd[1]: Started Ceph mds.h for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:16.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 -- 172.21.15.3:0/431624971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 msgr2=0x7f3290107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 --2- 172.21.15.3:0/431624971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f3290107fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3280009ca0 tx=0x7f328002c100 comp rx=0 tx=0).stop 2024-03-21T08:28:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 -- 172.21.15.3:0/431624971 shutdown_connections 2024-03-21T08:28:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 --2- 172.21.15.3:0/431624971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3290109770 0x7f3290110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 --2- 172.21.15.3:0/431624971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3290108db0 0x7f3290109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 --2- 172.21.15.3:0/431624971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f3290107fe0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 -- 172.21.15.3:0/431624971 >> 172.21.15.3:0/431624971 conn(0x7f3290076b60 msgr2=0x7f3290078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:16.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 -- 172.21.15.3:0/431624971 shutdown_connections 2024-03-21T08:28:16.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.684+0000 7f328fe00640 1 -- 172.21.15.3:0/431624971 wait complete. 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.685+0000 7f328fe00640 1 Processor -- start 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.685+0000 7f328fe00640 1 -- start start 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.685+0000 7f328fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f329019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.685+0000 7f328fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3290108db0 0x7f329019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3290109770 0x7f32901a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32901136c0 con 0x7f3290107bc0 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3290113540 con 0x7f3290109770 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3290113840 con 0x7f3290108db0 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f329019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f329019dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55402/0 (socket says 172.21.15.3:55402) 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 -- 172.21.15.3:0/2860970457 learned_addr learned my addr 172.21.15.3:0/2860970457 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f3286000640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3290108db0 0x7f329019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328f400640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3290109770 0x7f32901a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 -- 172.21.15.3:0/2860970457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3290108db0 msgr2=0x7f329019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3290108db0 0x7f329019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 -- 172.21.15.3:0/2860970457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3290109770 msgr2=0x7f32901a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3290109770 0x7f32901a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328ea00640 1 -- 172.21.15.3:0/2860970457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32901a5900 con 0x7f3290107bc0 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.686+0000 7f328f400640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3290109770 0x7f32901a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.687+0000 7f328ea00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f329019dcf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3280000f80 tx=0x7f3280035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.687+0000 7f3294a00640 1 -- 172.21.15.3:0/2860970457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280039040 con 0x7f3290107bc0 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.687+0000 7f3294a00640 1 -- 172.21.15.3:0/2860970457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3280044070 con 0x7f3290107bc0 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.687+0000 7f3294a00640 1 -- 172.21.15.3:0/2860970457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280004640 con 0x7f3290107bc0 2024-03-21T08:28:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.687+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32901a5b90 con 0x7f3290107bc0 2024-03-21T08:28:16.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.687+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32901a6080 con 0x7f3290107bc0 2024-03-21T08:28:16.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.688+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329010eee0 con 0x7f3290107bc0 2024-03-21T08:28:16.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.690+0000 7f3294a00640 1 -- 172.21.15.3:0/2860970457 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f32800327c0 con 0x7f3290107bc0 2024-03-21T08:28:16.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.691+0000 7f3294a00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f32740a2f10 0x7f32740a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:16.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.692+0000 7f3294a00640 1 -- 172.21.15.3:0/2860970457 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f32800f19d0 con 0x7f3290107bc0 2024-03-21T08:28:16.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.692+0000 7f3286000640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f32740a2f10 0x7f32740a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:16.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.692+0000 7f3286000640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f32740a2f10 0x7f32740a53d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3278005e10 tx=0x7f3278005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:16.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.696+0000 7f3294a00640 1 -- 172.21.15.3:0/2860970457 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f32800bec00 con 0x7f3290107bc0 2024-03-21T08:28:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.944+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32900622a0 con 0x7f3290107bc0 2024-03-21T08:28:16.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.945+0000 7f3294a00640 1 -- 172.21.15.3:0/2860970457 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6865 (secure 0 0 0) 0x7f32800be500 con 0x7f3290107bc0 2024-03-21T08:28:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:16.947 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T08:28:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f32740a2f10 msgr2=0x7f32740a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f32740a2f10 0x7f32740a53d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3278005e10 tx=0x7f3278005da0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 msgr2=0x7f329019dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f329019dcf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3280000f80 tx=0x7f3280035d60 comp rx=0 tx=0).stop 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 shutdown_connections 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3290109770 0x7f32901a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f32740a2f10 0x7f32740a53d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.948+0000 7f328fe00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3290108db0 0x7f329019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.949+0000 7f328fe00640 1 --2- 172.21.15.3:0/2860970457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3290107bc0 0x7f329019dcf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.949+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 >> 172.21.15.3:0/2860970457 conn(0x7f3290076b60 msgr2=0x7f329010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.949+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 shutdown_connections 2024-03-21T08:28:16.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:16.949+0000 7f328fe00640 1 -- 172.21.15.3:0/2860970457 wait complete. 2024-03-21T08:28:17.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:17.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] up:boot 2024-03-21T08:28:17.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: fsmap 8 up:standby 2024-03-21T08:28:17.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:28:17.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:17.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:17.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:17.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: Deploying daemon mds.i on smithi171 2024-03-21T08:28:17.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:17 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2860970457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:17.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:17.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] up:boot 2024-03-21T08:28:17.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: fsmap 8 up:standby 2024-03-21T08:28:17.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:28:17.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:17.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:17.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:17.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: Deploying daemon mds.i on smithi171 2024-03-21T08:28:17.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:17 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2860970457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] up:boot 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: fsmap 8 up:standby 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: Deploying daemon mds.i on smithi171 2024-03-21T08:28:17.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:17 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2860970457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:18.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:19.153 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:19 smithi171 ceph-mon[35388]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.202+0000 7f17b2200640 1 -- 172.21.15.3:0/146884204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac107ce0 msgr2=0x7f17ac108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.202+0000 7f17b2200640 1 --2- 172.21.15.3:0/146884204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac107ce0 0x7f17ac108100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f179c009900 tx=0x7f179c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.202+0000 7f17b2200640 1 -- 172.21.15.3:0/146884204 shutdown_connections 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.202+0000 7f17b2200640 1 --2- 172.21.15.3:0/146884204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17ac109890 0x7f17ac110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.203+0000 7f17b2200640 1 --2- 172.21.15.3:0/146884204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ac108ed0 0x7f17ac109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.203+0000 7f17b2200640 1 --2- 172.21.15.3:0/146884204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac107ce0 0x7f17ac108100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.203+0000 7f17b2200640 1 -- 172.21.15.3:0/146884204 >> 172.21.15.3:0/146884204 conn(0x7f17ac076ca0 msgr2=0x7f17ac0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.203+0000 7f17b2200640 1 -- 172.21.15.3:0/146884204 shutdown_connections 2024-03-21T08:28:19.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.203+0000 7f17b2200640 1 -- 172.21.15.3:0/146884204 wait complete. 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 Processor -- start 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 -- start start 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ac107ce0 0x7f17ac19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17ac108ed0 0x7f17ac19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac109890 0x7f17ac1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac113800 con 0x7f17ac109890 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac113680 con 0x7f17ac108ed0 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17b2200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac113980 con 0x7f17ac107ce0 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac109890 0x7f17ac1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.204+0000 7f17abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac109890 0x7f17ac1a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55434/0 (socket says 172.21.15.3:55434) 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17abe00640 1 -- 172.21.15.3:0/3672855195 learned_addr learned my addr 172.21.15.3:0/3672855195 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17ab400640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ac107ce0 0x7f17ac19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17a3e00640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17ac108ed0 0x7f17ac19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17abe00640 1 -- 172.21.15.3:0/3672855195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ac107ce0 msgr2=0x7f17ac19df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17abe00640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ac107ce0 0x7f17ac19df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17abe00640 1 -- 172.21.15.3:0/3672855195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17ac108ed0 msgr2=0x7f17ac19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17abe00640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17ac108ed0 0x7f17ac19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17abe00640 1 -- 172.21.15.3:0/3672855195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac1a5b50 con 0x7f17ac109890 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17abe00640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac109890 0x7f17ac1a54e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17a400eaa0 tx=0x7f17a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.205+0000 7f17a8c00640 1 -- 172.21.15.3:0/3672855195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a400cdb0 con 0x7f17ac109890 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.206+0000 7f17a8c00640 1 -- 172.21.15.3:0/3672855195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17a4005020 con 0x7f17ac109890 2024-03-21T08:28:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.206+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ac1a5e40 con 0x7f17ac109890 2024-03-21T08:28:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.206+0000 7f17a8c00640 1 -- 172.21.15.3:0/3672855195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a40116d0 con 0x7f17ac109890 2024-03-21T08:28:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.206+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac1a62b0 con 0x7f17ac109890 2024-03-21T08:28:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.207+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac10f120 con 0x7f17ac109890 2024-03-21T08:28:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.209+0000 7f17a8c00640 1 -- 172.21.15.3:0/3672855195 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f17a40051d0 con 0x7f17ac109890 2024-03-21T08:28:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.210+0000 7f17a8c00640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f17900a2f20 0x7f17900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.210+0000 7f17a8c00640 1 -- 172.21.15.3:0/3672855195 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17a40cf580 con 0x7f17ac109890 2024-03-21T08:28:19.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.211+0000 7f17ab400640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f17900a2f20 0x7f17900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.211+0000 7f17ab400640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f17900a2f20 0x7f17900a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f179c005ab0 tx=0x7f179c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.213+0000 7f17a8c00640 1 -- 172.21.15.3:0/3672855195 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17a4016030 con 0x7f17ac109890 2024-03-21T08:28:19.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:19 smithi084 ceph-mon[35477]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:19.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:19 smithi003 ceph-mon[31675]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:19.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.456+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17ac02d2c0 con 0x7f17ac109890 2024-03-21T08:28:19.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.457+0000 7f17a8c00640 1 -- 172.21.15.3:0/3672855195 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6865 (secure 0 0 0) 0x7f17a40bae20 con 0x7f17ac109890 2024-03-21T08:28:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:19.458 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T08:28:19.458 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2024-03-21T08:28:19.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f17900a2f20 msgr2=0x7f17900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f17900a2f20 0x7f17900a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f179c005ab0 tx=0x7f179c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac109890 msgr2=0x7f17ac1a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac109890 0x7f17ac1a54e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17a400eaa0 tx=0x7f17a400ef60 comp rx=0 tx=0).stop 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 shutdown_connections 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17ac109890 0x7f17ac1a54e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17ac108ed0 0x7f17ac19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f17900a2f20 0x7f17900a53e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 --2- 172.21.15.3:0/3672855195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ac107ce0 0x7f17ac19df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 >> 172.21.15.3:0/3672855195 conn(0x7f17ac076ca0 msgr2=0x7f17ac10c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.459+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 shutdown_connections 2024-03-21T08:28:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:19.460+0000 7f17b2200640 1 -- 172.21.15.3:0/3672855195 wait complete. 2024-03-21T08:28:19.516 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 systemd[1]: Starting Ceph mds.i for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:28:19.827 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 podman[66879]: 2024-03-21 08:28:19.41922033 +0000 UTC m=+0.032170341 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:28:19.827 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 podman[66879]: 2024-03-21 08:28:19.572077362 +0000 UTC m=+0.185027424 container create 5339140889dd01af6b1d23b3d98f4dbb238ec37480412eb587ba364b69b61029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T08:28:20.087 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 podman[66879]: 2024-03-21 08:28:19.825813452 +0000 UTC m=+0.438763472 container init 5339140889dd01af6b1d23b3d98f4dbb238ec37480412eb587ba364b69b61029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T08:28:20.088 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 podman[66879]: 2024-03-21 08:28:19.833443926 +0000 UTC m=+0.446393979 container start 5339140889dd01af6b1d23b3d98f4dbb238ec37480412eb587ba364b69b61029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:28:20.088 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 ceph-mds[66906]: -- 172.21.15.171:0/685115431 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556294713a40 con 0x5562946e6800 2024-03-21T08:28:20.088 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i[66897]: starting mds.i at 2024-03-21T08:28:20.088 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 bash[66879]: 5339140889dd01af6b1d23b3d98f4dbb238ec37480412eb587ba364b69b61029 2024-03-21T08:28:20.088 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:28:19 smithi171 systemd[1]: Started Ceph mds.i for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:20.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:20 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3672855195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:20.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:20 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3672855195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:20.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:20 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3672855195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:21.366 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:21.366 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.171:6838/1010033316,v1:172.21.15.171:6839/1010033316] up:boot 2024-03-21T08:28:21.366 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: fsmap 9 up:standby 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:21.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:21 smithi003 ceph-mon[31675]: Deploying daemon mds.g on smithi003 2024-03-21T08:28:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.171:6838/1010033316,v1:172.21.15.171:6839/1010033316] up:boot 2024-03-21T08:28:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: fsmap 9 up:standby 2024-03-21T08:28:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:28:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:21.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:21.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:21.379 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:21 smithi084 ceph-mon[35477]: Deploying daemon mds.g on smithi003 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.171:6838/1010033316,v1:172.21.15.171:6839/1010033316] up:boot 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: fsmap 9 up:standby 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:21.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:21.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:21.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:21.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:21 smithi171 ceph-mon[35388]: Deploying daemon mds.g on smithi003 2024-03-21T08:28:21.481 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:21.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.691+0000 7fa70b000640 1 -- 172.21.15.3:0/1004081066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704109890 msgr2=0x7fa704110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.691+0000 7fa70b000640 1 --2- 172.21.15.3:0/1004081066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704109890 0x7fa704110120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009ca0 tx=0x7fa6fc02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.691+0000 7fa70b000640 1 -- 172.21.15.3:0/1004081066 shutdown_connections 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.691+0000 7fa70b000640 1 --2- 172.21.15.3:0/1004081066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704109890 0x7fa704110120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.691+0000 7fa70b000640 1 --2- 172.21.15.3:0/1004081066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa704108ed0 0x7fa704109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.691+0000 7fa70b000640 1 --2- 172.21.15.3:0/1004081066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa704107ce0 0x7fa704108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.691+0000 7fa70b000640 1 -- 172.21.15.3:0/1004081066 >> 172.21.15.3:0/1004081066 conn(0x7fa704076ca0 msgr2=0x7fa7040790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.692+0000 7fa70b000640 1 -- 172.21.15.3:0/1004081066 shutdown_connections 2024-03-21T08:28:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.692+0000 7fa70b000640 1 -- 172.21.15.3:0/1004081066 wait complete. 2024-03-21T08:28:21.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 Processor -- start 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 -- start start 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa704107ce0 0x7fa70419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704108ed0 0x7fa70419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa704109890 0x7fa7041a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa704113810 con 0x7fa704108ed0 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa704113690 con 0x7fa704109890 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa70b000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa704113990 con 0x7fa704107ce0 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa6fbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704108ed0 0x7fa70419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa6fbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704108ed0 0x7fa70419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36154/0 (socket says 172.21.15.3:36154) 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.693+0000 7fa6fbe00640 1 -- 172.21.15.3:0/3975487215 learned_addr learned my addr 172.21.15.3:0/3975487215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa709000640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa704109890 0x7fa7041a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa703e00640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa704107ce0 0x7fa70419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa6fbe00640 1 -- 172.21.15.3:0/3975487215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa704107ce0 msgr2=0x7fa70419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa6fbe00640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa704107ce0 0x7fa70419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa6fbe00640 1 -- 172.21.15.3:0/3975487215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa704109890 msgr2=0x7fa7041a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa6fbe00640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa704109890 0x7fa7041a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa6fbe00640 1 -- 172.21.15.3:0/3975487215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7041a5b10 con 0x7fa704108ed0 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa6fbe00640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704108ed0 0x7fa70419e430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d770 tx=0x7fa6ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa701600640 1 -- 172.21.15.3:0/3975487215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec017070 con 0x7fa704108ed0 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa701600640 1 -- 172.21.15.3:0/3975487215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa6ec0054d0 con 0x7fa704108ed0 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7041a5e00 con 0x7fa704108ed0 2024-03-21T08:28:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa701600640 1 -- 172.21.15.3:0/3975487215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004cf0 con 0x7fa704108ed0 2024-03-21T08:28:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.694+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7041a62c0 con 0x7fa704108ed0 2024-03-21T08:28:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.696+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70410f020 con 0x7fa704108ed0 2024-03-21T08:28:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.698+0000 7fa701600640 1 -- 172.21.15.3:0/3975487215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fa6ec004e90 con 0x7fa704108ed0 2024-03-21T08:28:21.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.699+0000 7fa701600640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6e80a2fa0 0x7fa6e80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:21.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.699+0000 7fa701600640 1 -- 172.21.15.3:0/3975487215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6ec0cedd0 con 0x7fa704108ed0 2024-03-21T08:28:21.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.700+0000 7fa703e00640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6e80a2fa0 0x7fa6e80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:21.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.700+0000 7fa703e00640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6e80a2fa0 0x7fa6e80a5460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005fd0 tx=0x7fa6f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.702+0000 7fa701600640 1 -- 172.21.15.3:0/3975487215 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa6ec09c080 con 0x7fa704108ed0 2024-03-21T08:28:21.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.951+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa70402d110 con 0x7fa704108ed0 2024-03-21T08:28:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.952+0000 7fa701600640 1 -- 172.21.15.3:0/3975487215 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7661 (secure 0 0 0) 0x7fa6ec09b980 con 0x7fa704108ed0 2024-03-21T08:28:21.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:21.954 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T08:28:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 10 2024-03-21T08:28:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6e80a2fa0 msgr2=0x7fa6e80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6e80a2fa0 0x7fa6e80a5460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005fd0 tx=0x7fa6f4005850 comp rx=0 tx=0).stop 2024-03-21T08:28:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704108ed0 msgr2=0x7fa70419e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704108ed0 0x7fa70419e430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d770 tx=0x7fa6ec00dc30 comp rx=0 tx=0).stop 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 shutdown_connections 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa704109890 0x7fa7041a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa704108ed0 0x7fa70419e430 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6e80a2fa0 0x7fa6e80a5460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 --2- 172.21.15.3:0/3975487215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa704107ce0 0x7fa70419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 >> 172.21.15.3:0/3975487215 conn(0x7fa704076ca0 msgr2=0x7fa70410c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.955+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 shutdown_connections 2024-03-21T08:28:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:21.956+0000 7fa70b000640 1 -- 172.21.15.3:0/3975487215 wait complete. 2024-03-21T08:28:22.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:22 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3975487215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:22.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:22 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3975487215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:22.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:22 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3975487215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:23.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:23 smithi003 ceph-mon[31675]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:23.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:23 smithi084 ceph-mon[35477]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:23.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:23 smithi171 ceph-mon[35388]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:23.599 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:23 smithi003 systemd[1]: Starting Ceph mds.g for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:28:23.915 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:23 smithi003 podman[72934]: 2024-03-21 08:28:23.51030843 +0000 UTC m=+0.037566880 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:28:23.915 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:23 smithi003 podman[72934]: 2024-03-21 08:28:23.634789897 +0000 UTC m=+0.162048360 container create 01c0afe2d76de61515fe8108b2c467ae55a4df48987f0a2d23ee356ae84d47fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T08:28:23.915 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:23 smithi003 podman[72934]: 2024-03-21 08:28:23.91361304 +0000 UTC m=+0.440871510 container init 01c0afe2d76de61515fe8108b2c467ae55a4df48987f0a2d23ee356ae84d47fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T08:28:23.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.172+0000 7f4fb3200640 1 -- 172.21.15.3:0/4066374193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb40ffe20 msgr2=0x7f4fb4100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.172+0000 7f4fb3200640 1 --2- 172.21.15.3:0/4066374193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb40ffe20 0x7f4fb4100240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009ca0 tx=0x7f4fa402c100 comp rx=0 tx=0).stop 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.173+0000 7f4fb3200640 1 -- 172.21.15.3:0/4066374193 shutdown_connections 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.173+0000 7f4fb3200640 1 --2- 172.21.15.3:0/4066374193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb4101120 0x7f4fb41015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.173+0000 7f4fb3200640 1 --2- 172.21.15.3:0/4066374193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fb4100780 0x7f4fb4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.173+0000 7f4fb3200640 1 --2- 172.21.15.3:0/4066374193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb40ffe20 0x7f4fb4100240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.173+0000 7f4fb3200640 1 -- 172.21.15.3:0/4066374193 >> 172.21.15.3:0/4066374193 conn(0x7f4fb40fba20 msgr2=0x7f4fb40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.173+0000 7f4fb3200640 1 -- 172.21.15.3:0/4066374193 shutdown_connections 2024-03-21T08:28:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.173+0000 7f4fb3200640 1 -- 172.21.15.3:0/4066374193 wait complete. 2024-03-21T08:28:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 Processor -- start 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 -- start start 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb40ffe20 0x7f4fb41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4100780 0x7f4fb4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fb4101120 0x7f4fb41a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410f2c0 con 0x7f4fb4100780 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410f140 con 0x7f4fb4101120 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410f440 con 0x7f4fb40ffe20 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4100780 0x7f4fb4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4100780 0x7f4fb4199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36190/0 (socket says 172.21.15.3:36190) 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.174+0000 7f4fb1400640 1 -- 172.21.15.3:0/3411291718 learned_addr learned my addr 172.21.15.3:0/3411291718 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb1e00640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb40ffe20 0x7f4fb41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb2800640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fb4101120 0x7f4fb41a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb1400640 1 -- 172.21.15.3:0/3411291718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb40ffe20 msgr2=0x7f4fb41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb1400640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb40ffe20 0x7f4fb41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb1400640 1 -- 172.21.15.3:0/3411291718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fb4101120 msgr2=0x7f4fb41a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb1400640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fb4101120 0x7f4fb41a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb1400640 1 -- 172.21.15.3:0/3411291718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb41a15f0 con 0x7f4fb4100780 2024-03-21T08:28:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.175+0000 7f4fb1400640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4100780 0x7f4fb4199f10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00c920 tx=0x7f4f9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:24.178 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:23 smithi003 podman[72934]: 2024-03-21 08:28:23.923858484 +0000 UTC m=+0.451116961 container start 01c0afe2d76de61515fe8108b2c467ae55a4df48987f0a2d23ee356ae84d47fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T08:28:24.178 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:23 smithi003 ceph-mds[72960]: -- 172.21.15.3:0/882601494 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1b5f3ba40 con 0x55e1b5f0ec00 2024-03-21T08:28:24.178 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g[72952]: starting mds.g at 2024-03-21T08:28:24.178 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:24 smithi003 bash[72934]: 01c0afe2d76de61515fe8108b2c467ae55a4df48987f0a2d23ee356ae84d47fa 2024-03-21T08:28:24.179 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:28:24 smithi003 systemd[1]: Started Ceph mds.g for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.176+0000 7f4faaa00640 1 -- 172.21.15.3:0/3411291718 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c007c70 con 0x7f4fb4100780 2024-03-21T08:28:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.176+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb41a18e0 con 0x7f4fb4100780 2024-03-21T08:28:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.176+0000 7f4faaa00640 1 -- 172.21.15.3:0/3411291718 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4f9c007e10 con 0x7f4fb4100780 2024-03-21T08:28:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.176+0000 7f4faaa00640 1 -- 172.21.15.3:0/3411291718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c018820 con 0x7f4fb4100780 2024-03-21T08:28:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.176+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb41a1e80 con 0x7f4fb4100780 2024-03-21T08:28:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.177+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb410ab70 con 0x7f4fb4100780 2024-03-21T08:28:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.179+0000 7f4faaa00640 1 -- 172.21.15.3:0/3411291718 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f4f9c0189c0 con 0x7f4fb4100780 2024-03-21T08:28:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.181+0000 7f4faaa00640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f980a2f90 0x7f4f980a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.181+0000 7f4faaa00640 1 -- 172.21.15.3:0/3411291718 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f9c0cf3b0 con 0x7f4fb4100780 2024-03-21T08:28:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.181+0000 7f4fb1e00640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f980a2f90 0x7f4f980a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.182+0000 7f4fb1e00640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f980a2f90 0x7f4f980a5450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000f80 tx=0x7f4fa4032420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.183+0000 7f4faaa00640 1 -- 172.21.15.3:0/3411291718 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f9c014030 con 0x7f4fb4100780 2024-03-21T08:28:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.447+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fb402d2c0 con 0x7f4fb4100780 2024-03-21T08:28:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.447+0000 7f4faaa00640 1 -- 172.21.15.3:0/3411291718 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8450 (secure 0 0 0) 0x7f4f9c09c6f0 con 0x7f4fb4100780 2024-03-21T08:28:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:24.451 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T08:28:24.452 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 11 2024-03-21T08:28:24.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f980a2f90 msgr2=0x7f4f980a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:24.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f980a2f90 0x7f4f980a5450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000f80 tx=0x7f4fa4032420 comp rx=0 tx=0).stop 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4100780 msgr2=0x7f4fb4199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4100780 0x7f4fb4199f10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00c920 tx=0x7f4f9c00cde0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 shutdown_connections 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fb4101120 0x7f4fb41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fb4100780 0x7f4fb4199f10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f980a2f90 0x7f4f980a5450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 --2- 172.21.15.3:0/3411291718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb40ffe20 0x7f4fb41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 >> 172.21.15.3:0/3411291718 conn(0x7f4fb40fba20 msgr2=0x7f4fb40fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.450+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 shutdown_connections 2024-03-21T08:28:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:24.451+0000 7f4fb3200640 1 -- 172.21.15.3:0/3411291718 wait complete. 2024-03-21T08:28:25.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.3:6838/3720466130,v1:172.21.15.3:6839/3720466130] up:boot 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: fsmap 10 up:standby 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: Deploying daemon mds.a on smithi003 2024-03-21T08:28:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:25 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3411291718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.3:6838/3720466130,v1:172.21.15.3:6839/3720466130] up:boot 2024-03-21T08:28:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: fsmap 10 up:standby 2024-03-21T08:28:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:28:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:25.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:25.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:25.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: Deploying daemon mds.a on smithi003 2024-03-21T08:28:25.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:25 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3411291718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.3:6838/3720466130,v1:172.21.15.3:6839/3720466130] up:boot 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: fsmap 10 up:standby 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: Deploying daemon mds.a on smithi003 2024-03-21T08:28:25.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:25 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3411291718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:26.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:26.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.867+0000 7f705f400640 1 -- 172.21.15.3:0/1385076910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7058076060 msgr2=0x7f7058076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:26.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.867+0000 7f705f400640 1 --2- 172.21.15.3:0/1385076910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7058076060 0x7f7058076480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c02bc50 comp rx=0 tx=0).stop 2024-03-21T08:28:26.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.867+0000 7f705f400640 1 -- 172.21.15.3:0/1385076910 shutdown_connections 2024-03-21T08:28:26.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.867+0000 7f705f400640 1 --2- 172.21.15.3:0/1385076910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7058077c40 0x7f70580784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:26.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.867+0000 7f705f400640 1 --2- 172.21.15.3:0/1385076910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70580772a0 0x7f7058077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.867+0000 7f705f400640 1 --2- 172.21.15.3:0/1385076910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7058076060 0x7f7058076480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.867+0000 7f705f400640 1 -- 172.21.15.3:0/1385076910 >> 172.21.15.3:0/1385076910 conn(0x7f70580fef00 msgr2=0x7f7058101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.868+0000 7f705f400640 1 -- 172.21.15.3:0/1385076910 shutdown_connections 2024-03-21T08:28:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.868+0000 7f705f400640 1 -- 172.21.15.3:0/1385076910 wait complete. 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.869+0000 7f705f400640 1 Processor -- start 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.869+0000 7f705f400640 1 -- start start 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7058076060 0x7f7058070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70580772a0 0x7f7058071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7058077c40 0x7f7058072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705f400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705806b590 con 0x7f7058077c40 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705f400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705806b410 con 0x7f70580772a0 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705f400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705806b710 con 0x7f7058076060 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7058077c40 0x7f7058072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7058077c40 0x7f7058072460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36206/0 (socket says 172.21.15.3:36206) 2024-03-21T08:28:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705d400640 1 -- 172.21.15.3:0/621477087 learned_addr learned my addr 172.21.15.3:0/621477087 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.870+0000 7f705ca00640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7058076060 0x7f7058070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.871+0000 7f7057e00640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70580772a0 0x7f7058071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.871+0000 7f705d400640 1 -- 172.21.15.3:0/621477087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7058076060 msgr2=0x7f7058070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.871+0000 7f705d400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7058076060 0x7f7058070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.871+0000 7f705d400640 1 -- 172.21.15.3:0/621477087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70580772a0 msgr2=0x7f7058071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.871+0000 7f705d400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70580772a0 0x7f7058071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.871+0000 7f705d400640 1 -- 172.21.15.3:0/621477087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7058072ad0 con 0x7f7058077c40 2024-03-21T08:28:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.872+0000 7f705d400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7058077c40 0x7f7058072460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f705000eb10 tx=0x7f705000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.872+0000 7f7055600640 1 -- 172.21.15.3:0/621477087 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050004b20 con 0x7f7058077c40 2024-03-21T08:28:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.872+0000 7f7055600640 1 -- 172.21.15.3:0/621477087 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7050004cc0 con 0x7f7058077c40 2024-03-21T08:28:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.872+0000 7f7055600640 1 -- 172.21.15.3:0/621477087 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050010690 con 0x7f7058077c40 2024-03-21T08:28:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.872+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7058072dc0 con 0x7f7058077c40 2024-03-21T08:28:26.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.873+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70580732c0 con 0x7f7058077c40 2024-03-21T08:28:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.874+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70581103f0 con 0x7f7058077c40 2024-03-21T08:28:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.881+0000 7f7055600640 1 -- 172.21.15.3:0/621477087 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f7050005000 con 0x7f7058077c40 2024-03-21T08:28:26.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.884+0000 7f7055600640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70400a3060 0x7f70400a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.884+0000 7f7055600640 1 -- 172.21.15.3:0/621477087 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70500cfc70 con 0x7f7058077c40 2024-03-21T08:28:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.884+0000 7f705ca00640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70400a3060 0x7f70400a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:26.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.886+0000 7f705ca00640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70400a3060 0x7f70400a5520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c02c5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:26.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:26.887+0000 7f7055600640 1 -- 172.21.15.3:0/621477087 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7050026020 con 0x7f7058077c40 2024-03-21T08:28:27.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.136+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70580735e0 con 0x7f7058077c40 2024-03-21T08:28:27.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.137+0000 7f7055600640 1 -- 172.21.15.3:0/621477087 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8450 (secure 0 0 0) 0x7f705009cea0 con 0x7f7058077c40 2024-03-21T08:28:27.138 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:27.138 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T08:28:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 11 2024-03-21T08:28:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70400a3060 msgr2=0x7f70400a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70400a3060 0x7f70400a5520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c02c5f0 comp rx=0 tx=0).stop 2024-03-21T08:28:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7058077c40 msgr2=0x7f7058072460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7058077c40 0x7f7058072460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f705000eb10 tx=0x7f705000c2d0 comp rx=0 tx=0).stop 2024-03-21T08:28:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 shutdown_connections 2024-03-21T08:28:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7058077c40 0x7f7058072460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70580772a0 0x7f7058071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70400a3060 0x7f70400a5520 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 --2- 172.21.15.3:0/621477087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7058076060 0x7f7058070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 >> 172.21.15.3:0/621477087 conn(0x7f70580fef00 msgr2=0x7f70581005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 shutdown_connections 2024-03-21T08:28:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:27.140+0000 7f705f400640 1 -- 172.21.15.3:0/621477087 wait complete. 2024-03-21T08:28:27.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:27 smithi084 ceph-mon[35477]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:27.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:27 smithi171 ceph-mon[35388]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:27.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:27 smithi003 ceph-mon[31675]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:27.797 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:28:27 smithi003 podman[73329]: 2024-03-21 08:28:27.532891821 +0000 UTC m=+0.176464877 container create 04c79969f28d0a32a1b11d37f58c5737e38f28076689c2d28c61811b57498059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T08:28:27.797 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:28:27 smithi003 podman[73329]: 2024-03-21 08:28:27.795532757 +0000 UTC m=+0.439105829 container init 04c79969f28d0a32a1b11d37f58c5737e38f28076689c2d28c61811b57498059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T08:28:28.135 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:28:27 smithi003 podman[73329]: 2024-03-21 08:28:27.807443307 +0000 UTC m=+0.451016302 container start 04c79969f28d0a32a1b11d37f58c5737e38f28076689c2d28c61811b57498059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:28:28.135 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:28:27 smithi003 ceph-mds[73347]: -- 172.21.15.3:0/2959819530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561e8c769a40 con 0x561e8c73cc00 2024-03-21T08:28:28.135 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:28:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a[73343]: starting mds.a at 2024-03-21T08:28:28.135 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:28:27 smithi003 bash[73329]: 04c79969f28d0a32a1b11d37f58c5737e38f28076689c2d28c61811b57498059 2024-03-21T08:28:28.135 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:28:27 smithi003 systemd[1]: Started Ceph mds.a for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:28 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/621477087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:28.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:28.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:28 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:28.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:28 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/621477087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:28.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:28 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:28.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:28 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/621477087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:28:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:28:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:28 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:29.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:29.223 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:29 smithi084 ceph-mon[35477]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:29.223 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:29 smithi084 ceph-mon[35477]: Deploying daemon mds.b on smithi084 2024-03-21T08:28:29.223 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:29 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.3:6840/2348469157,v1:172.21.15.3:6841/2348469157] up:boot 2024-03-21T08:28:29.223 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:29 smithi084 ceph-mon[35477]: fsmap 11 up:standby 2024-03-21T08:28:29.223 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:28:29.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 -- 172.21.15.3:0/1117232844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109890 msgr2=0x7fc90c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:29.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 --2- 172.21.15.3:0/1117232844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109890 0x7fc90c110120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc904009ca0 tx=0x7fc90402c100 comp rx=0 tx=0).stop 2024-03-21T08:28:29.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 -- 172.21.15.3:0/1117232844 shutdown_connections 2024-03-21T08:28:29.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 --2- 172.21.15.3:0/1117232844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c109890 0x7fc90c110120 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 --2- 172.21.15.3:0/1117232844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c108ed0 0x7fc90c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 --2- 172.21.15.3:0/1117232844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc90c107ce0 0x7fc90c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 -- 172.21.15.3:0/1117232844 >> 172.21.15.3:0/1117232844 conn(0x7fc90c076ca0 msgr2=0x7fc90c0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 -- 172.21.15.3:0/1117232844 shutdown_connections 2024-03-21T08:28:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.400+0000 7fc912a00640 1 -- 172.21.15.3:0/1117232844 wait complete. 2024-03-21T08:28:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.401+0000 7fc912a00640 1 Processor -- start 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc912a00640 1 -- start start 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc912a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c107ce0 0x7fc90c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc912a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc90c108ed0 0x7fc90c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc912a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c109890 0x7fc90c1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc912a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c113870 con 0x7fc90c107ce0 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc912a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c1136f0 con 0x7fc90c108ed0 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc912a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c1139f0 con 0x7fc90c109890 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc90be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c107ce0 0x7fc90c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc90be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c107ce0 0x7fc90c19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36240/0 (socket says 172.21.15.3:36240) 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc90be00640 1 -- 172.21.15.3:0/3272467366 learned_addr learned my addr 172.21.15.3:0/3272467366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc90b400640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc90c108ed0 0x7fc90c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.402+0000 7fc910a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c109890 0x7fc90c1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc90be00640 1 -- 172.21.15.3:0/3272467366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c109890 msgr2=0x7fc90c1a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc90be00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c109890 0x7fc90c1a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc90be00640 1 -- 172.21.15.3:0/3272467366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc90c108ed0 msgr2=0x7fc90c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc90be00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc90c108ed0 0x7fc90c19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc90be00640 1 -- 172.21.15.3:0/3272467366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c1a5b30 con 0x7fc90c107ce0 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc910a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c109890 0x7fc90c1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc90be00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c107ce0 0x7fc90c19df10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e9c0 tx=0x7fc8fc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc908c00640 1 -- 172.21.15.3:0/3272467366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00ccc0 con 0x7fc90c107ce0 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc908c00640 1 -- 172.21.15.3:0/3272467366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc8fc0035f0 con 0x7fc90c107ce0 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c1a5e20 con 0x7fc90c107ce0 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc908c00640 1 -- 172.21.15.3:0/3272467366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00ce60 con 0x7fc90c107ce0 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.403+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c1a6360 con 0x7fc90c107ce0 2024-03-21T08:28:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.404+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c10f000 con 0x7fc90c107ce0 2024-03-21T08:28:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.407+0000 7fc908c00640 1 -- 172.21.15.3:0/3272467366 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fc8fc0037a0 con 0x7fc90c107ce0 2024-03-21T08:28:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.408+0000 7fc908c00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc8f00a2fa0 0x7fc8f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.408+0000 7fc908c00640 1 -- 172.21.15.3:0/3272467366 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc8fc0cf410 con 0x7fc90c107ce0 2024-03-21T08:28:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.408+0000 7fc90b400640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc8f00a2fa0 0x7fc8f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.409+0000 7fc90b400640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc8f00a2fa0 0x7fc8f00a5460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc90c19f420 tx=0x7fc8f4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:29.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.411+0000 7fc908c00640 1 -- 172.21.15.3:0/3272467366 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc8fc09c770 con 0x7fc90c107ce0 2024-03-21T08:28:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:29 smithi003 ceph-mon[31675]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:29 smithi003 ceph-mon[31675]: Deploying daemon mds.b on smithi084 2024-03-21T08:28:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:29 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.3:6840/2348469157,v1:172.21.15.3:6841/2348469157] up:boot 2024-03-21T08:28:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:29 smithi003 ceph-mon[31675]: fsmap 11 up:standby 2024-03-21T08:28:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:28:29.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:29 smithi171 ceph-mon[35388]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:29.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:29 smithi171 ceph-mon[35388]: Deploying daemon mds.b on smithi084 2024-03-21T08:28:29.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:29 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.3:6840/2348469157,v1:172.21.15.3:6841/2348469157] up:boot 2024-03-21T08:28:29.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:29 smithi171 ceph-mon[35388]: fsmap 11 up:standby 2024-03-21T08:28:29.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:28:29.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.654+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc90c02d070 con 0x7fc90c107ce0 2024-03-21T08:28:29.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.655+0000 7fc908c00640 1 -- 172.21.15.3:0/3272467366 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9239 (secure 0 0 0) 0x7fc8fc09c070 con 0x7fc90c107ce0 2024-03-21T08:28:29.656 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:29.656 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T08:28:29.657 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 12 2024-03-21T08:28:29.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc8f00a2fa0 msgr2=0x7fc8f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:29.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc8f00a2fa0 0x7fc8f00a5460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc90c19f420 tx=0x7fc8f4009110 comp rx=0 tx=0).stop 2024-03-21T08:28:29.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c107ce0 msgr2=0x7fc90c19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:29.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c107ce0 0x7fc90c19df10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e9c0 tx=0x7fc8fc00edc0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 shutdown_connections 2024-03-21T08:28:29.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc8f00a2fa0 0x7fc8f00a5460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c109890 0x7fc90c1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc90c108ed0 0x7fc90c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 --2- 172.21.15.3:0/3272467366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc90c107ce0 0x7fc90c19df10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 >> 172.21.15.3:0/3272467366 conn(0x7fc90c076ca0 msgr2=0x7fc90c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 shutdown_connections 2024-03-21T08:28:29.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:29.658+0000 7fc912a00640 1 -- 172.21.15.3:0/3272467366 wait complete. 2024-03-21T08:28:30.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:30 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3272467366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:30.503 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:30 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3272467366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:30.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:30 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3272467366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:31.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:31 smithi003 ceph-mon[31675]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:31.568 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:31 smithi084 ceph-mon[35477]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:31.569 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:31 smithi084 systemd[1]: Starting Ceph mds.b for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:28:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:31 smithi171 ceph-mon[35388]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:31.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:31.876 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:31 smithi084 podman[65426]: 2024-03-21 08:28:31.472640782 +0000 UTC m=+0.036713059 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:28:31.877 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:31 smithi084 podman[65426]: 2024-03-21 08:28:31.622870136 +0000 UTC m=+0.186942461 container create 8f5c231aee30c77f823c39a23e14268ff23b21e2bb8f8e0be85c2a751c4d2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T08:28:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 -- 172.21.15.3:0/2901730590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 msgr2=0x7f04181019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 --2- 172.21.15.3:0/2901730590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f04181019e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f040c009900 tx=0x7f040c02bc40 comp rx=0 tx=0).stop 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 -- 172.21.15.3:0/2901730590 shutdown_connections 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 --2- 172.21.15.3:0/2901730590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0418103170 0x7f0418109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 --2- 172.21.15.3:0/2901730590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04181027b0 0x7f0418102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 --2- 172.21.15.3:0/2901730590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f04181019e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 -- 172.21.15.3:0/2901730590 >> 172.21.15.3:0/2901730590 conn(0x7f04180fcb30 msgr2=0x7f04180fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 -- 172.21.15.3:0/2901730590 shutdown_connections 2024-03-21T08:28:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.917+0000 7f041e400640 1 -- 172.21.15.3:0/2901730590 wait complete. 2024-03-21T08:28:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.918+0000 7f041e400640 1 Processor -- start 2024-03-21T08:28:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041e400640 1 -- start start 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041e400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f0418199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041e400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04181027b0 0x7f0418199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041e400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0418103170 0x7f04181a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041e400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041810d0c0 con 0x7f04181015c0 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041e400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041810cf40 con 0x7f04181027b0 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041e400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041810d240 con 0x7f0418103170 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f0418199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f0418199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50192/0 (socket says 172.21.15.3:50192) 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f041d000640 1 -- 172.21.15.3:0/1865998402 learned_addr learned my addr 172.21.15.3:0/1865998402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.919+0000 7f0417e00640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04181027b0 0x7f0418199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041da00640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0418103170 0x7f04181a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041d000640 1 -- 172.21.15.3:0/1865998402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0418103170 msgr2=0x7f04181a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041d000640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0418103170 0x7f04181a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041d000640 1 -- 172.21.15.3:0/1865998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04181027b0 msgr2=0x7f0418199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041d000640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04181027b0 0x7f0418199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041d000640 1 -- 172.21.15.3:0/1865998402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04181a15f0 con 0x7f04181015c0 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041d000640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f0418199a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f040c000c00 tx=0x7f040c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f0415600640 1 -- 172.21.15.3:0/1865998402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c043020 con 0x7f04181015c0 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f0415600640 1 -- 172.21.15.3:0/1865998402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f040c004aa0 con 0x7f04181015c0 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04181a1880 con 0x7f04181015c0 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.920+0000 7f0415600640 1 -- 172.21.15.3:0/1865998402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c003d00 con 0x7f04181015c0 2024-03-21T08:28:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.921+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04181a1e20 con 0x7f04181015c0 2024-03-21T08:28:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.922+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04181a2050 con 0x7f04181015c0 2024-03-21T08:28:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.924+0000 7f0415600640 1 -- 172.21.15.3:0/1865998402 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f040c046020 con 0x7f04181015c0 2024-03-21T08:28:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.925+0000 7f0415600640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04000a2f90 0x7f04000a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.925+0000 7f0415600640 1 -- 172.21.15.3:0/1865998402 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f040c0f1350 con 0x7f04181015c0 2024-03-21T08:28:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.925+0000 7f0417e00640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04000a2f90 0x7f04000a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:31.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.926+0000 7f0417e00640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04000a2f90 0x7f04000a5450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f041819aee0 tx=0x7f040400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:31.929+0000 7f0415600640 1 -- 172.21.15.3:0/1865998402 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f040c04d020 con 0x7f04181015c0 2024-03-21T08:28:32.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.174+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04181a2220 con 0x7f04181015c0 2024-03-21T08:28:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.176+0000 7f0415600640 1 -- 172.21.15.3:0/1865998402 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10031 (secure 0 0 0) 0x7f040c0be580 con 0x7f04181015c0 2024-03-21T08:28:32.177 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:32.177 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T08:28:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 13 2024-03-21T08:28:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.178+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04000a2f90 msgr2=0x7f04000a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.178+0000 7f041e400640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04000a2f90 0x7f04000a5450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f041819aee0 tx=0x7f040400b410 comp rx=0 tx=0).stop 2024-03-21T08:28:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.178+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 msgr2=0x7f0418199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.178+0000 7f041e400640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f0418199a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f040c000c00 tx=0x7f040c004270 comp rx=0 tx=0).stop 2024-03-21T08:28:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 shutdown_connections 2024-03-21T08:28:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04000a2f90 0x7f04000a5450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0418103170 0x7f04181a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04181027b0 0x7f0418199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 --2- 172.21.15.3:0/1865998402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04181015c0 0x7f0418199a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 >> 172.21.15.3:0/1865998402 conn(0x7f04180fcb30 msgr2=0x7f04180738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 shutdown_connections 2024-03-21T08:28:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.179+0000 7f041e400640 1 -- 172.21.15.3:0/1865998402 wait complete. 2024-03-21T08:28:32.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:28:32.376 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:31 smithi084 podman[65426]: 2024-03-21 08:28:31.877190714 +0000 UTC m=+0.441263026 container init 8f5c231aee30c77f823c39a23e14268ff23b21e2bb8f8e0be85c2a751c4d2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-21T08:28:32.376 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:31 smithi084 podman[65426]: 2024-03-21 08:28:31.887721575 +0000 UTC m=+0.451793877 container start 8f5c231aee30c77f823c39a23e14268ff23b21e2bb8f8e0be85c2a751c4d2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-21T08:28:32.376 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:31 smithi084 ceph-mds[65445]: -- 172.21.15.84:0/69233420 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a3c3a85a40 con 0x55a3c3a58800 2024-03-21T08:28:32.376 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b[65441]: starting mds.b at 2024-03-21T08:28:32.376 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:32 smithi084 bash[65426]: 8f5c231aee30c77f823c39a23e14268ff23b21e2bb8f8e0be85c2a751c4d2a1c 2024-03-21T08:28:32.376 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:28:32 smithi084 systemd[1]: Started Ceph mds.b for 544fc91e-e75b-11ee-95cd-87774f69a715. 2024-03-21T08:28:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 -- 172.21.15.3:0/2661326012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 msgr2=0x7fb9ec102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/2661326012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec102c30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009900 tx=0x7fb9d402bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 -- 172.21.15.3:0/2661326012 shutdown_connections 2024-03-21T08:28:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/2661326012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9ec103170 0x7fb9ec109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/2661326012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec102c30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/2661326012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9ec1015c0 0x7fb9ec1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 -- 172.21.15.3:0/2661326012 >> 172.21.15.3:0/2661326012 conn(0x7fb9ec0fcb30 msgr2=0x7fb9ec0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 -- 172.21.15.3:0/2661326012 shutdown_connections 2024-03-21T08:28:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.405+0000 7fb9ebe00640 1 -- 172.21.15.3:0/2661326012 wait complete. 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 Processor -- start 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 -- start start 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9ec1015c0 0x7fb9ec1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9ec103170 0x7fb9ec19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec10d0f0 con 0x7fb9ec1027b0 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec10cf70 con 0x7fb9ec103170 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.406+0000 7fb9ebe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec10d270 con 0x7fb9ec1015c0 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50204/0 (socket says 172.21.15.3:50204) 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 -- 172.21.15.3:0/4163354768 learned_addr learned my addr 172.21.15.3:0/4163354768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9eb400640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9ec103170 0x7fb9ec19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9eaa00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9ec1015c0 0x7fb9ec1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 -- 172.21.15.3:0/4163354768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9ec1015c0 msgr2=0x7fb9ec1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9ec1015c0 0x7fb9ec1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 -- 172.21.15.3:0/4163354768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9ec103170 msgr2=0x7fb9ec19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9ec103170 0x7fb9ec19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 -- 172.21.15.3:0/4163354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec19f410 con 0x7fb9ec1027b0 2024-03-21T08:28:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.407+0000 7fb9ea000640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec197d30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4006010 tx=0x7fb9d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.408+0000 7fb9e3400640 1 -- 172.21.15.3:0/4163354768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4043020 con 0x7fb9ec1027b0 2024-03-21T08:28:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.408+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec19f6a0 con 0x7fb9ec1027b0 2024-03-21T08:28:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.408+0000 7fb9e3400640 1 -- 172.21.15.3:0/4163354768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9d4004440 con 0x7fb9ec1027b0 2024-03-21T08:28:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.408+0000 7fb9e3400640 1 -- 172.21.15.3:0/4163354768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4003bf0 con 0x7fb9ec1027b0 2024-03-21T08:28:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.408+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec19fc40 con 0x7fb9ec1027b0 2024-03-21T08:28:32.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.409+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec1089b0 con 0x7fb9ec1027b0 2024-03-21T08:28:32.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.411+0000 7fb9e3400640 1 -- 172.21.15.3:0/4163354768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fb9d4046020 con 0x7fb9ec1027b0 2024-03-21T08:28:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.413+0000 7fb9e3400640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb9d00a2f10 0x7fb9d00a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.413+0000 7fb9e3400640 1 -- 172.21.15.3:0/4163354768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9d40f1a80 con 0x7fb9ec1027b0 2024-03-21T08:28:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.413+0000 7fb9eaa00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb9d00a2f10 0x7fb9d00a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.414+0000 7fb9eaa00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb9d00a2f10 0x7fb9d00a53d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc005c40 tx=0x7fb9dc005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.416+0000 7fb9e3400640 1 -- 172.21.15.3:0/4163354768 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb9d404d020 con 0x7fb9ec1027b0 2024-03-21T08:28:32.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.619+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9ec02d1f0 con 0x7fb9ec1027b0 2024-03-21T08:28:32.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.621+0000 7fb9e3400640 1 -- 172.21.15.3:0/4163354768 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+18363 (secure 0 0 0) 0x7fb9d403e450 con 0x7fb9ec1027b0 2024-03-21T08:28:32.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:32.627 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:27:30.128868+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb9d00a2f10 msgr2=0x7fb9d00a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb9d00a2f10 0x7fb9d00a53d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc005c40 tx=0x7fb9dc005bd0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 msgr2=0x7fb9ec197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec197d30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4006010 tx=0x7fb9d4004040 comp rx=0 tx=0).stop 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 shutdown_connections 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9ec103170 0x7fb9ec19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ec1027b0 0x7fb9ec197d30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb9d00a2f10 0x7fb9d00a53d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 --2- 172.21.15.3:0/4163354768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9ec1015c0 0x7fb9ec1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 >> 172.21.15.3:0/4163354768 conn(0x7fb9ec0fcb30 msgr2=0x7fb9ec105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 shutdown_connections 2024-03-21T08:28:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.624+0000 7fb9ebe00640 1 -- 172.21.15.3:0/4163354768 wait complete. 2024-03-21T08:28:32.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:28:32.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:28:32.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.833+0000 7f0448a00640 1 -- 172.21.15.3:0/2196501802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 msgr2=0x7f0444106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.833+0000 7f0448a00640 1 --2- 172.21.15.3:0/2196501802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444106c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f042c009900 tx=0x7f042c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.833+0000 7f0448a00640 1 -- 172.21.15.3:0/2196501802 shutdown_connections 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.833+0000 7f0448a00640 1 --2- 172.21.15.3:0/2196501802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04441071d0 0x7f0444109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.833+0000 7f0448a00640 1 --2- 172.21.15.3:0/2196501802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444106c90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.833+0000 7f0448a00640 1 --2- 172.21.15.3:0/2196501802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0444101f70 0x7f0444104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.833+0000 7f0448a00640 1 -- 172.21.15.3:0/2196501802 >> 172.21.15.3:0/2196501802 conn(0x7f04440fb980 msgr2=0x7f04440fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.834+0000 7f0448a00640 1 -- 172.21.15.3:0/2196501802 shutdown_connections 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.834+0000 7f0448a00640 1 -- 172.21.15.3:0/2196501802 wait complete. 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.834+0000 7f0448a00640 1 Processor -- start 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0448a00640 1 -- start start 2024-03-21T08:28:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0448a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0444101f70 0x7f0444197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0448a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0448a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04441071d0 0x7f044419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0448a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044410d030 con 0x7f04441048a0 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0448a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044410ceb0 con 0x7f04441071d0 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0448a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044410d1b0 con 0x7f0444101f70 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f043be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f043be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50228/0 (socket says 172.21.15.3:50228) 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f043be00640 1 -- 172.21.15.3:0/480406202 learned_addr learned my addr 172.21.15.3:0/480406202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0442800640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0444101f70 0x7f0444197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.835+0000 7f0443200640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04441071d0 0x7f044419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043be00640 1 -- 172.21.15.3:0/480406202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0444101f70 msgr2=0x7f0444197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043be00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0444101f70 0x7f0444197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043be00640 1 -- 172.21.15.3:0/480406202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04441071d0 msgr2=0x7f044419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043be00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04441071d0 0x7f044419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043be00640 1 -- 172.21.15.3:0/480406202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044419f370 con 0x7f04441048a0 2024-03-21T08:28:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f0442800640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0444101f70 0x7f0444197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043be00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444197c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f042c000c00 tx=0x7f042c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043b400640 1 -- 172.21.15.3:0/480406202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c043020 con 0x7f04441048a0 2024-03-21T08:28:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043b400640 1 -- 172.21.15.3:0/480406202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f042c004440 con 0x7f04441048a0 2024-03-21T08:28:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044419f600 con 0x7f04441048a0 2024-03-21T08:28:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.836+0000 7f043b400640 1 -- 172.21.15.3:0/480406202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c003c90 con 0x7f04441048a0 2024-03-21T08:28:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.837+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044419fba0 con 0x7f04441048a0 2024-03-21T08:28:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.838+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04440fd470 con 0x7f04441048a0 2024-03-21T08:28:32.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.840+0000 7f043b400640 1 -- 172.21.15.3:0/480406202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f042c046020 con 0x7f04441048a0 2024-03-21T08:28:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.841+0000 7f043b400640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04240a2fa0 0x7f04240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.841+0000 7f043b400640 1 -- 172.21.15.3:0/480406202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f042c0f12d0 con 0x7f04441048a0 2024-03-21T08:28:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.841+0000 7f0442800640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04240a2fa0 0x7f04240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.842+0000 7f0442800640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04240a2fa0 0x7f04240a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04340058c0 tx=0x7f0434005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:32.845+0000 7f043b400640 1 -- 172.21.15.3:0/480406202 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f042c04d020 con 0x7f04441048a0 2024-03-21T08:28:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.048+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04440622a0 con 0x7f04441048a0 2024-03-21T08:28:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.049+0000 7f043b400640 1 -- 172.21.15.3:0/480406202 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f042c03c350 con 0x7f04441048a0 2024-03-21T08:28:33.050 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04240a2fa0 msgr2=0x7f04240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04240a2fa0 0x7f04240a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04340058c0 tx=0x7f0434005850 comp rx=0 tx=0).stop 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 msgr2=0x7f0444197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444197c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f042c000c00 tx=0x7f042c004040 comp rx=0 tx=0).stop 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 shutdown_connections 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04441071d0 0x7f044419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04441048a0 0x7f0444197c90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04240a2fa0 0x7f04240a5460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.051+0000 7f0448a00640 1 --2- 172.21.15.3:0/480406202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0444101f70 0x7f0444197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.052+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 >> 172.21.15.3:0/480406202 conn(0x7f04440fb980 msgr2=0x7f0444102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.052+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 shutdown_connections 2024-03-21T08:28:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.052+0000 7f0448a00640 1 -- 172.21.15.3:0/480406202 wait complete. 2024-03-21T08:28:33.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-21T08:28:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 -- 172.21.15.3:0/2494301137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a20104af0 msgr2=0x7f8a20106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 --2- 172.21.15.3:0/2494301137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a20104af0 0x7f8a20106ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009900 tx=0x7f8a0802bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 -- 172.21.15.3:0/2494301137 shutdown_connections 2024-03-21T08:28:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 --2- 172.21.15.3:0/2494301137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a20107420 0x7f8a20109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 --2- 172.21.15.3:0/2494301137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a20104af0 0x7f8a20106ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 --2- 172.21.15.3:0/2494301137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a201021c0 0x7f8a201045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 -- 172.21.15.3:0/2494301137 >> 172.21.15.3:0/2494301137 conn(0x7f8a200fbbb0 msgr2=0x7f8a200fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:33.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 -- 172.21.15.3:0/2494301137 shutdown_connections 2024-03-21T08:28:33.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.283+0000 7f8a25200640 1 -- 172.21.15.3:0/2494301137 wait complete. 2024-03-21T08:28:33.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.284+0000 7f8a25200640 1 Processor -- start 2024-03-21T08:28:33.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.284+0000 7f8a25200640 1 -- start start 2024-03-21T08:28:33.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a25200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a201021c0 0x7f8a20196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a25200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a20104af0 0x7f8a201967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a25200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a20107420 0x7f8a20192e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a25200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2010d240 con 0x7f8a201021c0 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a25200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2010d0c0 con 0x7f8a20104af0 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a25200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2010d3c0 con 0x7f8a20107420 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a1de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a201021c0 0x7f8a20196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a1de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a201021c0 0x7f8a20196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50248/0 (socket says 172.21.15.3:50248) 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a1de00640 1 -- 172.21.15.3:0/1494322648 learned_addr learned my addr 172.21.15.3:0/1494322648 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a15400640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a20104af0 0x7f8a201967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.285+0000 7f8a1e800640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a20107420 0x7f8a20192e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a1de00640 1 -- 172.21.15.3:0/1494322648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a20107420 msgr2=0x7f8a20192e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a1de00640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a20107420 0x7f8a20192e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a1de00640 1 -- 172.21.15.3:0/1494322648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a20104af0 msgr2=0x7f8a201967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a1de00640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a20104af0 0x7f8a201967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a1de00640 1 -- 172.21.15.3:0/1494322648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a201936c0 con 0x7f8a201021c0 2024-03-21T08:28:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a1de00640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a201021c0 0x7f8a20196290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000ebc0 tx=0x7f8a1000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a17400640 1 -- 172.21.15.3:0/1494322648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10010630 con 0x7f8a201021c0 2024-03-21T08:28:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a17400640 1 -- 172.21.15.3:0/1494322648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a10010c80 con 0x7f8a201021c0 2024-03-21T08:28:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a201a5f80 con 0x7f8a201021c0 2024-03-21T08:28:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a17400640 1 -- 172.21.15.3:0/1494322648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10018b30 con 0x7f8a201021c0 2024-03-21T08:28:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.286+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a201a6330 con 0x7f8a201021c0 2024-03-21T08:28:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.288+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a20068970 con 0x7f8a201021c0 2024-03-21T08:28:33.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.290+0000 7f8a17400640 1 -- 172.21.15.3:0/1494322648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8a10018cd0 con 0x7f8a201021c0 2024-03-21T08:28:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.291+0000 7f8a17400640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8a040a2fa0 0x7f8a040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.292+0000 7f8a17400640 1 -- 172.21.15.3:0/1494322648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a100cf6a0 con 0x7f8a201021c0 2024-03-21T08:28:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.292+0000 7f8a15400640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8a040a2fa0 0x7f8a040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.293+0000 7f8a15400640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8a040a2fa0 0x7f8a040a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802c240 tx=0x7f8a08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.296+0000 7f8a17400640 1 -- 172.21.15.3:0/1494322648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8a1009c8d0 con 0x7f8a201021c0 2024-03-21T08:28:33.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:33.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6838/1820502310,v1:172.21.15.84:6839/1820502310] up:boot 2024-03-21T08:28:33.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: fsmap 12 up:standby 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1865998402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4163354768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:33 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/480406202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:33.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.509+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8a200622a0 con 0x7f8a201021c0 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.510+0000 7f8a17400640 1 -- 172.21.15.3:0/1494322648 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8a1009c1d0 con 0x7f8a201021c0 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:28:33.511 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-03-21T08:28:33.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8a040a2fa0 msgr2=0x7f8a040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8a040a2fa0 0x7f8a040a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802c240 tx=0x7f8a08005fd0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a201021c0 msgr2=0x7f8a20196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a201021c0 0x7f8a20196290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000ebc0 tx=0x7f8a1000c6a0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 shutdown_connections 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8a040a2fa0 0x7f8a040a5460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a20107420 0x7f8a20192e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a20104af0 0x7f8a201967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 --2- 172.21.15.3:0/1494322648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a201021c0 0x7f8a20196290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 >> 172.21.15.3:0/1494322648 conn(0x7f8a200fbbb0 msgr2=0x7f8a201019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:33.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 shutdown_connections 2024-03-21T08:28:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.513+0000 7f8a25200640 1 -- 172.21.15.3:0/1494322648 wait complete. 2024-03-21T08:28:33.535 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-21T08:28:33.535 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6838/1820502310,v1:172.21.15.84:6839/1820502310] up:boot 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: fsmap 12 up:standby 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1865998402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4163354768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:33 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/480406202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6838/1820502310,v1:172.21.15.84:6839/1820502310] up:boot 2024-03-21T08:28:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: fsmap 12 up:standby 2024-03-21T08:28:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1865998402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4163354768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:28:33.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:33 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/480406202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 -- 172.21.15.3:0/1256667942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 msgr2=0x7fd42c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 --2- 172.21.15.3:0/1256667942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c106c90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd418009900 tx=0x7fd41802bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 -- 172.21.15.3:0/1256667942 shutdown_connections 2024-03-21T08:28:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 --2- 172.21.15.3:0/1256667942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd42c1071d0 0x7fd42c109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 --2- 172.21.15.3:0/1256667942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c106c90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 --2- 172.21.15.3:0/1256667942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd42c101f70 0x7fd42c104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 -- 172.21.15.3:0/1256667942 >> 172.21.15.3:0/1256667942 conn(0x7fd42c0fb980 msgr2=0x7fd42c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 -- 172.21.15.3:0/1256667942 shutdown_connections 2024-03-21T08:28:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.742+0000 7fd433000640 1 -- 172.21.15.3:0/1256667942 wait complete. 2024-03-21T08:28:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.743+0000 7fd433000640 1 Processor -- start 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd433000640 1 -- start start 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd433000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd42c101f70 0x7fd42c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd433000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd433000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd42c1071d0 0x7fd42c1a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd433000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c10d0c0 con 0x7fd42c1048a0 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd433000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c10cf40 con 0x7fd42c101f70 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd433000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c10d240 con 0x7fd42c1071d0 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd431200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd431200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50264/0 (socket says 172.21.15.3:50264) 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.744+0000 7fd431200640 1 -- 172.21.15.3:0/1922901498 learned_addr learned my addr 172.21.15.3:0/1922901498 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431c00640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd42c101f70 0x7fd42c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd432600640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd42c1071d0 0x7fd42c1a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431200640 1 -- 172.21.15.3:0/1922901498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd42c1071d0 msgr2=0x7fd42c1a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431200640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd42c1071d0 0x7fd42c1a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431200640 1 -- 172.21.15.3:0/1922901498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd42c101f70 msgr2=0x7fd42c1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431200640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd42c101f70 0x7fd42c1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431200640 1 -- 172.21.15.3:0/1922901498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c1a15d0 con 0x7fd42c1048a0 2024-03-21T08:28:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431c00640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd42c101f70 0x7fd42c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd432600640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd42c1071d0 0x7fd42c1a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:33.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.745+0000 7fd431200640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c199ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd418005ab0 tx=0x7fd4180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:33.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.746+0000 7fd42a000640 1 -- 172.21.15.3:0/1922901498 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418043020 con 0x7fd42c1048a0 2024-03-21T08:28:33.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.746+0000 7fd42a000640 1 -- 172.21.15.3:0/1922901498 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd4180054c0 con 0x7fd42c1048a0 2024-03-21T08:28:33.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.746+0000 7fd42a000640 1 -- 172.21.15.3:0/1922901498 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418003d10 con 0x7fd42c1048a0 2024-03-21T08:28:33.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.746+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c1a1860 con 0x7fd42c1048a0 2024-03-21T08:28:33.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.746+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c1a1e80 con 0x7fd42c1048a0 2024-03-21T08:28:33.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.747+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c04f400 con 0x7fd42c1048a0 2024-03-21T08:28:33.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.750+0000 7fd42a000640 1 -- 172.21.15.3:0/1922901498 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd418046020 con 0x7fd42c1048a0 2024-03-21T08:28:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.751+0000 7fd42a000640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4100a2fa0 0x7fd4100a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.751+0000 7fd42a000640 1 -- 172.21.15.3:0/1922901498 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd4180f1380 con 0x7fd42c1048a0 2024-03-21T08:28:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.751+0000 7fd431c00640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4100a2fa0 0x7fd4100a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:33.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.752+0000 7fd431c00640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4100a2fa0 0x7fd4100a5460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0058c0 tx=0x7fd41c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:33.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.756+0000 7fd42a000640 1 -- 172.21.15.3:0/1922901498 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd41804d020 con 0x7fd42c1048a0 2024-03-21T08:28:33.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:33.971+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd42c0622a0 con 0x7fd42c1048a0 2024-03-21T08:28:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.596+0000 7fd42a000640 1 -- 172.21.15.3:0/1922901498 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd4180be6d0 con 0x7fd42c1048a0 2024-03-21T08:28:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-03-21T08:28:34.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.598+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4100a2fa0 msgr2=0x7fd4100a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.598+0000 7fd433000640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4100a2fa0 0x7fd4100a5460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0058c0 tx=0x7fd41c005850 comp rx=0 tx=0).stop 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.598+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 msgr2=0x7fd42c199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.598+0000 7fd433000640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c199ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd418005ab0 tx=0x7fd4180042f0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 shutdown_connections 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4100a2fa0 0x7fd4100a5460 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd42c1071d0 0x7fd42c1a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42c1048a0 0x7fd42c199ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 --2- 172.21.15.3:0/1922901498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd42c101f70 0x7fd42c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 >> 172.21.15.3:0/1922901498 conn(0x7fd42c0fb980 msgr2=0x7fd42c101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 shutdown_connections 2024-03-21T08:28:34.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.599+0000 7fd433000640 1 -- 172.21.15.3:0/1922901498 wait complete. 2024-03-21T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:34 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1494322648' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T08:28:34.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-21T08:28:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 -- 172.21.15.3:0/1742910036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 msgr2=0x7f82a0101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 --2- 172.21.15.3:0/1742910036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0101ac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8294009900 tx=0x7f829402bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 -- 172.21.15.3:0/1742910036 shutdown_connections 2024-03-21T08:28:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 --2- 172.21.15.3:0/1742910036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a0103250 0x7f82a0109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 --2- 172.21.15.3:0/1742910036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a0102890 0x7f82a0102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 --2- 172.21.15.3:0/1742910036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0101ac0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 -- 172.21.15.3:0/1742910036 >> 172.21.15.3:0/1742910036 conn(0x7f82a00fcbf0 msgr2=0x7f82a00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 -- 172.21.15.3:0/1742910036 shutdown_connections 2024-03-21T08:28:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.829+0000 7f82a7a00640 1 -- 172.21.15.3:0/1742910036 wait complete. 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 Processor -- start 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 -- start start 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a0102890 0x7f82a019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a0103250 0x7f82a01a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a010d1d0 con 0x7f82a01016a0 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a010d050 con 0x7f82a0102890 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.830+0000 7f82a7a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a010d350 con 0x7f82a0103250 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50282/0 (socket says 172.21.15.3:50282) 2024-03-21T08:28:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 -- 172.21.15.3:0/3860158377 learned_addr learned my addr 172.21.15.3:0/3860158377 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5a00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a0103250 0x7f82a01a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f829fe00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a0102890 0x7f82a019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 -- 172.21.15.3:0/3860158377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a0103250 msgr2=0x7f82a01a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a0103250 0x7f82a01a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 -- 172.21.15.3:0/3860158377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a0102890 msgr2=0x7f82a019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a0102890 0x7f82a019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 -- 172.21.15.3:0/3860158377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a01a17b0 con 0x7f82a01016a0 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f829fe00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a0102890 0x7f82a019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.831+0000 7f82a5000640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0199b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f8294004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.832+0000 7f829d600640 1 -- 172.21.15.3:0/3860158377 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294043020 con 0x7f82a01016a0 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.832+0000 7f829d600640 1 -- 172.21.15.3:0/3860158377 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8294004440 con 0x7f82a01016a0 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.832+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a01a1a40 con 0x7f82a01016a0 2024-03-21T08:28:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.832+0000 7f829d600640 1 -- 172.21.15.3:0/3860158377 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294003c90 con 0x7f82a01016a0 2024-03-21T08:28:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.832+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a01a1f90 con 0x7f82a01016a0 2024-03-21T08:28:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.833+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a01a2160 con 0x7f82a01016a0 2024-03-21T08:28:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.836+0000 7f829d600640 1 -- 172.21.15.3:0/3860158377 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8294046020 con 0x7f82a01016a0 2024-03-21T08:28:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.837+0000 7f829d600640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82880a2f20 0x7f82880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.837+0000 7f829d600640 1 -- 172.21.15.3:0/3860158377 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f82940f17f0 con 0x7f82a01016a0 2024-03-21T08:28:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.837+0000 7f829fe00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82880a2f20 0x7f82880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.838+0000 7f829fe00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82880a2f20 0x7f82880a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82a019b0a0 tx=0x7f828c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:34.840+0000 7f829d600640 1 -- 172.21.15.3:0/3860158377 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f829404d020 con 0x7f82a01016a0 2024-03-21T08:28:34.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:34 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1494322648' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T08:28:34.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:34 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1494322648' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T08:28:35.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.060+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f82a01a2330 con 0x7f82a01016a0 2024-03-21T08:28:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.638+0000 7f829d600640 1 -- 172.21.15.3:0/3860158377 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) v1 ==== 171+0+0 (secure 0 0 0) 0x7f82940be9d0 con 0x7f82a01016a0 2024-03-21T08:28:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-03-21T08:28:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82880a2f20 msgr2=0x7f82880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82880a2f20 0x7f82880a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82a019b0a0 tx=0x7f828c005f90 comp rx=0 tx=0).stop 2024-03-21T08:28:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 msgr2=0x7f82a0199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0199b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f8294004040 comp rx=0 tx=0).stop 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 shutdown_connections 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82880a2f20 0x7f82880a53e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a0103250 0x7f82a01a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82a0102890 0x7f82a019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 --2- 172.21.15.3:0/3860158377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a01016a0 0x7f82a0199b90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.641+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 >> 172.21.15.3:0/3860158377 conn(0x7f82a00fcbf0 msgr2=0x7f82a00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.642+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 shutdown_connections 2024-03-21T08:28:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.642+0000 7f82a7a00640 1 -- 172.21.15.3:0/3860158377 wait complete. 2024-03-21T08:28:35.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 -- 172.21.15.3:0/2430377359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914104ac0 msgr2=0x7f8914104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 --2- 172.21.15.3:0/2430377359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914104ac0 0x7f8914104f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8900009900 tx=0x7f890002bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 -- 172.21.15.3:0/2430377359 shutdown_connections 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 --2- 172.21.15.3:0/2430377359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8914105480 0x7f891410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 --2- 172.21.15.3:0/2430377359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914104ac0 0x7f8914104f40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 --2- 172.21.15.3:0/2430377359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89141038d0 0x7f8914103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 -- 172.21.15.3:0/2430377359 >> 172.21.15.3:0/2430377359 conn(0x7f89140fee60 msgr2=0x7f8914101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 -- 172.21.15.3:0/2430377359 shutdown_connections 2024-03-21T08:28:35.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.857+0000 7f891b600640 1 -- 172.21.15.3:0/2430377359 wait complete. 2024-03-21T08:28:35.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 Processor -- start 2024-03-21T08:28:35.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 -- start start 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89141038d0 0x7f8914199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8914104ac0 0x7f891419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914105480 0x7f89141a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891410f450 con 0x7f8914105480 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891410f2d0 con 0x7f89141038d0 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f891b600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891410f5d0 con 0x7f8914104ac0 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f8919600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914105480 0x7f89141a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f8918c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89141038d0 0x7f8914199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f8919600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914105480 0x7f89141a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50308/0 (socket says 172.21.15.3:50308) 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f8919600640 1 -- 172.21.15.3:0/1770029963 learned_addr learned my addr 172.21.15.3:0/1770029963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.858+0000 7f8913e00640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8914104ac0 0x7f891419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8919600640 1 -- 172.21.15.3:0/1770029963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8914104ac0 msgr2=0x7f891419a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8919600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8914104ac0 0x7f891419a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8919600640 1 -- 172.21.15.3:0/1770029963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89141038d0 msgr2=0x7f8914199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8919600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89141038d0 0x7f8914199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8919600640 1 -- 172.21.15.3:0/1770029963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89141a1740 con 0x7f8914105480 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8919600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914105480 0x7f89141a10d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f890c00eaa0 tx=0x7f890c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8911600640 1 -- 172.21.15.3:0/1770029963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00cdb0 con 0x7f8914105480 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f8911600640 1 -- 172.21.15.3:0/1770029963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f890c005020 con 0x7f8914105480 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.859+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89141a1a30 con 0x7f8914105480 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.860+0000 7f8911600640 1 -- 172.21.15.3:0/1770029963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c0116d0 con 0x7f8914105480 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.860+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89141a1f00 con 0x7f8914105480 2024-03-21T08:28:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.861+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891410ac70 con 0x7f8914105480 2024-03-21T08:28:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.863+0000 7f8911600640 1 -- 172.21.15.3:0/1770029963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f890c0051d0 con 0x7f8914105480 2024-03-21T08:28:35.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.864+0000 7f8911600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88fc0a2f20 0x7f88fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:35.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.864+0000 7f8911600640 1 -- 172.21.15.3:0/1770029963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f890c0cf760 con 0x7f8914105480 2024-03-21T08:28:35.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.865+0000 7f8918c00640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88fc0a2f20 0x7f88fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:35.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.865+0000 7f8918c00640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88fc0a2f20 0x7f88fc0a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8914104920 tx=0x7f8908005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:35.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:35.872+0000 7f8911600640 1 -- 172.21.15.3:0/1770029963 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f890c016030 con 0x7f8914105480 2024-03-21T08:28:35.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:35 smithi171 ceph-mon[35388]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:35.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:35 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1922901498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T08:28:35.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:35 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1922901498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T08:28:35.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:35 smithi171 ceph-mon[35388]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T08:28:35.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:35 smithi003 ceph-mon[31675]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:35.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:35 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1922901498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T08:28:35.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:35 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1922901498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T08:28:35.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:35 smithi003 ceph-mon[31675]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T08:28:36.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.112+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f891402d070 con 0x7f8914105480 2024-03-21T08:28:36.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:35 smithi084 ceph-mon[35477]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:36.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:35 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1922901498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T08:28:36.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:35 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1922901498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T08:28:36.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:35 smithi084 ceph-mon[35477]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T08:28:36.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.705+0000 7f8911600640 1 -- 172.21.15.3:0/1770029963 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f890c09c050 con 0x7f8914105480 2024-03-21T08:28:36.706 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88fc0a2f20 msgr2=0x7f88fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88fc0a2f20 0x7f88fc0a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8914104920 tx=0x7f8908005f90 comp rx=0 tx=0).stop 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914105480 msgr2=0x7f89141a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914105480 0x7f89141a10d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f890c00eaa0 tx=0x7f890c00ef60 comp rx=0 tx=0).stop 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 shutdown_connections 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8914105480 0x7f89141a10d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f88fc0a2f20 0x7f88fc0a53e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8914104ac0 0x7f891419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 --2- 172.21.15.3:0/1770029963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89141038d0 0x7f8914199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 >> 172.21.15.3:0/1770029963 conn(0x7f89140fee60 msgr2=0x7f8914107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:36.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 shutdown_connections 2024-03-21T08:28:36.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.708+0000 7f891b600640 1 -- 172.21.15.3:0/1770029963 wait complete. 2024-03-21T08:28:36.731 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-21T08:28:36.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-21T08:28:36.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3860158377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T08:28:36.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: pgmap v360: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3860158377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1770029963' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T08:28:36.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:36 smithi171 ceph-mon[35388]: daemon mds.h assigned to filesystem cephfs as rank 0 2024-03-21T08:28:36.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3860158377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T08:28:36.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: pgmap v360: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3860158377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1770029963' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T08:28:36.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:36 smithi003 ceph-mon[31675]: daemon mds.h assigned to filesystem cephfs as rank 0 2024-03-21T08:28:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.928+0000 7f9593a00640 1 -- 172.21.15.3:0/2141363549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 msgr2=0x7f958c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.928+0000 7f9593a00640 1 --2- 172.21.15.3:0/2141363549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c109af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9584009ca0 tx=0x7f958402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.929+0000 7f9593a00640 1 -- 172.21.15.3:0/2141363549 shutdown_connections 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.929+0000 7f9593a00640 1 --2- 172.21.15.3:0/2141363549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c109af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.929+0000 7f9593a00640 1 --2- 172.21.15.3:0/2141363549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f958c1028a0 0x7f958c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.929+0000 7f9593a00640 1 --2- 172.21.15.3:0/2141363549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f958c1016b0 0x7f958c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.929+0000 7f9593a00640 1 -- 172.21.15.3:0/2141363549 >> 172.21.15.3:0/2141363549 conn(0x7f958c0fcc20 msgr2=0x7f958c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.929+0000 7f9593a00640 1 -- 172.21.15.3:0/2141363549 shutdown_connections 2024-03-21T08:28:36.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.929+0000 7f9593a00640 1 -- 172.21.15.3:0/2141363549 wait complete. 2024-03-21T08:28:36.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 Processor -- start 2024-03-21T08:28:36.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 -- start start 2024-03-21T08:28:36.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f958c1016b0 0x7f958c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f958c1028a0 0x7f958c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c10d1e0 con 0x7f958c103260 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c10d060 con 0x7f958c1028a0 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.930+0000 7f9593a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c10d360 con 0x7f958c1016b0 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c1a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50320/0 (socket says 172.21.15.3:50320) 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 -- 172.21.15.3:0/2388063795 learned_addr learned my addr 172.21.15.3:0/2388063795 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591000640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f958c1016b0 0x7f958c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f958be00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f958c1028a0 0x7f958c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 -- 172.21.15.3:0/2388063795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f958c1016b0 msgr2=0x7f958c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f958c1016b0 0x7f958c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 -- 172.21.15.3:0/2388063795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f958c1028a0 msgr2=0x7f958c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f958c1028a0 0x7f958c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 -- 172.21.15.3:0/2388063795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c1a1710 con 0x7f958c103260 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f958be00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f958c1028a0 0x7f958c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.931+0000 7f9591a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c1a10a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9584032380 tx=0x7f9584032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.932+0000 7f9589600640 1 -- 172.21.15.3:0/2388063795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584041020 con 0x7f958c103260 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.932+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c1a19a0 con 0x7f958c103260 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.932+0000 7f9589600640 1 -- 172.21.15.3:0/2388063795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9584003e20 con 0x7f958c103260 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.932+0000 7f9589600640 1 -- 172.21.15.3:0/2388063795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584039040 con 0x7f958c103260 2024-03-21T08:28:36.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.932+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c1a1ee0 con 0x7f958c103260 2024-03-21T08:28:36.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.933+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c1a21d0 con 0x7f958c103260 2024-03-21T08:28:36.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.935+0000 7f9589600640 1 -- 172.21.15.3:0/2388063795 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f9584044020 con 0x7f958c103260 2024-03-21T08:28:36.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.936+0000 7f9589600640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f95740a2fa0 0x7f95740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:36.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.937+0000 7f9589600640 1 -- 172.21.15.3:0/2388063795 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f95840f17a0 con 0x7f958c103260 2024-03-21T08:28:36.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.937+0000 7f9591000640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f95740a2fa0 0x7f95740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:36.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.937+0000 7f9591000640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f95740a2fa0 0x7f95740a5460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f958c102700 tx=0x7f9580005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:36.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:36.939+0000 7f9589600640 1 -- 172.21.15.3:0/2388063795 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f958404b020 con 0x7f958c103260 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3860158377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: pgmap v360: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3860158377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1770029963' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T08:28:37.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:36 smithi084 ceph-mon[35477]: daemon mds.h assigned to filesystem cephfs as rank 0 2024-03-21T08:28:37.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.153+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f958c1a23a0 con 0x7f958c103260 2024-03-21T08:28:37.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.632+0000 7f9589600640 1 -- 172.21.15.3:0/2388063795 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v88) v1 ==== 157+0+0 (secure 0 0 0) 0x7f95840333a0 con 0x7f958c103260 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f95740a2fa0 msgr2=0x7f95740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f95740a2fa0 0x7f95740a5460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f958c102700 tx=0x7f9580005f90 comp rx=0 tx=0).stop 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 msgr2=0x7f958c1a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c1a10a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9584032380 tx=0x7f9584032820 comp rx=0 tx=0).stop 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 shutdown_connections 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c103260 0x7f958c1a10a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f958c1028a0 0x7f958c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f95740a2fa0 0x7f95740a5460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 --2- 172.21.15.3:0/2388063795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f958c1016b0 0x7f958c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.636+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 >> 172.21.15.3:0/2388063795 conn(0x7f958c0fcc20 msgr2=0x7f958c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:37.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.637+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 shutdown_connections 2024-03-21T08:28:37.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.637+0000 7f9593a00640 1 -- 172.21.15.3:0/2388063795 wait complete. 2024-03-21T08:28:37.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-21T08:28:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 -- 172.21.15.3:0/3130913409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c1052b0 msgr2=0x7f169c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 --2- 172.21.15.3:0/3130913409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c1052b0 0x7f169c10bb40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1694009ca0 tx=0x7f169402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 -- 172.21.15.3:0/3130913409 shutdown_connections 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 --2- 172.21.15.3:0/3130913409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c1052b0 0x7f169c10bb40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 --2- 172.21.15.3:0/3130913409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f169c1048f0 0x7f169c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 --2- 172.21.15.3:0/3130913409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f169c103700 0x7f169c103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 -- 172.21.15.3:0/3130913409 >> 172.21.15.3:0/3130913409 conn(0x7f169c0fecb0 msgr2=0x7f169c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 -- 172.21.15.3:0/3130913409 shutdown_connections 2024-03-21T08:28:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.883+0000 7f169be00640 1 -- 172.21.15.3:0/3130913409 wait complete. 2024-03-21T08:28:37.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.885+0000 7f169be00640 1 Processor -- start 2024-03-21T08:28:37.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.885+0000 7f169be00640 1 -- start start 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.885+0000 7f169be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c103700 0x7f169c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f169c1048f0 0x7f169c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f169c1052b0 0x7f169c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c10f230 con 0x7f169c103700 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c10f0b0 con 0x7f169c1048f0 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c10f3b0 con 0x7f169c1052b0 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c103700 0x7f169c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c103700 0x7f169c199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50344/0 (socket says 172.21.15.3:50344) 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 -- 172.21.15.3:0/3947321935 learned_addr learned my addr 172.21.15.3:0/3947321935 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169a000640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f169c1048f0 0x7f169c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169b400640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f169c1052b0 0x7f169c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 -- 172.21.15.3:0/3947321935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f169c1052b0 msgr2=0x7f169c1a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f169c1052b0 0x7f169c1a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 -- 172.21.15.3:0/3947321935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f169c1048f0 msgr2=0x7f169c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f169c1048f0 0x7f169c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169aa00640 1 -- 172.21.15.3:0/3947321935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c1a1490 con 0x7f169c103700 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.886+0000 7f169b400640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f169c1052b0 0x7f169c1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.887+0000 7f169aa00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c103700 0x7f169c199870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d950 tx=0x7f168c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.887+0000 7f1693400640 1 -- 172.21.15.3:0/3947321935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c017070 con 0x7f169c103700 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.887+0000 7f1693400640 1 -- 172.21.15.3:0/3947321935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f168c009c30 con 0x7f169c103700 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.887+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c1a1780 con 0x7f169c103700 2024-03-21T08:28:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.887+0000 7f1693400640 1 -- 172.21.15.3:0/3947321935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c00f620 con 0x7f169c103700 2024-03-21T08:28:37.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.887+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c1a1c80 con 0x7f169c103700 2024-03-21T08:28:37.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.888+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c10aa20 con 0x7f169c103700 2024-03-21T08:28:37.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.891+0000 7f1693400640 1 -- 172.21.15.3:0/3947321935 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f168c01a070 con 0x7f169c103700 2024-03-21T08:28:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.892+0000 7f1693400640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f16800a2f10 0x7f16800a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.892+0000 7f1693400640 1 -- 172.21.15.3:0/3947321935 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f168c0cefc0 con 0x7f169c103700 2024-03-21T08:28:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.892+0000 7f169a000640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f16800a2f10 0x7f16800a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.893+0000 7f169a000640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f16800a2f10 0x7f16800a53d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1688005fd0 tx=0x7f1688005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:37.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:37.900+0000 7f1693400640 1 -- 172.21.15.3:0/3947321935 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f168c009de0 con 0x7f169c103700 2024-03-21T08:28:37.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1770029963' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: fsmap cephfs:1 {0=h=up:creating} 11 up:standby 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: daemon mds.h is now active in filesystem cephfs as rank 0 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2388063795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] up:active 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:37 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:28:38.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:38.104+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f169c02d070 con 0x7f169c103700 2024-03-21T08:28:38.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T08:28:38.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1770029963' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T08:28:38.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: fsmap cephfs:1 {0=h=up:creating} 11 up:standby 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: daemon mds.h is now active in filesystem cephfs as rank 0 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2388063795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] up:active 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:28:38.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:37 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:28:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T08:28:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1770029963' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T08:28:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: fsmap cephfs:1 {0=h=up:creating} 11 up:standby 2024-03-21T08:28:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: daemon mds.h is now active in filesystem cephfs as rank 0 2024-03-21T08:28:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:28:38.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:38.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2388063795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T08:28:38.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] up:active 2024-03-21T08:28:38.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:28:38.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:37 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:28:39.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:39 smithi003 ceph-mon[31675]: pgmap v363: 97 pgs: 23 creating+peering, 6 active+clean, 68 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:39.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:39 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2388063795' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T08:28:39.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:39 smithi003 ceph-mon[31675]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T08:28:39.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:39 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3947321935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T08:28:39.597 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:39 smithi003 ceph-mon[31675]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T08:28:39.597 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:39 smithi003 ceph-mon[31675]: Cluster is now healthy 2024-03-21T08:28:39.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:39 smithi084 ceph-mon[35477]: pgmap v363: 97 pgs: 23 creating+peering, 6 active+clean, 68 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:39.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:39 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2388063795' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T08:28:39.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:39 smithi084 ceph-mon[35477]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T08:28:39.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:39 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3947321935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T08:28:39.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:39 smithi084 ceph-mon[35477]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T08:28:39.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:39 smithi084 ceph-mon[35477]: Cluster is now healthy 2024-03-21T08:28:39.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:39 smithi171 ceph-mon[35388]: pgmap v363: 97 pgs: 23 creating+peering, 6 active+clean, 68 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:39.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:39 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2388063795' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T08:28:39.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:39 smithi171 ceph-mon[35388]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T08:28:39.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:39 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3947321935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T08:28:39.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:39 smithi171 ceph-mon[35388]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T08:28:39.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:39 smithi171 ceph-mon[35388]: Cluster is now healthy 2024-03-21T08:28:39.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.649+0000 7f1693400640 1 -- 172.21.15.3:0/3947321935 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v90) v1 ==== 228+0+0 (secure 0 0 0) 0x7f168c09bd60 con 0x7f169c103700 2024-03-21T08:28:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' created 2024-03-21T08:28:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f16800a2f10 msgr2=0x7f16800a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f16800a2f10 0x7f16800a53d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1688005fd0 tx=0x7f1688005850 comp rx=0 tx=0).stop 2024-03-21T08:28:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c103700 msgr2=0x7f169c199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c103700 0x7f169c199870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d950 tx=0x7f168c00de10 comp rx=0 tx=0).stop 2024-03-21T08:28:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 shutdown_connections 2024-03-21T08:28:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f16800a2f10 0x7f16800a53d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f169c1052b0 0x7f169c1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f169c1048f0 0x7f169c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 --2- 172.21.15.3:0/3947321935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f169c103700 0x7f169c199870 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d950 tx=0x7f168c00de10 comp rx=0 tx=0).stop 2024-03-21T08:28:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 >> 172.21.15.3:0/3947321935 conn(0x7f169c0fecb0 msgr2=0x7f169c107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:39.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.653+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 shutdown_connections 2024-03-21T08:28:39.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.654+0000 7f169be00640 1 -- 172.21.15.3:0/3947321935 wait complete. 2024-03-21T08:28:39.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-21T08:28:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 -- 172.21.15.3:0/2121814267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4103270 msgr2=0x7f70f4109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 --2- 172.21.15.3:0/2121814267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4103270 0x7f70f4109b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009ca0 tx=0x7f70ec02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 -- 172.21.15.3:0/2121814267 shutdown_connections 2024-03-21T08:28:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 --2- 172.21.15.3:0/2121814267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4103270 0x7f70f4109b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 --2- 172.21.15.3:0/2121814267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41028b0 0x7f70f4102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 --2- 172.21.15.3:0/2121814267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70f41016c0 0x7f70f4101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 -- 172.21.15.3:0/2121814267 >> 172.21.15.3:0/2121814267 conn(0x7f70f40fcc30 msgr2=0x7f70f40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.878+0000 7f70fc400640 1 -- 172.21.15.3:0/2121814267 shutdown_connections 2024-03-21T08:28:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.879+0000 7f70fc400640 1 -- 172.21.15.3:0/2121814267 wait complete. 2024-03-21T08:28:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 Processor -- start 2024-03-21T08:28:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 -- start start 2024-03-21T08:28:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41016c0 0x7f70f41978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f41028b0 0x7f70f4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70f4103270 0x7f70f419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f410d1f0 con 0x7f70f41028b0 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f410d070 con 0x7f70f41016c0 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.880+0000 7f70fc400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f410d370 con 0x7f70f4103270 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f41028b0 0x7f70f4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41016c0 0x7f70f41978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f41028b0 0x7f70f4197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50368/0 (socket says 172.21.15.3:50368) 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41016c0 0x7f70f41978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:47006/0 (socket says 172.21.15.3:47006) 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9000640 1 -- 172.21.15.3:0/1165857522 learned_addr learned my addr 172.21.15.3:0/1165857522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70fa400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70f4103270 0x7f70f419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9a00640 1 -- 172.21.15.3:0/1165857522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70f4103270 msgr2=0x7f70f419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:39.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9a00640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70f4103270 0x7f70f419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9a00640 1 -- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f41028b0 msgr2=0x7f70f4197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9a00640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f41028b0 0x7f70f4197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9a00640 1 -- 172.21.15.3:0/1165857522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f419f4c0 con 0x7f70f41016c0 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70f9000640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f41028b0 0x7f70f4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.881+0000 7f70fa400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70f4103270 0x7f70f419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.882+0000 7f70f9a00640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41016c0 0x7f70f41978a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70f4102710 tx=0x7f70e800dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.882+0000 7f70f2000640 1 -- 172.21.15.3:0/1165857522 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8017070 con 0x7f70f41016c0 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.882+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f419f7b0 con 0x7f70f41016c0 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.882+0000 7f70f2000640 1 -- 172.21.15.3:0/1165857522 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70e8010460 con 0x7f70f41016c0 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.882+0000 7f70f2000640 1 -- 172.21.15.3:0/1165857522 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800f600 con 0x7f70f41016c0 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.882+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f419fcb0 con 0x7f70f41016c0 2024-03-21T08:28:39.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.884+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f41089e0 con 0x7f70f41016c0 2024-03-21T08:28:39.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.886+0000 7f70f2000640 1 -- 172.21.15.3:0/1165857522 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f70e801a070 con 0x7f70f41016c0 2024-03-21T08:28:39.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.887+0000 7f70f2000640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70dc0a2f20 0x7f70dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:39.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.887+0000 7f70f2000640 1 -- 172.21.15.3:0/1165857522 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f70e80cefd0 con 0x7f70f41016c0 2024-03-21T08:28:39.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.888+0000 7f70f9000640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70dc0a2f20 0x7f70dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:39.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.888+0000 7f70f9000640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70dc0a2f20 0x7f70dc0a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70f4198db0 tx=0x7f70e000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:39.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:39.893+0000 7f70f2000640 1 -- 172.21.15.3:0/1165857522 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f70e809bbe0 con 0x7f70f41016c0 2024-03-21T08:28:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.110+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f70f402d070 con 0x7f70f41016c0 2024-03-21T08:28:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:40 smithi084 ceph-mon[35477]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T08:28:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:40 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3947321935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T08:28:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:40 smithi084 ceph-mon[35477]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T08:28:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:40 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1165857522' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T08:28:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:40 smithi084 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T08:28:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:40.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:40 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:40.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:40 smithi171 ceph-mon[35388]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T08:28:40.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:40 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3947321935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T08:28:40.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:40 smithi171 ceph-mon[35388]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T08:28:40.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:40 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1165857522' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T08:28:40.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:40 smithi171 ceph-mon[35388]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T08:28:40.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:40.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:40 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:40.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.665+0000 7f70f2000640 1 -- 172.21.15.3:0/1165857522 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f70e809b4e0 con 0x7f70f41016c0 2024-03-21T08:28:40.667 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 4 allow_ec_overwrites to true 2024-03-21T08:28:40.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:40 smithi003 ceph-mon[31675]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T08:28:40.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:40 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3947321935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T08:28:40.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:40 smithi003 ceph-mon[31675]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T08:28:40.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:40 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1165857522' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T08:28:40.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:40 smithi003 ceph-mon[31675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T08:28:40.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:40.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:40 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.675+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70dc0a2f20 msgr2=0x7f70dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.676+0000 7f70fc400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70dc0a2f20 0x7f70dc0a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70f4198db0 tx=0x7f70e000b480 comp rx=0 tx=0).stop 2024-03-21T08:28:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.676+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41016c0 msgr2=0x7f70f41978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.676+0000 7f70fc400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41016c0 0x7f70f41978a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70f4102710 tx=0x7f70e800dbb0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.677+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 shutdown_connections 2024-03-21T08:28:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.677+0000 7f70fc400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f70dc0a2f20 0x7f70dc0a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.677+0000 7f70fc400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70f4103270 0x7f70f419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.677+0000 7f70fc400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f41028b0 0x7f70f4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.677+0000 7f70fc400640 1 --2- 172.21.15.3:0/1165857522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70f41016c0 0x7f70f41978a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.677+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 >> 172.21.15.3:0/1165857522 conn(0x7f70f40fcc30 msgr2=0x7f70f4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:40.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.682+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 shutdown_connections 2024-03-21T08:28:40.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.683+0000 7f70fc400640 1 -- 172.21.15.3:0/1165857522 wait complete. 2024-03-21T08:28:40.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-21T08:28:40.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 -- 172.21.15.3:0/3852755303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 msgr2=0x7ff088103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:40.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 --2- 172.21.15.3:0/3852755303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088103cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009900 tx=0x7ff07c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:40.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 -- 172.21.15.3:0/3852755303 shutdown_connections 2024-03-21T08:28:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 --2- 172.21.15.3:0/3852755303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff088105480 0x7ff08810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 --2- 172.21.15.3:0/3852755303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff088104ac0 0x7ff088104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 --2- 172.21.15.3:0/3852755303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088103cf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 -- 172.21.15.3:0/3852755303 >> 172.21.15.3:0/3852755303 conn(0x7ff0880fee60 msgr2=0x7ff088101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 -- 172.21.15.3:0/3852755303 shutdown_connections 2024-03-21T08:28:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.891+0000 7ff090800640 1 -- 172.21.15.3:0/3852755303 wait complete. 2024-03-21T08:28:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.892+0000 7ff090800640 1 Processor -- start 2024-03-21T08:28:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.892+0000 7ff090800640 1 -- start start 2024-03-21T08:28:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff090800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff090800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff088104ac0 0x7ff08819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff090800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff088105480 0x7ff0881a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff090800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08810f400 con 0x7ff0881038d0 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff090800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08810f280 con 0x7ff088105480 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff090800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08810f580 con 0x7ff088104ac0 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57896/0 (socket says 172.21.15.3:57896) 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 -- 172.21.15.3:0/2232344861 learned_addr learned my addr 172.21.15.3:0/2232344861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08e800640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff088105480 0x7ff0881a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08d400640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff088104ac0 0x7ff08819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 -- 172.21.15.3:0/2232344861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff088104ac0 msgr2=0x7ff08819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff088104ac0 0x7ff08819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 -- 172.21.15.3:0/2232344861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff088105480 msgr2=0x7ff0881a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff088105480 0x7ff0881a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.893+0000 7ff08de00640 1 -- 172.21.15.3:0/2232344861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0881a1750 con 0x7ff0881038d0 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.894+0000 7ff08de00640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088199b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff07c000c00 tx=0x7ff07c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.894+0000 7ff082a00640 1 -- 172.21.15.3:0/2232344861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c043020 con 0x7ff0881038d0 2024-03-21T08:28:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.894+0000 7ff082a00640 1 -- 172.21.15.3:0/2232344861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff07c004aa0 con 0x7ff0881038d0 2024-03-21T08:28:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.894+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0881a19e0 con 0x7ff0881038d0 2024-03-21T08:28:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.894+0000 7ff082a00640 1 -- 172.21.15.3:0/2232344861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c003d00 con 0x7ff0881038d0 2024-03-21T08:28:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.894+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0881a1f30 con 0x7ff0881038d0 2024-03-21T08:28:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.896+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08810acc0 con 0x7ff0881038d0 2024-03-21T08:28:40.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.898+0000 7ff082a00640 1 -- 172.21.15.3:0/2232344861 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7ff07c046020 con 0x7ff0881038d0 2024-03-21T08:28:40.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.900+0000 7ff082a00640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0700a2f20 0x7ff0700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.900+0000 7ff082a00640 1 -- 172.21.15.3:0/2232344861 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8770+0+0 (secure 0 0 0) 0x7ff07c0f2120 con 0x7ff0881038d0 2024-03-21T08:28:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.900+0000 7ff08d400640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0700a2f20 0x7ff0700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.901+0000 7ff08d400640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0700a2f20 0x7ff0700a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff078005fd0 tx=0x7ff078005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:40.903+0000 7ff082a00640 1 -- 172.21.15.3:0/2232344861 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff07c04d020 con 0x7ff0881038d0 2024-03-21T08:28:41.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.153+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7ff08802d1d0 con 0x7ff0881038d0 2024-03-21T08:28:41.625 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h[65129]: 2024-03-21T08:28:41.142+0000 7f8c7ec00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: pgmap v366: 97 pgs: 23 creating+peering, 60 active+clean, 14 unknown; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 8 op/s 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T08:28:41.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T08:28:41.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:41 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2232344861' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: pgmap v366: 97 pgs: 23 creating+peering, 60 active+clean, 14 unknown; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 8 op/s 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T08:28:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:41 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2232344861' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T08:28:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.656+0000 7ff082a00640 1 -- 172.21.15.3:0/2232344861 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7ff07c0becd0 con 0x7ff0881038d0 2024-03-21T08:28:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:added data pool 4 to fsmap 2024-03-21T08:28:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0700a2f20 msgr2=0x7ff0700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0700a2f20 0x7ff0700a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff078005fd0 tx=0x7ff078005850 comp rx=0 tx=0).stop 2024-03-21T08:28:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 msgr2=0x7ff088199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088199b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff07c000c00 tx=0x7ff07c004270 comp rx=0 tx=0).stop 2024-03-21T08:28:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 shutdown_connections 2024-03-21T08:28:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff088105480 0x7ff0881a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff0700a2f20 0x7ff0700a53e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff088104ac0 0x7ff08819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 --2- 172.21.15.3:0/2232344861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0881038d0 0x7ff088199b30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 >> 172.21.15.3:0/2232344861 conn(0x7ff0880fee60 msgr2=0x7ff088107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 shutdown_connections 2024-03-21T08:28:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.660+0000 7ff090800640 1 -- 172.21.15.3:0/2232344861 wait complete. 2024-03-21T08:28:41.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: pgmap v366: 97 pgs: 23 creating+peering, 60 active+clean, 14 unknown; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 8 op/s 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T08:28:41.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:41 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2232344861' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T08:28:41.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:41.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 -- 172.21.15.3:0/1567479303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 msgr2=0x7f8b201045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:41.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 --2- 172.21.15.3:0/1567479303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b201045b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009ca0 tx=0x7f8b1002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 -- 172.21.15.3:0/1567479303 shutdown_connections 2024-03-21T08:28:41.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 --2- 172.21.15.3:0/1567479303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b20107420 0x7f8b20109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 --2- 172.21.15.3:0/1567479303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b20104af0 0x7f8b20106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 --2- 172.21.15.3:0/1567479303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b201045b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 -- 172.21.15.3:0/1567479303 >> 172.21.15.3:0/1567479303 conn(0x7f8b200fbbb0 msgr2=0x7f8b200fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:41.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 -- 172.21.15.3:0/1567479303 shutdown_connections 2024-03-21T08:28:41.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.906+0000 7f8b25400640 1 -- 172.21.15.3:0/1567479303 wait complete. 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.907+0000 7f8b25400640 1 Processor -- start 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b25400640 1 -- start start 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b25400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b20101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b25400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b20104af0 0x7f8b20100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b25400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b20107420 0x7f8b201006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b25400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2010d210 con 0x7f8b201021c0 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b25400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2010d090 con 0x7f8b20104af0 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b25400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2010d390 con 0x7f8b20107420 2024-03-21T08:28:41.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b1e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b20101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b1e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b20101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57920/0 (socket says 172.21.15.3:57920) 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.908+0000 7f8b1e800640 1 -- 172.21.15.3:0/475255079 learned_addr learned my addr 172.21.15.3:0/475255079 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b15e00640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b20104af0 0x7f8b20100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1f200640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b20107420 0x7f8b201006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1e800640 1 -- 172.21.15.3:0/475255079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b20107420 msgr2=0x7f8b201006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1e800640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b20107420 0x7f8b201006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1e800640 1 -- 172.21.15.3:0/475255079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b20104af0 msgr2=0x7f8b20100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1e800640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b20104af0 0x7f8b20100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1e800640 1 -- 172.21.15.3:0/475255079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20100f40 con 0x7f8b201021c0 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1f200640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b20107420 0x7f8b201006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b1e800640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b20101b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b1002c700 tx=0x7f8b100328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b17e00640 1 -- 172.21.15.3:0/475255079 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10041020 con 0x7f8b201021c0 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b17e00640 1 -- 172.21.15.3:0/475255079 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b10003e20 con 0x7f8b201021c0 2024-03-21T08:28:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b201a1950 con 0x7f8b201021c0 2024-03-21T08:28:41.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.909+0000 7f8b17e00640 1 -- 172.21.15.3:0/475255079 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10039040 con 0x7f8b201021c0 2024-03-21T08:28:41.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.910+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b201a1ef0 con 0x7f8b201021c0 2024-03-21T08:28:41.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.911+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b200fd6a0 con 0x7f8b201021c0 2024-03-21T08:28:41.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.913+0000 7f8b17e00640 1 -- 172.21.15.3:0/475255079 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8b10044020 con 0x7f8b201021c0 2024-03-21T08:28:41.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.914+0000 7f8b17e00640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b040a2fa0 0x7f8b040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:41.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.914+0000 7f8b17e00640 1 -- 172.21.15.3:0/475255079 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8b100f1980 con 0x7f8b201021c0 2024-03-21T08:28:41.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.914+0000 7f8b15e00640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b040a2fa0 0x7f8b040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.915+0000 7f8b15e00640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b040a2fa0 0x7f8b040a5460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005fd0 tx=0x7f8b08005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:41.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:41.919+0000 7f8b17e00640 1 -- 172.21.15.3:0/475255079 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b1004b020 con 0x7f8b201021c0 2024-03-21T08:28:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.164+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b2002d070 con 0x7f8b201021c0 2024-03-21T08:28:42.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.166+0000 7f8b17e00640 1 -- 172.21.15.3:0/475255079 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11316 (secure 0 0 0) 0x7f8b100be510 con 0x7f8b201021c0 2024-03-21T08:28:42.168 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:42.168 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:28:41.155339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34472,"qdb_cluster":[34472]},"id":1}]} 2024-03-21T08:28:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 16 2024-03-21T08:28:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b040a2fa0 msgr2=0x7f8b040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b040a2fa0 0x7f8b040a5460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005fd0 tx=0x7f8b08005850 comp rx=0 tx=0).stop 2024-03-21T08:28:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 msgr2=0x7f8b20101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b20101b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b1002c700 tx=0x7f8b100328a0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 shutdown_connections 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b040a2fa0 0x7f8b040a5460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b20107420 0x7f8b201006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b20104af0 0x7f8b20100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 --2- 172.21.15.3:0/475255079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b201021c0 0x7f8b20101b30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 >> 172.21.15.3:0/475255079 conn(0x7f8b200fbbb0 msgr2=0x7f8b20102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 shutdown_connections 2024-03-21T08:28:42.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.169+0000 7f8b25400640 1 -- 172.21.15.3:0/475255079 wait complete. 2024-03-21T08:28:42.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:28:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.416+0000 7fd11e200640 1 -- 172.21.15.3:0/1556886827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 msgr2=0x7fd118109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.416+0000 7fd11e200640 1 --2- 172.21.15.3:0/1556886827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd118109350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd100009900 tx=0x7fd10002bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.417+0000 7fd11e200640 1 -- 172.21.15.3:0/1556886827 shutdown_connections 2024-03-21T08:28:42.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.417+0000 7fd11e200640 1 --2- 172.21.15.3:0/1556886827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd118109890 0x7fd118110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.417+0000 7fd11e200640 1 --2- 172.21.15.3:0/1556886827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd118109350 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.417+0000 7fd11e200640 1 --2- 172.21.15.3:0/1556886827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd118107ce0 0x7fd118108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.417+0000 7fd11e200640 1 -- 172.21.15.3:0/1556886827 >> 172.21.15.3:0/1556886827 conn(0x7fd118076ca0 msgr2=0x7fd1180790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.417+0000 7fd11e200640 1 -- 172.21.15.3:0/1556886827 shutdown_connections 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.417+0000 7fd11e200640 1 -- 172.21.15.3:0/1556886827 wait complete. 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 Processor -- start 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 -- start start 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd118107ce0 0x7fd11819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd11819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd118109890 0x7fd1181a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd118113860 con 0x7fd118108ed0 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1181136e0 con 0x7fd118109890 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd11e200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1181139e0 con 0x7fd118107ce0 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd10fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd11819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd10fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd11819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57938/0 (socket says 172.21.15.3:57938) 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.418+0000 7fd10fe00640 1 -- 172.21.15.3:0/2582259162 learned_addr learned my addr 172.21.15.3:0/2582259162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd117400640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd118107ce0 0x7fd11819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd117e00640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd118109890 0x7fd1181a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd10fe00640 1 -- 172.21.15.3:0/2582259162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd118107ce0 msgr2=0x7fd11819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd10fe00640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd118107ce0 0x7fd11819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd10fe00640 1 -- 172.21.15.3:0/2582259162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd118109890 msgr2=0x7fd1181a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd10fe00640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd118109890 0x7fd1181a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd10fe00640 1 -- 172.21.15.3:0/2582259162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1181a5af0 con 0x7fd118108ed0 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd117400640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd118107ce0 0x7fd11819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd117e00640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd118109890 0x7fd1181a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd10fe00640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd11819e410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd100000c00 tx=0x7fd1000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd114c00640 1 -- 172.21.15.3:0/2582259162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100043020 con 0x7fd118108ed0 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd114c00640 1 -- 172.21.15.3:0/2582259162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd100004aa0 con 0x7fd118108ed0 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd114c00640 1 -- 172.21.15.3:0/2582259162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100003d00 con 0x7fd118108ed0 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.419+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1181a5d80 con 0x7fd118108ed0 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.420+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1181a6270 con 0x7fd118108ed0 2024-03-21T08:28:42.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.421+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11810f080 con 0x7fd118108ed0 2024-03-21T08:28:42.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.423+0000 7fd114c00640 1 -- 172.21.15.3:0/2582259162 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd100046020 con 0x7fd118108ed0 2024-03-21T08:28:42.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.424+0000 7fd114c00640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd0fc0a2f20 0x7fd0fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.424+0000 7fd114c00640 1 -- 172.21.15.3:0/2582259162 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd1000f1a50 con 0x7fd118108ed0 2024-03-21T08:28:42.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.424+0000 7fd117400640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd0fc0a2f20 0x7fd0fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:42.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.425+0000 7fd117400640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd0fc0a2f20 0x7fd0fc0a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd118108d30 tx=0x7fd108005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.428+0000 7fd114c00640 1 -- 172.21.15.3:0/2582259162 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd10004d020 con 0x7fd118108ed0 2024-03-21T08:28:42.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:42 smithi084 ceph-mon[35477]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T08:28:42.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:42 smithi084 ceph-mon[35477]: pgmap v370: 129 pgs: 18 creating+peering, 95 active+clean, 16 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2024-03-21T08:28:42.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:42 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2232344861' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T08:28:42.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:42 smithi084 ceph-mon[35477]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:28:42.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:42 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/475255079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:42.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:42 smithi171 ceph-mon[35388]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T08:28:42.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:42 smithi171 ceph-mon[35388]: pgmap v370: 129 pgs: 18 creating+peering, 95 active+clean, 16 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2024-03-21T08:28:42.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:42 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2232344861' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T08:28:42.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:42 smithi171 ceph-mon[35388]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:28:42.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:42 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/475255079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:42.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.633+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd118062340 con 0x7fd118108ed0 2024-03-21T08:28:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.637+0000 7fd114c00640 1 -- 172.21.15.3:0/2582259162 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24035 (secure 0 0 0) 0x7fd1000be700 con 0x7fd118108ed0 2024-03-21T08:28:42.641 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:42.641 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:28:41.154980+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:28:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd0fc0a2f20 msgr2=0x7fd0fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd0fc0a2f20 0x7fd0fc0a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd118108d30 tx=0x7fd108005f90 comp rx=0 tx=0).stop 2024-03-21T08:28:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 msgr2=0x7fd11819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd11819e410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd100000c00 tx=0x7fd1000042f0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 shutdown_connections 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd118109890 0x7fd1181a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd118108ed0 0x7fd11819e410 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd0fc0a2f20 0x7fd0fc0a53e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 --2- 172.21.15.3:0/2582259162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd118107ce0 0x7fd11819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 >> 172.21.15.3:0/2582259162 conn(0x7fd118076ca0 msgr2=0x7fd11810c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 shutdown_connections 2024-03-21T08:28:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.640+0000 7fd11e200640 1 -- 172.21.15.3:0/2582259162 wait complete. 2024-03-21T08:28:42.657 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:42 smithi003 ceph-mon[31675]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T08:28:42.658 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:42 smithi003 ceph-mon[31675]: pgmap v370: 129 pgs: 18 creating+peering, 95 active+clean, 16 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2024-03-21T08:28:42.658 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:42 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2232344861' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T08:28:42.658 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:42 smithi003 ceph-mon[31675]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:28:42.658 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:42 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/475255079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:42.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 -- 172.21.15.3:0/4044364433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0109770 msgr2=0x7f4bd0110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/4044364433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0109770 0x7f4bd0110000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009ca0 tx=0x7f4bc802c100 comp rx=0 tx=0).stop 2024-03-21T08:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 -- 172.21.15.3:0/4044364433 shutdown_connections 2024-03-21T08:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/4044364433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0109770 0x7f4bd0110000 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/4044364433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bd0108db0 0x7f4bd0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/4044364433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bd0107bc0 0x7f4bd0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 -- 172.21.15.3:0/4044364433 >> 172.21.15.3:0/4044364433 conn(0x7f4bd0076b60 msgr2=0x7f4bd0078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:42.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 -- 172.21.15.3:0/4044364433 shutdown_connections 2024-03-21T08:28:42.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.890+0000 7f4bcfe00640 1 -- 172.21.15.3:0/4044364433 wait complete. 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 Processor -- start 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 -- start start 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0107bc0 0x7f4bd019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bd0108db0 0x7f4bd019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bd0109770 0x7f4bd01a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd01136f0 con 0x7f4bd0107bc0 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0113570 con 0x7f4bd0108db0 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcfe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0113870 con 0x7f4bd0109770 2024-03-21T08:28:42.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.891+0000 7f4bcea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0107bc0 0x7f4bd019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0107bc0 0x7f4bd019dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57958/0 (socket says 172.21.15.3:57958) 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 -- 172.21.15.3:0/438692068 learned_addr learned my addr 172.21.15.3:0/438692068 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcf400640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bd0109770 0x7f4bd01a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 -- 172.21.15.3:0/438692068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bd0109770 msgr2=0x7f4bd01a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bd0109770 0x7f4bd01a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 -- 172.21.15.3:0/438692068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bd0108db0 msgr2=0x7f4bd019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bd0108db0 0x7f4bd019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 -- 172.21.15.3:0/438692068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd01a5900 con 0x7f4bd0107bc0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.892+0000 7f4bcea00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0107bc0 0x7f4bd019dcf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000d950 tx=0x7f4bc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.893+0000 7f4bc7400640 1 -- 172.21.15.3:0/438692068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0017070 con 0x7f4bd0107bc0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.893+0000 7f4bc7400640 1 -- 172.21.15.3:0/438692068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4bc0009c30 con 0x7f4bd0107bc0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.893+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd01a5bf0 con 0x7f4bd0107bc0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.893+0000 7f4bc7400640 1 -- 172.21.15.3:0/438692068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000f620 con 0x7f4bd0107bc0 2024-03-21T08:28:42.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.893+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd01a6140 con 0x7f4bd0107bc0 2024-03-21T08:28:42.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.894+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd010ee80 con 0x7f4bd0107bc0 2024-03-21T08:28:42.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.896+0000 7f4bc7400640 1 -- 172.21.15.3:0/438692068 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f4bc001a070 con 0x7f4bd0107bc0 2024-03-21T08:28:42.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.897+0000 7f4bc7400640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4bb40a2f90 0x7f4bb40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:42.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.897+0000 7f4bc7400640 1 -- 172.21.15.3:0/438692068 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4bc00cf1c0 con 0x7f4bd0107bc0 2024-03-21T08:28:42.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.898+0000 7f4bce000640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4bb40a2f90 0x7f4bb40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:42.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.898+0000 7f4bce000640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4bb40a2f90 0x7f4bb40a5450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800a990 tx=0x7f4bb8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:42.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:42.902+0000 7f4bc7400640 1 -- 172.21.15.3:0/438692068 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4bc0009de0 con 0x7f4bd0107bc0 2024-03-21T08:28:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.110+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd00622d0 con 0x7f4bd0107bc0 2024-03-21T08:28:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.114+0000 7f4bc7400640 1 -- 172.21.15.3:0/438692068 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24035 (secure 0 0 0) 0x7f4bc009bd50 con 0x7f4bd0107bc0 2024-03-21T08:28:43.115 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:43.116 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:28:41.154980+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:28:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4bb40a2f90 msgr2=0x7f4bb40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4bb40a2f90 0x7f4bb40a5450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800a990 tx=0x7f4bb8005c70 comp rx=0 tx=0).stop 2024-03-21T08:28:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0107bc0 msgr2=0x7f4bd019dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0107bc0 0x7f4bd019dcf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000d950 tx=0x7f4bc000de10 comp rx=0 tx=0).stop 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 shutdown_connections 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4bb40a2f90 0x7f4bb40a5450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bd0109770 0x7f4bd01a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bd0108db0 0x7f4bd019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 --2- 172.21.15.3:0/438692068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bd0107bc0 0x7f4bd019dcf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 >> 172.21.15.3:0/438692068 conn(0x7f4bd0076b60 msgr2=0x7f4bd010c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.117+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 shutdown_connections 2024-03-21T08:28:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.118+0000 7f4bcfe00640 1 -- 172.21.15.3:0/438692068 wait complete. 2024-03-21T08:28:43.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:28:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 -- 172.21.15.3:0/2289128957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 msgr2=0x7fe9ec073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 --2- 172.21.15.3:0/2289128957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec073b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009900 tx=0x7fe9e002bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 -- 172.21.15.3:0/2289128957 shutdown_connections 2024-03-21T08:28:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 --2- 172.21.15.3:0/2289128957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec109480 0x7fe9ec10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 --2- 172.21.15.3:0/2289128957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9ec074100 0x7fe9ec074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 --2- 172.21.15.3:0/2289128957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec073b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 -- 172.21.15.3:0/2289128957 >> 172.21.15.3:0/2289128957 conn(0x7fe9ec0fcb30 msgr2=0x7fe9ec0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 -- 172.21.15.3:0/2289128957 shutdown_connections 2024-03-21T08:28:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.381+0000 7fe9f2800640 1 -- 172.21.15.3:0/2289128957 wait complete. 2024-03-21T08:28:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 Processor -- start 2024-03-21T08:28:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 -- start start 2024-03-21T08:28:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9ec074100 0x7fe9ec199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec109480 0x7fe9ec1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10f320 con 0x7fe9ec0756e0 2024-03-21T08:28:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10f1a0 con 0x7fe9ec109480 2024-03-21T08:28:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.383+0000 7fe9f2800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10f4a0 con 0x7fe9ec074100 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57974/0 (socket says 172.21.15.3:57974) 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 -- 172.21.15.3:0/3936658352 learned_addr learned my addr 172.21.15.3:0/3936658352 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f1400640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9ec074100 0x7fe9ec199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f1e00640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec109480 0x7fe9ec1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 -- 172.21.15.3:0/3936658352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9ec074100 msgr2=0x7fe9ec199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9ec074100 0x7fe9ec199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 -- 172.21.15.3:0/3936658352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec109480 msgr2=0x7fe9ec1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec109480 0x7fe9ec1a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 -- 172.21.15.3:0/3936658352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec1a1610 con 0x7fe9ec0756e0 2024-03-21T08:28:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f1400640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9ec074100 0x7fe9ec199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.384+0000 7fe9f0a00640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec199f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800e9f0 tx=0x7fe9d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.385+0000 7fe9ea000640 1 -- 172.21.15.3:0/3936658352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800cd80 con 0x7fe9ec0756e0 2024-03-21T08:28:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.385+0000 7fe9ea000640 1 -- 172.21.15.3:0/3936658352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe9d80054d0 con 0x7fe9ec0756e0 2024-03-21T08:28:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.385+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec1a1900 con 0x7fe9ec0756e0 2024-03-21T08:28:43.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.385+0000 7fe9ea000640 1 -- 172.21.15.3:0/3936658352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8010490 con 0x7fe9ec0756e0 2024-03-21T08:28:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.385+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec1a1e50 con 0x7fe9ec0756e0 2024-03-21T08:28:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.386+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec1081b0 con 0x7fe9ec0756e0 2024-03-21T08:28:43.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.389+0000 7fe9ea000640 1 -- 172.21.15.3:0/3936658352 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fe9d8005020 con 0x7fe9ec0756e0 2024-03-21T08:28:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.391+0000 7fe9ea000640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9d40a2f90 0x7fe9d40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.391+0000 7fe9ea000640 1 -- 172.21.15.3:0/3936658352 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe9d80cfac0 con 0x7fe9ec0756e0 2024-03-21T08:28:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.391+0000 7fe9f1400640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9d40a2f90 0x7fe9d40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.392+0000 7fe9f1400640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9d40a2f90 0x7fe9d40a5450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0006010 tx=0x7fe9e002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.395+0000 7fe9ea000640 1 -- 172.21.15.3:0/3936658352 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe9d809c650 con 0x7fe9ec0756e0 2024-03-21T08:28:43.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:43 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2582259162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:43.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:43 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/438692068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:43.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:43 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2582259162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:43.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:43 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/438692068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:43.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:43 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2582259162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:43.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:43 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/438692068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.640+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9ec0622a0 con 0x7fe9ec0756e0 2024-03-21T08:28:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.642+0000 7fe9ea000640 1 -- 172.21.15.3:0/3936658352 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11316 (secure 0 0 0) 0x7fe9d809bf50 con 0x7fe9ec0756e0 2024-03-21T08:28:43.643 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:43.643 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34409,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34439,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:28:41.155339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34472,"qdb_cluster":[34472]},"id":1}]} 2024-03-21T08:28:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 16 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9d40a2f90 msgr2=0x7fe9d40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9d40a2f90 0x7fe9d40a5450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0006010 tx=0x7fe9e002c7e0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 msgr2=0x7fe9ec199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec199f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800e9f0 tx=0x7fe9d800edf0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 shutdown_connections 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9ec109480 0x7fe9ec1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9ec0756e0 0x7fe9ec199f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9d40a2f90 0x7fe9d40a5450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 --2- 172.21.15.3:0/3936658352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9ec074100 0x7fe9ec199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 >> 172.21.15.3:0/3936658352 conn(0x7fe9ec0fcb30 msgr2=0x7fe9ec105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 shutdown_connections 2024-03-21T08:28:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.645+0000 7fe9f2800640 1 -- 172.21.15.3:0/3936658352 wait complete. 2024-03-21T08:28:43.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.886+0000 7f990fe00640 1 -- 172.21.15.3:0/3103726459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 msgr2=0x7f9910073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.886+0000 7f990fe00640 1 --2- 172.21.15.3:0/3103726459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910073ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9900009900 tx=0x7f990002bcd0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.887+0000 7f990fe00640 1 -- 172.21.15.3:0/3103726459 shutdown_connections 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.887+0000 7f990fe00640 1 --2- 172.21.15.3:0/3103726459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9910109450 0x7f991010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.887+0000 7f990fe00640 1 --2- 172.21.15.3:0/3103726459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99100740a0 0x7f9910074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.887+0000 7f990fe00640 1 --2- 172.21.15.3:0/3103726459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910073ad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.887+0000 7f990fe00640 1 -- 172.21.15.3:0/3103726459 >> 172.21.15.3:0/3103726459 conn(0x7f99100fcb30 msgr2=0x7f99100fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.887+0000 7f990fe00640 1 -- 172.21.15.3:0/3103726459 shutdown_connections 2024-03-21T08:28:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.887+0000 7f990fe00640 1 -- 172.21.15.3:0/3103726459 wait complete. 2024-03-21T08:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 Processor -- start 2024-03-21T08:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 -- start start 2024-03-21T08:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99100740a0 0x7f99101999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9910109450 0x7f99101a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991010f390 con 0x7f9910075680 2024-03-21T08:28:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991010f210 con 0x7f99100740a0 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.888+0000 7f990fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991010f510 con 0x7f9910109450 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57990/0 (socket says 172.21.15.3:57990) 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 -- 172.21.15.3:0/1968838625 learned_addr learned my addr 172.21.15.3:0/1968838625 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990ea00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99100740a0 0x7f99101999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990f400640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9910109450 0x7f99101a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 -- 172.21.15.3:0/1968838625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9910109450 msgr2=0x7f99101a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9910109450 0x7f99101a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 -- 172.21.15.3:0/1968838625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99100740a0 msgr2=0x7f99101999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99100740a0 0x7f99101999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 -- 172.21.15.3:0/1968838625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99101a1610 con 0x7f9910075680 2024-03-21T08:28:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.889+0000 7f990e000640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910199f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00b4a0 tx=0x7f98fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.890+0000 7f9907400640 1 -- 172.21.15.3:0/1968838625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc014070 con 0x7f9910075680 2024-03-21T08:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.890+0000 7f9907400640 1 -- 172.21.15.3:0/1968838625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98fc010ac0 con 0x7f9910075680 2024-03-21T08:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.890+0000 7f9907400640 1 -- 172.21.15.3:0/1968838625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc017c30 con 0x7f9910075680 2024-03-21T08:28:43.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.890+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99101a1900 con 0x7f9910075680 2024-03-21T08:28:43.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.890+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99101a1e80 con 0x7f9910075680 2024-03-21T08:28:43.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.891+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9910108210 con 0x7f9910075680 2024-03-21T08:28:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.894+0000 7f9907400640 1 -- 172.21.15.3:0/1968838625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f98fc017070 con 0x7f9910075680 2024-03-21T08:28:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.895+0000 7f9907400640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f98f40a2f90 0x7f98f40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.895+0000 7f9907400640 1 -- 172.21.15.3:0/1968838625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f98fc0cea30 con 0x7f9910075680 2024-03-21T08:28:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.895+0000 7f990ea00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f98f40a2f90 0x7f98f40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.896+0000 7f990ea00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f98f40a2f90 0x7f98f40a5450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f990002c8e0 tx=0x7f9900000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:43.899+0000 7f9907400640 1 -- 172.21.15.3:0/1968838625 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f98fc09b5c0 con 0x7f9910075680 2024-03-21T08:28:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.105+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99100622a0 con 0x7f9910075680 2024-03-21T08:28:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.110+0000 7f9907400640 1 -- 172.21.15.3:0/1968838625 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24035 (secure 0 0 0) 0x7f98fc09aec0 con 0x7f9910075680 2024-03-21T08:28:44.111 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:44.111 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:28:41.154980+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:28:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.112+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f98f40a2f90 msgr2=0x7f98f40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.112+0000 7f990fe00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f98f40a2f90 0x7f98f40a5450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f990002c8e0 tx=0x7f9900000bc0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.112+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 msgr2=0x7f9910199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.112+0000 7f990fe00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910199f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00b4a0 tx=0x7f98fc00b960 comp rx=0 tx=0).stop 2024-03-21T08:28:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.112+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 shutdown_connections 2024-03-21T08:28:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.112+0000 7f990fe00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f98f40a2f90 0x7f98f40a5450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.112+0000 7f990fe00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9910109450 0x7f99101a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.113+0000 7f990fe00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9910075680 0x7f9910199f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.113+0000 7f990fe00640 1 --2- 172.21.15.3:0/1968838625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99100740a0 0x7f99101999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.113+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 >> 172.21.15.3:0/1968838625 conn(0x7f99100fcb30 msgr2=0x7f99101053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.113+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 shutdown_connections 2024-03-21T08:28:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.113+0000 7f990fe00640 1 -- 172.21.15.3:0/1968838625 wait complete. 2024-03-21T08:28:44.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-21T08:28:36.113879+0000', 'modified': '2024-03-21T08:28:41.155339+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.84:6837/3332280865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 3332280865}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 3332280865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34472, 'qdb_cluster': [34472]} 2024-03-21T08:28:44.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-21T08:28:44.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 -- 172.21.15.3:0/3213231525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1052b0 msgr2=0x7fd22c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 --2- 172.21.15.3:0/3213231525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1052b0 0x7fd22c10bb40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd224009ca0 tx=0x7fd22402c100 comp rx=0 tx=0).stop 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 -- 172.21.15.3:0/3213231525 shutdown_connections 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 --2- 172.21.15.3:0/3213231525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1052b0 0x7fd22c10bb40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 --2- 172.21.15.3:0/3213231525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd22c1048f0 0x7fd22c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 --2- 172.21.15.3:0/3213231525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd22c103700 0x7fd22c103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 -- 172.21.15.3:0/3213231525 >> 172.21.15.3:0/3213231525 conn(0x7fd22c0fec70 msgr2=0x7fd22c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 -- 172.21.15.3:0/3213231525 shutdown_connections 2024-03-21T08:28:44.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.352+0000 7fd22b200640 1 -- 172.21.15.3:0/3213231525 wait complete. 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 Processor -- start 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 -- start start 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd22c103700 0x7fd22c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1048f0 0x7fd22c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd22c1052b0 0x7fd22c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c10f210 con 0x7fd22c1048f0 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c10f090 con 0x7fd22c1052b0 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd22b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c10f390 con 0x7fd22c103700 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd223e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1048f0 0x7fd22c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd223e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1048f0 0x7fd22c199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58012/0 (socket says 172.21.15.3:58012) 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.353+0000 7fd223e00640 1 -- 172.21.15.3:0/3595761842 learned_addr learned my addr 172.21.15.3:0/3595761842 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd22a800640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd22c1052b0 0x7fd22c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd229e00640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd22c103700 0x7fd22c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd223e00640 1 -- 172.21.15.3:0/3595761842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd22c103700 msgr2=0x7fd22c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd223e00640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd22c103700 0x7fd22c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd223e00640 1 -- 172.21.15.3:0/3595761842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd22c1052b0 msgr2=0x7fd22c1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd223e00640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd22c1052b0 0x7fd22c1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd223e00640 1 -- 172.21.15.3:0/3595761842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c1a1470 con 0x7fd22c1048f0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd22a800640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd22c1052b0 0x7fd22c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd223e00640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1048f0 0x7fd22c199d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd21400d950 tx=0x7fd21400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd222a00640 1 -- 172.21.15.3:0/3595761842 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214017070 con 0x7fd22c1048f0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd222a00640 1 -- 172.21.15.3:0/3595761842 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd214009c30 con 0x7fd22c1048f0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd222a00640 1 -- 172.21.15.3:0/3595761842 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400f620 con 0x7fd22c1048f0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c1a1760 con 0x7fd22c1048f0 2024-03-21T08:28:44.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.354+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c1a1d00 con 0x7fd22c1048f0 2024-03-21T08:28:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 Processor -- start 2024-03-21T08:28:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd222a00640 1 -- 172.21.15.3:0/3595761842 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd2140109a0 con 0x7fd22c1048f0 2024-03-21T08:28:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 -- start start 2024-03-21T08:28:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2795d80 0x55f7c27961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c279b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c279dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c26fac90 con 0x55f7c2796770 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c26a3980 con 0x55f7c279b8e0 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.358+0000 7fd233642740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c26f6d30 con 0x55f7c2795d80 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c279b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c279b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58028/0 (socket says 172.21.15.3:58028) 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 -- 172.21.15.3:0/3555733761 learned_addr learned my addr 172.21.15.3:0/3555733761 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd22a800640 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2795d80 0x55f7c27961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd223e00640 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c279dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 -- 172.21.15.3:0/3555733761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2795d80 msgr2=0x55f7c27961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2795d80 0x55f7c27961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 -- 172.21.15.3:0/3555733761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 msgr2=0x55f7c279dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c279dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 -- 172.21.15.3:0/3555733761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7c26bb510 con 0x55f7c2796770 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd22a800640 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2795d80 0x55f7c27961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd222a00640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd2100a2fa0 0x7fd2100a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.359+0000 7fd229e00640 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c279b270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00cc40 tx=0x7fd21c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71167ad3b15b77e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd222a00640 1 -- 172.21.15.3:0/3595761842 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd2140cf300 con 0x7fd22c1048f0 2024-03-21T08:28:44.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd22a800640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd2100a2fa0 0x7fd2100a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd21b000640 1 -- 172.21.15.3:0/3555733761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c014070 con 0x55f7c2796770 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd21b000640 1 -- 172.21.15.3:0/3555733761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd21c0054d0 con 0x55f7c2796770 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd21b000640 1 -- 172.21.15.3:0/3555733761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c004cf0 con 0x55f7c2796770 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd233642740 1 -- 172.21.15.3:0/3555733761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 msgr2=0x55f7c279b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd22a800640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd2100a2fa0 0x7fd2100a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd22402c640 tx=0x7fd224004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.360+0000 7fd233642740 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c279b270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00cc40 tx=0x7fd21c00e720 comp rx=0 tx=0).stop 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 -- 172.21.15.3:0/3555733761 shutdown_connections 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c279dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c279b270 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 --2- 172.21.15.3:0/3555733761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2795d80 0x55f7c27961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 -- 172.21.15.3:0/3555733761 >> 172.21.15.3:0/3555733761 conn(0x55f7c27912f0 msgr2=0x55f7c27916f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 -- 172.21.15.3:0/3555733761 shutdown_connections 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 -- 172.21.15.3:0/3555733761 wait complete. 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 -- start start 2024-03-21T08:28:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.361+0000 7fd233642740 1 Processor -- start 2024-03-21T08:28:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd233642740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd233642740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 0x55f7c27a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd233642740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd233642740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c279e890 con 0x55f7c2795d80 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd233642740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c26b6dd0 con 0x55f7c279b8e0 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd233642740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c279ea10 con 0x55f7c2796770 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd233642740 10 client.0 _ll_register_callbacks cb 0x55f7c240eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58044/0 (socket says 172.21.15.3:58044) 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 -- 172.21.15.3:0/1832547239 learned_addr learned my addr 172.21.15.3:0/1832547239 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd22a800640 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 -- 172.21.15.3:0/1832547239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 msgr2=0x55f7c27a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 0x55f7c27a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 -- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 msgr2=0x55f7c27a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.362+0000 7fd229e00640 1 -- 172.21.15.3:0/1832547239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7c240f2d0 con 0x55f7c2795d80 2024-03-21T08:28:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd229e00640 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a0290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd21c000c00 tx=0x7fd21c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff12c64a56fed35 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd20f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd20f400640 1 -- 172.21.15.3:0/1832547239 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c014040 con 0x55f7c2795d80 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd20f400640 1 -- 172.21.15.3:0/1832547239 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd21c010430 con 0x55f7c2795d80 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd21c0054d0 con 0x55f7c2795d80 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd20f400640 1 -- 172.21.15.3:0/1832547239 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd21c005320 con 0x55f7c2795d80 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd233642740 10 client.14775 fetch_fsmap learned FSMap version 16 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x55f7c24d96b0 con 0x55f7c2795d80 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.363+0000 7fd20f400640 1 -- 172.21.15.3:0/1832547239 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd21c0054d0 con 0x55f7c2795d80 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd20f400640 1 -- 172.21.15.3:0/1832547239 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 16) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd21c018410 con 0x55f7c2795d80 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd233642740 10 client.14775 fetch_fsmap finished waiting for FSMap version 16 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd233642740 10 client.14775 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd233642740 20 client.14775 populate_metadata read hostname 'smithi003' 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd233642740 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c27ac870 0x55f7c27aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd233642740 4 client.14775 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:28:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7c269c520 con 0x55f7c27ac870 2024-03-21T08:28:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.364+0000 7fd223e00640 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c27ac870 0x55f7c27aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.365+0000 7fd223e00640 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c27ac870 0x55f7c27aba80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c32810ffea9bc9 server_cookie=eeda9eecea88fea6 in_seq=0 out_seq=0 2024-03-21T08:28:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.365+0000 7fd20f400640 10 client.14775 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd20f400640 1 -- 172.21.15.3:0/1832547239 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f7c269c520 con 0x55f7c27ac870 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd20f400640 10 client.14775 handle_command_reply: tid=0 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 client.14775 shutdown 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 msgr2=0x55f7c27a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a0290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd21c000c00 tx=0x7fd21c005230 comp rx=0 tx=0).stop 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 shutdown_connections 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c27ac870 0x55f7c27aba80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 0x55f7c27a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 --2- 172.21.15.3:0/1832547239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a0290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd20f400640 0 client.14775 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:28:44.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 >> 172.21.15.3:0/1832547239 conn(0x55f7c27912f0 msgr2=0x55f7c27904f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:44.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 shutdown_connections 2024-03-21T08:28:44.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 1 -- 172.21.15.3:0/1832547239 wait complete. 2024-03-21T08:28:44.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.368+0000 7fd233642740 20 client.14775 trim_cache size 0 max 16384 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 Processor -- start 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 -- start start 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c2795d80 0x55f7c2797c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c2798170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c27ac090 con 0x55f7c2796770 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c27ac6f0 con 0x55f7c2795d80 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd233642740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c27acd50 con 0x55f7c279b8e0 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.386+0000 7fd223e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c2798170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c2798170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58060/0 (socket says 172.21.15.3:58060) 2024-03-21T08:28:44.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 -- 172.21.15.3:0/3116005040 learned_addr learned my addr 172.21.15.3:0/3116005040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd229e00640 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c2795d80 0x55f7c2797c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 -- 172.21.15.3:0/3116005040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c279b8e0 msgr2=0x55f7c27a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd22a800640 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 -- 172.21.15.3:0/3116005040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c2795d80 msgr2=0x55f7c2797c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c2795d80 0x55f7c2797c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 -- 172.21.15.3:0/3116005040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7c27abdf0 con 0x55f7c2796770 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd22a800640 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.387+0000 7fd223e00640 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c2798170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd21409b780 tx=0x7fd21400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee2d8eca2967a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd21a600640 1 -- 172.21.15.3:0/3116005040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214020040 con 0x55f7c2796770 2024-03-21T08:28:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd21a600640 1 -- 172.21.15.3:0/3116005040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd214009de0 con 0x55f7c2796770 2024-03-21T08:28:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd21a600640 1 -- 172.21.15.3:0/3116005040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214099af0 con 0x55f7c2796770 2024-03-21T08:28:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 -- 172.21.15.3:0/3116005040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 msgr2=0x55f7c2798170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c2798170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd21409b780 tx=0x7fd21400bf60 comp rx=0 tx=0).stop 2024-03-21T08:28:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 -- 172.21.15.3:0/3116005040 shutdown_connections 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c279b8e0 0x55f7c27a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2796770 0x55f7c2798170 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 --2- 172.21.15.3:0/3116005040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c2795d80 0x55f7c2797c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 -- 172.21.15.3:0/3116005040 >> 172.21.15.3:0/3116005040 conn(0x55f7c27912f0 msgr2=0x55f7c27916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.388+0000 7fd233642740 1 -- 172.21.15.3:0/3116005040 shutdown_connections 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 -- 172.21.15.3:0/3116005040 wait complete. 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 -- start start 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 Processor -- start 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 0x55f7c27a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c27ada10 con 0x55f7c2795d80 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c27ad3b0 con 0x55f7c279b8e0 2024-03-21T08:28:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c27ae070 con 0x55f7c2796770 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.389+0000 7fd233642740 10 client.0 _ll_register_callbacks cb 0x7fd21c00ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58068/0 (socket says 172.21.15.3:58068) 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 -- 172.21.15.3:0/1978498478 learned_addr learned my addr 172.21.15.3:0/1978498478 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd223e00640 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 0x55f7c27a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 -- 172.21.15.3:0/1978498478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 msgr2=0x55f7c27a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd22a800640 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 0x55f7c27a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 -- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 msgr2=0x55f7c27a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 -- 172.21.15.3:0/1978498478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7c27a6cc0 con 0x55f7c2795d80 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd229e00640 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a3170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd21c005b70 tx=0x7fd21c018410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a64b71aeb0f51e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.390+0000 7fd20f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:28:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd20f400640 1 -- 172.21.15.3:0/1978498478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0054b0 con 0x55f7c2795d80 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd20f400640 1 -- 172.21.15.3:0/1978498478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd21c005650 con 0x55f7c2795d80 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd214009de0 con 0x55f7c2795d80 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd20f400640 1 -- 172.21.15.3:0/1978498478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd21c03b050 con 0x55f7c2795d80 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 10 client.14781 fetch_fsmap learned FSMap version 16 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x55f7c2807a30 con 0x55f7c2795d80 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd20f400640 1 -- 172.21.15.3:0/1978498478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd214009de0 con 0x55f7c2795d80 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd20f400640 1 -- 172.21.15.3:0/1978498478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 16) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd21c032070 con 0x55f7c2795d80 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 10 client.14781 fetch_fsmap finished waiting for FSMap version 16 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 10 client.14781 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 20 client.14781 populate_metadata read hostname 'smithi003' 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c280c0d0 0x55f7c2807cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 4 client.14781 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:28:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.391+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f7c27081d0 con 0x55f7c280c0d0 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.392+0000 7fd223e00640 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c280c0d0 0x55f7c2807cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.393+0000 7fd223e00640 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c280c0d0 0x55f7c2807cb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301c110bc4d6c61e server_cookie=1defdfdee6e633d2 in_seq=0 out_seq=0 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.393+0000 7fd20f400640 10 client.14781 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd20f400640 1 -- 172.21.15.3:0/1978498478 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f7c27081d0 con 0x55f7c280c0d0 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd20f400640 10 client.14781 handle_command_reply: tid=0 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 client.14781 shutdown 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 msgr2=0x55f7c27a3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a3170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd21c005b70 tx=0x7fd21c018410 comp rx=0 tx=0).stop 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 shutdown_connections 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7c279b8e0 0x55f7c27a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55f7c280c0d0 0x55f7c2807cb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f7c2796770 0x55f7c27a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 --2- 172.21.15.3:0/1978498478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7c2795d80 0x55f7c27a3170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 >> 172.21.15.3:0/1978498478 conn(0x55f7c27912f0 msgr2=0x55f7c2794550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd20f400640 0 client.14781 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 shutdown_connections 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 1 -- 172.21.15.3:0/1978498478 wait complete. 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.394+0000 7fd233642740 20 client.14781 trim_cache size 0 max 16384 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-03-21T08:28:44.398 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 34472, 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 7.6866521629999998, 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 16, 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 92, 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 92, 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 28.528929731000002 2024-03-21T08:28:44.399 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd2100a2fa0 msgr2=0x7fd2100a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd2100a2fa0 0x7fd2100a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd22402c640 tx=0x7fd224004a40 comp rx=0 tx=0).stop 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1048f0 msgr2=0x7fd22c199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1048f0 0x7fd22c199d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd21400d950 tx=0x7fd21400de10 comp rx=0 tx=0).stop 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 shutdown_connections 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd22c1052b0 0x7fd22c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd22c1048f0 0x7fd22c199d90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd2100a2fa0 0x7fd2100a5460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 --2- 172.21.15.3:0/3595761842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd22c103700 0x7fd22c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 >> 172.21.15.3:0/3595761842 conn(0x7fd22c0fec70 msgr2=0x7fd22c107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 shutdown_connections 2024-03-21T08:28:44.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:44.395+0000 7fd22b200640 1 -- 172.21.15.3:0/3595761842 wait complete. 2024-03-21T08:28:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:44 smithi003 ceph-mon[31675]: pgmap v371: 129 pgs: 10 creating+peering, 110 active+clean, 9 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-03-21T08:28:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:44 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3936658352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:44 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1968838625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:44.412 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T08:28:44.412 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:44.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T08:28:44.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:44 smithi084 ceph-mon[35477]: pgmap v371: 129 pgs: 10 creating+peering, 110 active+clean, 9 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-03-21T08:28:44.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:44 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3936658352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:44 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1968838625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:44.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:44 smithi171 ceph-mon[35388]: pgmap v371: 129 pgs: 10 creating+peering, 110 active+clean, 9 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-03-21T08:28:44.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:44 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3936658352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:28:44.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:44 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1968838625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:44.826 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:28:46.644 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:46 smithi003 ceph-mon[31675]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-21T08:28:46.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:46 smithi084 ceph-mon[35477]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-21T08:28:46.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:46 smithi171 ceph-mon[35388]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-21T08:28:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 -- 172.21.15.3:0/132400726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac1044d0 msgr2=0x7fefac1048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 --2- 172.21.15.3:0/132400726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac1044d0 0x7fefac1048b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00b0d0 tx=0x7fef9c00b3e0 comp rx=0 tx=0).stop 2024-03-21T08:28:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 -- 172.21.15.3:0/132400726 shutdown_connections 2024-03-21T08:28:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 --2- 172.21.15.3:0/132400726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac1044d0 0x7fefac1048b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 --2- 172.21.15.3:0/132400726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac1095b0 0x7fefac111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 --2- 172.21.15.3:0/132400726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefac104e80 0x7fefac108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 -- 172.21.15.3:0/132400726 >> 172.21.15.3:0/132400726 conn(0x7fefac0ffd40 msgr2=0x7fefac102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 -- 172.21.15.3:0/132400726 shutdown_connections 2024-03-21T08:28:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.919+0000 7fefb0a00700 1 -- 172.21.15.3:0/132400726 wait complete. 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.920+0000 7fefb0a00700 1 Processor -- start 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.920+0000 7fefb0a00700 1 -- start start 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefb0a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefac1044d0 0x7fefac1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefb0a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac104e80 0x7fefac1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefb0a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac1095b0 0x7fefac19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefb0a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac114490 con 0x7fefac104e80 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefb0a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac114310 con 0x7fefac1095b0 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefb0a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac114610 con 0x7fefac1044d0 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefa1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac104e80 0x7fefac1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefa1e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac104e80 0x7fefac1a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58072/0 (socket says 172.21.15.3:58072) 2024-03-21T08:28:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefa1e00700 1 -- 172.21.15.3:0/207042078 learned_addr learned my addr 172.21.15.3:0/207042078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefaa800700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefac1044d0 0x7fefac1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.921+0000 7fefab200700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac1095b0 0x7fefac19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa1e00700 1 -- 172.21.15.3:0/207042078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefac1044d0 msgr2=0x7fefac1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa1e00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefac1044d0 0x7fefac1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa1e00700 1 -- 172.21.15.3:0/207042078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac1095b0 msgr2=0x7fefac19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa1e00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac1095b0 0x7fefac19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa1e00700 1 -- 172.21.15.3:0/207042078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac19aff0 con 0x7fefac104e80 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefaa800700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefac1044d0 0x7fefac1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa1e00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac104e80 0x7fefac1a0b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef9400c8f0 tx=0x7fef9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa3e00700 1 -- 172.21.15.3:0/207042078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94007b00 con 0x7fefac104e80 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.922+0000 7fefa3e00700 1 -- 172.21.15.3:0/207042078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef94007ca0 con 0x7fefac104e80 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.923+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac19b2e0 con 0x7fefac104e80 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.923+0000 7fefa3e00700 1 -- 172.21.15.3:0/207042078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94018660 con 0x7fefac104e80 2024-03-21T08:28:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.923+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac1a74a0 con 0x7fefac104e80 2024-03-21T08:28:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.924+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac1059b0 con 0x7fefac104e80 2024-03-21T08:28:46.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.926+0000 7fefa3e00700 1 -- 172.21.15.3:0/207042078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fef9400f450 con 0x7fefac104e80 2024-03-21T08:28:46.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.927+0000 7fefa3e00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef900a2fb0 0x7fef900a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:46.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.928+0000 7fefa3e00700 1 -- 172.21.15.3:0/207042078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fef940cf670 con 0x7fefac104e80 2024-03-21T08:28:46.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.928+0000 7fefaa800700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef900a2fb0 0x7fef900a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:46.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.928+0000 7fefaa800700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef900a2fb0 0x7fef900a5470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00d010 tx=0x7fef9c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:46.931+0000 7fefa3e00700 1 -- 172.21.15.3:0/207042078 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef94014030 con 0x7fefac104e80 2024-03-21T08:28:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.210+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fefac02da30 con 0x7fefac104e80 2024-03-21T08:28:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.211+0000 7fefa3e00700 1 -- 172.21.15.3:0/207042078 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7fef9409c200 con 0x7fefac104e80 2024-03-21T08:28:47.212 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T08:28:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef900a2fb0 msgr2=0x7fef900a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef900a2fb0 0x7fef900a5470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00d010 tx=0x7fef9c00b040 comp rx=0 tx=0).stop 2024-03-21T08:28:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac104e80 msgr2=0x7fefac1a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac104e80 0x7fefac1a0b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef9400c8f0 tx=0x7fef9400ccb0 comp rx=0 tx=0).stop 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 shutdown_connections 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fef900a2fb0 0x7fef900a5470 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac1095b0 0x7fefac19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefac1044d0 0x7fefac1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 --2- 172.21.15.3:0/207042078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefac104e80 0x7fefac1a0b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 >> 172.21.15.3:0/207042078 conn(0x7fefac0ffd40 msgr2=0x7fefac101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 shutdown_connections 2024-03-21T08:28:47.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:47.214+0000 7fefb0a00700 1 -- 172.21.15.3:0/207042078 wait complete. 2024-03-21T08:28:47.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:47 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/207042078' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:28:47.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:47 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/207042078' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:28:47.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:47 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/207042078' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:28:47.990 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-21T08:28:47.990 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T08:28:47.991 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-03-21T08:28:47.991 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-21T08:28:47.991 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:47.991 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-21T08:28:47.991 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T08:28:47.991 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T08:28:47.991 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T08:28:47.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:47.991 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-03-21T08:28:48.053 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T08:28:48.053 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T08:28:48.053 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T08:28:48.053 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:28:48.054 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-03-21T08:28:48.054 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:28:48.054 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:28:48.054 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-03-21T08:28:48.054 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-03-21T08:28:48.054 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:28:48.054 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-03-21T08:28:48.055 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-03-21T08:28:48.055 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T08:28:48.055 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-03-21T08:28:48.055 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-03-21T08:28:48.055 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T08:28:48.055 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41881sec preferred_lft 41881sec 2024-03-21T08:28:48.055 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-03-21T08:28:48.056 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:28:48.056 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:28:48.056 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-03-21T08:28:48.056 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-03-21T08:28:48.058 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T08:28:48.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:48.059 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:28:48.059 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:28:48.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-03-21T08:28:48.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-03-21T08:28:48.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-03-21T08:28:48.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T08:28:48.059 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:28:48.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:48 smithi003 ceph-mon[31675]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 933 B/s wr, 0 op/s 2024-03-21T08:28:48.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:48.503 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T08:28:48.589 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-03-21T08:28:48.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:48.593 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-03-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-03-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T08:28:48.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:48.665 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:28:48.665 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:28:48.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T08:28:48.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T08:28:48.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T08:28:48.666 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:28:48.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:48 smithi171 ceph-mon[35388]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 933 B/s wr, 0 op/s 2024-03-21T08:28:48.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:48 smithi084 ceph-mon[35477]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 933 B/s wr, 0 op/s 2024-03-21T08:28:49.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:49.415 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-03-21T08:28:49.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:49.475 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-03-21T08:28:49.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:49.535 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:28:49.536 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:28:49.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:28:49.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-21T08:28:49.536 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:28:49.687 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-21T08:28:49.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:49.688 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:28:49.688 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:28:49.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-21T08:28:49.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T08:28:49.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-21T08:28:49.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-21T08:28:49.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-21T08:28:49.689 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:28:50.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:50.022 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:28:50.022 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:28:50.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-03-21T08:28:50.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T08:28:50.022 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:28:50.193 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-21T08:28:50.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T08:28:50.193 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:50.241 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T08:28:50.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T08:28:50.241 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:50.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-03-21T08:28:50.394 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-03-21T08:28:50.448 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T08:28:50.449 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/eb1ed2432cfab63a4b72798347886aed47dc3ba69fef7b22c63c56f77899a1db/merged 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7ad661500301612b71fa5ceea623c4b888e83eaff29188df8423fee2ec212813/merged 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/b9244e850b1ee0c649d575f2fdee3f518ddafe0e2b860aa6d628fc386f11ece2/merged 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/ba82b0150aae1b7b22cd0c1a93cc366a0346de48a14645dafbe478a5034d8383/merged 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/2abedc3682b4d2a8a526875f63e14e8c524e036f6675003c7eef2a3f604290b7/merged 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/dc79872ffba797f1b2411b7614686e9c6312bdb341dbbfb58f75e22706e78896/merged 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d52178c390940302fa16cd17170672e4efd32de7adc4d4e17da2b92a351d6929/merged 2024-03-21T08:28:50.450 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7d0f2664a3a6bd2838e50b42462ffe6462e94a514442b1560ad834f4aa3dca76/merged 2024-03-21T08:28:50.451 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7c6b09c1743db251c08db3fbde3e05413fa0445a49aa70cf6e9f1ee920a875ad/merged 2024-03-21T08:28:50.451 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7bc152af8d338f0ee3e0f5f249984bc70707b6c24e83c45803c2ff641c74077d/merged 2024-03-21T08:28:50.451 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-03-21T08:28:50.451 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:28:50.451 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:28:50.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:50.452 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-03-21T08:28:50.504 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T08:28:50.505 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-21T08:28:50.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-03-21T08:28:50.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:50 smithi003 ceph-mon[31675]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 825 B/s wr, 0 op/s 2024-03-21T08:28:50.589 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T08:28:50.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:50 smithi171 ceph-mon[35388]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 825 B/s wr, 0 op/s 2024-03-21T08:28:50.635 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-03-21T08:28:50.635 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-03-21T08:28:50.635 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-03-21T08:28:50.635 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-03-21T08:28:50.635 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:28:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/eb1ed2432cfab63a4b72798347886aed47dc3ba69fef7b22c63c56f77899a1db/merged 2024-03-21T08:28:50.637 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7ad661500301612b71fa5ceea623c4b888e83eaff29188df8423fee2ec212813/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/b9244e850b1ee0c649d575f2fdee3f518ddafe0e2b860aa6d628fc386f11ece2/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/ba82b0150aae1b7b22cd0c1a93cc366a0346de48a14645dafbe478a5034d8383/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/2abedc3682b4d2a8a526875f63e14e8c524e036f6675003c7eef2a3f604290b7/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/dc79872ffba797f1b2411b7614686e9c6312bdb341dbbfb58f75e22706e78896/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d52178c390940302fa16cd17170672e4efd32de7adc4d4e17da2b92a351d6929/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7d0f2664a3a6bd2838e50b42462ffe6462e94a514442b1560ad834f4aa3dca76/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7c6b09c1743db251c08db3fbde3e05413fa0445a49aa70cf6e9f1ee920a875ad/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7bc152af8d338f0ee3e0f5f249984bc70707b6c24e83c45803c2ff641c74077d/merged 2024-03-21T08:28:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-03-21T08:28:50.639 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:28:50.639 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:28:50.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:50.640 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-03-21T08:28:50.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:2024-03-21T08:28:50.710+0000 7f6ae360b480 -1 init, newargv = 0x55cbce6e8eb0 newargc=15 2024-03-21T08:28:50.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[75566]: starting ceph client 2024-03-21T08:28:50.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[75566]: starting fuse 2024-03-21T08:28:50.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:50 smithi084 ceph-mon[35477]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 825 B/s wr, 0 op/s 2024-03-21T08:28:51.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-03-21T08:28:51.891 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-03-21T08:28:51.953 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T08:28:51.954 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/eb1ed2432cfab63a4b72798347886aed47dc3ba69fef7b22c63c56f77899a1db/merged 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7ad661500301612b71fa5ceea623c4b888e83eaff29188df8423fee2ec212813/merged 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/b9244e850b1ee0c649d575f2fdee3f518ddafe0e2b860aa6d628fc386f11ece2/merged 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/ba82b0150aae1b7b22cd0c1a93cc366a0346de48a14645dafbe478a5034d8383/merged 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/2abedc3682b4d2a8a526875f63e14e8c524e036f6675003c7eef2a3f604290b7/merged 2024-03-21T08:28:51.955 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/dc79872ffba797f1b2411b7614686e9c6312bdb341dbbfb58f75e22706e78896/merged 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d52178c390940302fa16cd17170672e4efd32de7adc4d4e17da2b92a351d6929/merged 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7d0f2664a3a6bd2838e50b42462ffe6462e94a514442b1560ad834f4aa3dca76/merged 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7c6b09c1743db251c08db3fbde3e05413fa0445a49aa70cf6e9f1ee920a875ad/merged 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7bc152af8d338f0ee3e0f5f249984bc70707b6c24e83c45803c2ff641c74077d/merged 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:28:51.956 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:51.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:51.957 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-03-21T08:28:52.015 INFO:teuthology.orchestra.run.smithi003.stdout:118 2024-03-21T08:28:52.015 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-21T08:28:52.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-03-21T08:28:52.015 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-03-21T08:28:52.015 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-03-21T08:28:52.015 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-03-21T08:28:52.016 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.admin")) 2024-03-21T08:28:52.017 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-03-21T08:28:52.206 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.admin.75566.asok 2024-03-21T08:28:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T08:28:52.224 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75566.asok 2024-03-21T08:28:52.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:52.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75566.asok status 2024-03-21T08:28:52.474 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:28:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-03-21T08:28:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "2edfee68ad7243bed422a049b6f5f58619300f11", 2024-03-21T08:28:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:28:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "admin", 2024-03-21T08:28:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-03-21T08:28:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "75566", 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 14793, 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 14793 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-03-21T08:28:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2682162918 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2682162918 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.14793 192.168.0.1:0/2682162918", 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2682162918", 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 16, 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 92, 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-03-21T08:28:52.476 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:28:52.485 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:52 smithi003 ceph-mon[31675]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:52.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:52.490 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-03-21T08:28:52.490 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:52.544 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-03-21T08:28:52.544 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:52.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:52.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:52.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:52.653 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-21T08:28:52.748 INFO:teuthology.orchestra.run.smithi003.stdout:# file: . 2024-03-21T08:28:52.749 INFO:teuthology.orchestra.run.smithi003.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-21T08:28:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:52.750 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:28:52.750 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T08:28:52.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:28:52.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:52 smithi084 ceph-mon[35477]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:52.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:52 smithi171 ceph-mon[35388]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.039+0000 7f09a7400640 1 -- 172.21.15.3:0/1683312265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0109890 msgr2=0x7f09a0110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.039+0000 7f09a7400640 1 --2- 172.21.15.3:0/1683312265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0109890 0x7f09a0110120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0998009900 tx=0x7f099802bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.040+0000 7f09a7400640 1 -- 172.21.15.3:0/1683312265 shutdown_connections 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.040+0000 7f09a7400640 1 --2- 172.21.15.3:0/1683312265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0109890 0x7f09a0110120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.040+0000 7f09a7400640 1 --2- 172.21.15.3:0/1683312265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a0108ed0 0x7f09a0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.040+0000 7f09a7400640 1 --2- 172.21.15.3:0/1683312265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09a0107ce0 0x7f09a0108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.040+0000 7f09a7400640 1 -- 172.21.15.3:0/1683312265 >> 172.21.15.3:0/1683312265 conn(0x7f09a0076ca0 msgr2=0x7f09a00790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.040+0000 7f09a7400640 1 -- 172.21.15.3:0/1683312265 shutdown_connections 2024-03-21T08:28:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.040+0000 7f09a7400640 1 -- 172.21.15.3:0/1683312265 wait complete. 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 Processor -- start 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 -- start start 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0107ce0 0x7f09a019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09a0108ed0 0x7f09a019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a0109890 0x7f09a01a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a0113860 con 0x7f09a0107ce0 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a01136e0 con 0x7f09a0109890 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.041+0000 7f09a7400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a01139e0 con 0x7f09a0108ed0 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0107ce0 0x7f09a019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0107ce0 0x7f09a019de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60874/0 (socket says 172.21.15.3:60874) 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 -- 172.21.15.3:0/2277485519 learned_addr learned my addr 172.21.15.3:0/2277485519 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f099fe00640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09a0108ed0 0x7f09a019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a5400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a0109890 0x7f09a01a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 -- 172.21.15.3:0/2277485519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09a0108ed0 msgr2=0x7f09a019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09a0108ed0 0x7f09a019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 -- 172.21.15.3:0/2277485519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a0109890 msgr2=0x7f09a01a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a0109890 0x7f09a01a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 -- 172.21.15.3:0/2277485519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a01a5a90 con 0x7f09a0107ce0 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a5400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a0109890 0x7f09a01a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.042+0000 7f09a4a00640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0107ce0 0x7f09a019de70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f099000eb70 tx=0x7f099000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:53.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.043+0000 7f099d600640 1 -- 172.21.15.3:0/2277485519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990003b70 con 0x7f09a0107ce0 2024-03-21T08:28:53.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.043+0000 7f099d600640 1 -- 172.21.15.3:0/2277485519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f099000ee70 con 0x7f09a0107ce0 2024-03-21T08:28:53.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.043+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a01a5d80 con 0x7f09a0107ce0 2024-03-21T08:28:53.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.043+0000 7f099d600640 1 -- 172.21.15.3:0/2277485519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990010650 con 0x7f09a0107ce0 2024-03-21T08:28:53.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.043+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a01a61c0 con 0x7f09a0107ce0 2024-03-21T08:28:53.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.044+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a010f000 con 0x7f09a0107ce0 2024-03-21T08:28:53.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.046+0000 7f099d600640 1 -- 172.21.15.3:0/2277485519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f0990004020 con 0x7f09a0107ce0 2024-03-21T08:28:53.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.048+0000 7f099d600640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f09840a2f20 0x7f09840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.048+0000 7f099d600640 1 -- 172.21.15.3:0/2277485519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f09900cfe50 con 0x7f09a0107ce0 2024-03-21T08:28:53.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.048+0000 7f099fe00640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f09840a2f20 0x7f09840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.049+0000 7f099fe00640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f09840a2f20 0x7f09840a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09a019f380 tx=0x7f098800b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:53.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.052+0000 7f099d600640 1 -- 172.21.15.3:0/2277485519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f099009c9e0 con 0x7f09a0107ce0 2024-03-21T08:28:53.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.260+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09a00623f0 con 0x7f09a0107ce0 2024-03-21T08:28:53.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.263+0000 7f099d600640 1 -- 172.21.15.3:0/2277485519 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24035 (secure 0 0 0) 0x7f099009c2e0 con 0x7f09a0107ce0 2024-03-21T08:28:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:53.265 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:28:41.154980+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:28:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f09840a2f20 msgr2=0x7f09840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f09840a2f20 0x7f09840a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09a019f380 tx=0x7f098800b590 comp rx=0 tx=0).stop 2024-03-21T08:28:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0107ce0 msgr2=0x7f09a019de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0107ce0 0x7f09a019de70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f099000eb70 tx=0x7f099000c6a0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 shutdown_connections 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a0109890 0x7f09a01a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f09840a2f20 0x7f09840a53e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09a0108ed0 0x7f09a019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 --2- 172.21.15.3:0/2277485519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0107ce0 0x7f09a019de70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 >> 172.21.15.3:0/2277485519 conn(0x7f09a0076ca0 msgr2=0x7f09a010c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.265+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 shutdown_connections 2024-03-21T08:28:53.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.266+0000 7f09a7400640 1 -- 172.21.15.3:0/2277485519 wait complete. 2024-03-21T08:28:53.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:28:53.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:28:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.520+0000 7fa171000640 1 -- 172.21.15.3:0/3380543888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c1016a0 msgr2=0x7fa16c101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.520+0000 7fa171000640 1 --2- 172.21.15.3:0/3380543888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c1016a0 0x7fa16c101ac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009ca0 tx=0x7fa15c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.521+0000 7fa171000640 1 -- 172.21.15.3:0/3380543888 shutdown_connections 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.521+0000 7fa171000640 1 --2- 172.21.15.3:0/3380543888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa16c103250 0x7fa16c109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.521+0000 7fa171000640 1 --2- 172.21.15.3:0/3380543888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa16c102890 0x7fa16c102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.521+0000 7fa171000640 1 --2- 172.21.15.3:0/3380543888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c1016a0 0x7fa16c101ac0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.521+0000 7fa171000640 1 -- 172.21.15.3:0/3380543888 >> 172.21.15.3:0/3380543888 conn(0x7fa16c0fcbf0 msgr2=0x7fa16c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.521+0000 7fa171000640 1 -- 172.21.15.3:0/3380543888 shutdown_connections 2024-03-21T08:28:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.521+0000 7fa171000640 1 -- 172.21.15.3:0/3380543888 wait complete. 2024-03-21T08:28:53.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 Processor -- start 2024-03-21T08:28:53.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 -- start start 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa16c1016a0 0x7fa16c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa16c102890 0x7fa16c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c103250 0x7fa16c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c10d1a0 con 0x7fa16c103250 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c10d020 con 0x7fa16c1016a0 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.522+0000 7fa171000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c10d320 con 0x7fa16c102890 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c103250 0x7fa16c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c103250 0x7fa16c1a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60892/0 (socket says 172.21.15.3:60892) 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 -- 172.21.15.3:0/2952858256 learned_addr learned my addr 172.21.15.3:0/2952858256 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa169e00640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa16c1016a0 0x7fa16c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa169400640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa16c102890 0x7fa16c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 -- 172.21.15.3:0/2952858256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa16c102890 msgr2=0x7fa16c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa16c102890 0x7fa16c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 -- 172.21.15.3:0/2952858256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa16c1016a0 msgr2=0x7fa16c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa16c1016a0 0x7fa16c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.523+0000 7fa16a800640 1 -- 172.21.15.3:0/2952858256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c1a1760 con 0x7fa16c103250 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.524+0000 7fa16a800640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c103250 0x7fa16c1a10f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c970 tx=0x7fa16400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.524+0000 7fa162a00640 1 -- 172.21.15.3:0/2952858256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164007bb0 con 0x7fa16c103250 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.524+0000 7fa162a00640 1 -- 172.21.15.3:0/2952858256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa164007d50 con 0x7fa16c103250 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.524+0000 7fa162a00640 1 -- 172.21.15.3:0/2952858256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164004e10 con 0x7fa16c103250 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.524+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c1a1a50 con 0x7fa16c103250 2024-03-21T08:28:53.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.524+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c1a1f50 con 0x7fa16c103250 2024-03-21T08:28:53.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.525+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c1a21e0 con 0x7fa16c103250 2024-03-21T08:28:53.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.528+0000 7fa162a00640 1 -- 172.21.15.3:0/2952858256 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fa164005000 con 0x7fa16c103250 2024-03-21T08:28:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.529+0000 7fa162a00640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa1500a2f20 0x7fa1500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.529+0000 7fa162a00640 1 -- 172.21.15.3:0/2952858256 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa1640cf820 con 0x7fa16c103250 2024-03-21T08:28:53.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.529+0000 7fa169e00640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa1500a2f20 0x7fa1500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.530+0000 7fa169e00640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa1500a2f20 0x7fa1500a53e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa15c02c630 tx=0x7fa15c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:53.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.532+0000 7fa162a00640 1 -- 172.21.15.3:0/2952858256 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa164014030 con 0x7fa16c103250 2024-03-21T08:28:53.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:53 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2277485519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:53.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.735+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa16c1a23b0 con 0x7fa16c103250 2024-03-21T08:28:53.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.736+0000 7fa162a00640 1 -- 172.21.15.3:0/2952858256 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa16409c3b0 con 0x7fa16c103250 2024-03-21T08:28:53.738 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa1500a2f20 msgr2=0x7fa1500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa1500a2f20 0x7fa1500a53e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa15c02c630 tx=0x7fa15c000f40 comp rx=0 tx=0).stop 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c103250 msgr2=0x7fa16c1a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c103250 0x7fa16c1a10f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c970 tx=0x7fa16400ce30 comp rx=0 tx=0).stop 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 shutdown_connections 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa16c103250 0x7fa16c1a10f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa1500a2f20 0x7fa1500a53e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa16c102890 0x7fa16c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 --2- 172.21.15.3:0/2952858256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa16c1016a0 0x7fa16c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 >> 172.21.15.3:0/2952858256 conn(0x7fa16c0fcbf0 msgr2=0x7fa16c108080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 shutdown_connections 2024-03-21T08:28:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.739+0000 7fa171000640 1 -- 172.21.15.3:0/2952858256 wait complete. 2024-03-21T08:28:53.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:28:53.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:53 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2277485519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:53.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:53 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2277485519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:53.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 -- 172.21.15.3:0/3870592930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc101f70 msgr2=0x7fd5cc104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 --2- 172.21.15.3:0/3870592930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc101f70 0x7fd5cc104360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009900 tx=0x7fd5c002bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:53.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 -- 172.21.15.3:0/3870592930 shutdown_connections 2024-03-21T08:28:53.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 --2- 172.21.15.3:0/3870592930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc1071d0 0x7fd5cc109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 --2- 172.21.15.3:0/3870592930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5cc1048a0 0x7fd5cc106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 --2- 172.21.15.3:0/3870592930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc101f70 0x7fd5cc104360 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 -- 172.21.15.3:0/3870592930 >> 172.21.15.3:0/3870592930 conn(0x7fd5cc0fb980 msgr2=0x7fd5cc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:53.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 -- 172.21.15.3:0/3870592930 shutdown_connections 2024-03-21T08:28:53.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.982+0000 7fd5d2800640 1 -- 172.21.15.3:0/3870592930 wait complete. 2024-03-21T08:28:53.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.983+0000 7fd5d2800640 1 Processor -- start 2024-03-21T08:28:53.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.983+0000 7fd5d2800640 1 -- start start 2024-03-21T08:28:53.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc101f70 0x7fd5cc197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc1048a0 0x7fd5cc197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5cc1071d0 0x7fd5cc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d2800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc10d080 con 0x7fd5cc1048a0 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d2800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc10cf00 con 0x7fd5cc101f70 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d2800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc10d200 con 0x7fd5cc1071d0 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc1048a0 0x7fd5cc197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc1048a0 0x7fd5cc197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60918/0 (socket says 172.21.15.3:60918) 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d0a00640 1 -- 172.21.15.3:0/1865696298 learned_addr learned my addr 172.21.15.3:0/1865696298 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d1400640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc101f70 0x7fd5cc197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.984+0000 7fd5d1e00640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5cc1071d0 0x7fd5cc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5d0a00640 1 -- 172.21.15.3:0/1865696298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5cc1071d0 msgr2=0x7fd5cc19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5d0a00640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5cc1071d0 0x7fd5cc19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5d0a00640 1 -- 172.21.15.3:0/1865696298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc101f70 msgr2=0x7fd5cc197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5d0a00640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc101f70 0x7fd5cc197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5d0a00640 1 -- 172.21.15.3:0/1865696298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc19f380 con 0x7fd5cc1048a0 2024-03-21T08:28:53.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5d0a00640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc1048a0 0x7fd5cc197ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9c0 tx=0x7fd5b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:53.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5ca000640 1 -- 172.21.15.3:0/1865696298 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800cd80 con 0x7fd5cc1048a0 2024-03-21T08:28:53.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5ca000640 1 -- 172.21.15.3:0/1865696298 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5b80054d0 con 0x7fd5cc1048a0 2024-03-21T08:28:53.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5cc19f670 con 0x7fd5cc1048a0 2024-03-21T08:28:53.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.985+0000 7fd5ca000640 1 -- 172.21.15.3:0/1865696298 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8010430 con 0x7fd5cc1048a0 2024-03-21T08:28:53.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.986+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc19fbc0 con 0x7fd5cc1048a0 2024-03-21T08:28:53.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.987+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5cc0fd470 con 0x7fd5cc1048a0 2024-03-21T08:28:53.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.990+0000 7fd5ca000640 1 -- 172.21.15.3:0/1865696298 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd5b8004fd0 con 0x7fd5cc1048a0 2024-03-21T08:28:53.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.991+0000 7fd5ca000640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd5b40a2f20 0x7fd5b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:53.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.991+0000 7fd5ca000640 1 -- 172.21.15.3:0/1865696298 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd5b80cfc10 con 0x7fd5cc1048a0 2024-03-21T08:28:53.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.992+0000 7fd5d1400640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd5b40a2f20 0x7fd5b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:53.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.993+0000 7fd5d1400640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd5b40a2f20 0x7fd5b40a53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0006010 tx=0x7fd5c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:53.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:53.996+0000 7fd5ca000640 1 -- 172.21.15.3:0/1865696298 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd5b800b2c0 con 0x7fd5cc1048a0 2024-03-21T08:28:54.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.204+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5cc0622a0 con 0x7fd5cc1048a0 2024-03-21T08:28:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.205+0000 7fd5ca000640 1 -- 172.21.15.3:0/1865696298 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5b809c7a0 con 0x7fd5cc1048a0 2024-03-21T08:28:54.207 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd5b40a2f20 msgr2=0x7fd5b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd5b40a2f20 0x7fd5b40a53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0006010 tx=0x7fd5c0000bc0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc1048a0 msgr2=0x7fd5cc197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc1048a0 0x7fd5cc197ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9c0 tx=0x7fd5b800ee80 comp rx=0 tx=0).stop 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 shutdown_connections 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd5b40a2f20 0x7fd5b40a53e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5cc1071d0 0x7fd5cc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5cc1048a0 0x7fd5cc197ca0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 --2- 172.21.15.3:0/1865696298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5cc101f70 0x7fd5cc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 >> 172.21.15.3:0/1865696298 conn(0x7fd5cc0fb980 msgr2=0x7fd5cc1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 shutdown_connections 2024-03-21T08:28:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.208+0000 7fd5d2800640 1 -- 172.21.15.3:0/1865696298 wait complete. 2024-03-21T08:28:54.228 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.456+0000 7f8ec7200640 1 -- 172.21.15.3:0/3070326408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 msgr2=0x7f8ec8107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.456+0000 7f8ec7200640 1 --2- 172.21.15.3:0/3070326408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec8107f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009900 tx=0x7f8eb802bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.457+0000 7f8ec7200640 1 -- 172.21.15.3:0/3070326408 shutdown_connections 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.457+0000 7f8ec7200640 1 --2- 172.21.15.3:0/3070326408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec81096b0 0x7f8ec810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.457+0000 7f8ec7200640 1 --2- 172.21.15.3:0/3070326408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ec8108cf0 0x7f8ec8109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.457+0000 7f8ec7200640 1 --2- 172.21.15.3:0/3070326408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec8107f20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.457+0000 7f8ec7200640 1 -- 172.21.15.3:0/3070326408 >> 172.21.15.3:0/3070326408 conn(0x7f8ec8076a90 msgr2=0x7f8ec8078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.457+0000 7f8ec7200640 1 -- 172.21.15.3:0/3070326408 shutdown_connections 2024-03-21T08:28:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.457+0000 7f8ec7200640 1 -- 172.21.15.3:0/3070326408 wait complete. 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 Processor -- start 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 -- start start 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8108cf0 0x7f8ec819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ec81096b0 0x7f8ec81a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8113600 con 0x7f8ec8107b00 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8113480 con 0x7f8ec81096b0 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.458+0000 7f8ec7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8113780 con 0x7f8ec8108cf0 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60934/0 (socket says 172.21.15.3:60934) 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 -- 172.21.15.3:0/2463561705 learned_addr learned my addr 172.21.15.3:0/2463561705 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:54.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec6800640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ec81096b0 0x7f8ec81a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 -- 172.21.15.3:0/2463561705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8108cf0 msgr2=0x7f8ec819e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8108cf0 0x7f8ec819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 -- 172.21.15.3:0/2463561705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ec81096b0 msgr2=0x7f8ec81a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ec81096b0 0x7f8ec81a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 -- 172.21.15.3:0/2463561705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec81a59b0 con 0x7f8ec8107b00 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.459+0000 7f8ec5e00640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec819dda0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8000c00 tx=0x7f8eb8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.460+0000 7f8ebea00640 1 -- 172.21.15.3:0/2463561705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8043020 con 0x7f8ec8107b00 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.460+0000 7f8ebea00640 1 -- 172.21.15.3:0/2463561705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8eb8004aa0 con 0x7f8ec8107b00 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.460+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec81a5c40 con 0x7f8ec8107b00 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.460+0000 7f8ebea00640 1 -- 172.21.15.3:0/2463561705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8003d00 con 0x7f8ec8107b00 2024-03-21T08:28:54.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.460+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec81a6190 con 0x7f8ec8107b00 2024-03-21T08:28:54.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.461+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec810ef40 con 0x7f8ec8107b00 2024-03-21T08:28:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.463+0000 7f8ebea00640 1 -- 172.21.15.3:0/2463561705 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8eb8046020 con 0x7f8ec8107b00 2024-03-21T08:28:54.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.465+0000 7f8ebea00640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8eac0a2fa0 0x7f8eac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.465+0000 7f8ebea00640 1 -- 172.21.15.3:0/2463561705 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8eb80f1ca0 con 0x7f8ec8107b00 2024-03-21T08:28:54.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.465+0000 7f8ec5400640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8eac0a2fa0 0x7f8eac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:54.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.466+0000 7f8ec5400640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8eac0a2fa0 0x7f8eac0a5460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005fd0 tx=0x7f8eb4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:54.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.469+0000 7f8ebea00640 1 -- 172.21.15.3:0/2463561705 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8eb804d020 con 0x7f8ec8107b00 2024-03-21T08:28:54.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.672+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ec8062340 con 0x7f8ec8107b00 2024-03-21T08:28:54.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.673+0000 7f8ebea00640 1 -- 172.21.15.3:0/2463561705 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8eb80be890 con 0x7f8ec8107b00 2024-03-21T08:28:54.675 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:28:54.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:54 smithi003 ceph-mon[31675]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:54.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:54 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2952858256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:54.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:54 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1865696298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:28:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8eac0a2fa0 msgr2=0x7f8eac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8eac0a2fa0 0x7f8eac0a5460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005fd0 tx=0x7f8eb4005850 comp rx=0 tx=0).stop 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 msgr2=0x7f8ec819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec819dda0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8000c00 tx=0x7f8eb8004270 comp rx=0 tx=0).stop 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 shutdown_connections 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ec81096b0 0x7f8ec81a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8eac0a2fa0 0x7f8eac0a5460 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8108cf0 0x7f8ec819e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 --2- 172.21.15.3:0/2463561705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ec8107b00 0x7f8ec819dda0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.675+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 >> 172.21.15.3:0/2463561705 conn(0x7f8ec8076a90 msgr2=0x7f8ec810c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.676+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 shutdown_connections 2024-03-21T08:28:54.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.676+0000 7f8ec7200640 1 -- 172.21.15.3:0/2463561705 wait complete. 2024-03-21T08:28:54.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:28:54.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:54 smithi084 ceph-mon[35477]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:54.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:54 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2952858256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:54 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1865696298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:28:54.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:54 smithi171 ceph-mon[35388]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:28:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:54 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2952858256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:54 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1865696298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:28:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2762583106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 msgr2=0x7fc1c4104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2762583106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4104530 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009ca0 tx=0x7fc1b402c100 comp rx=0 tx=0).stop 2024-03-21T08:28:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2762583106 shutdown_connections 2024-03-21T08:28:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2762583106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c41073a0 0x7fc1c41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2762583106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1c4104a70 0x7fc1c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2762583106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4104530 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2762583106 >> 172.21.15.3:0/2762583106 conn(0x7fc1c40fbb30 msgr2=0x7fc1c40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2762583106 shutdown_connections 2024-03-21T08:28:54.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.913+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2762583106 wait complete. 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.914+0000 7fc1c9e00640 1 Processor -- start 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.914+0000 7fc1c9e00640 1 -- start start 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.914+0000 7fc1c9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.914+0000 7fc1c9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4104a70 0x7fc1c41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1c41073a0 0x7fc1c4100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c9e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c410d290 con 0x7fc1c4102140 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c9e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c410d110 con 0x7fc1c4104a70 2024-03-21T08:28:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c9e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c410d410 con 0x7fc1c41073a0 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60960/0 (socket says 172.21.15.3:60960) 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 -- 172.21.15.3:0/2842771043 learned_addr learned my addr 172.21.15.3:0/2842771043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2000640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4104a70 0x7fc1c41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c3400640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1c41073a0 0x7fc1c4100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 -- 172.21.15.3:0/2842771043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1c41073a0 msgr2=0x7fc1c4100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1c41073a0 0x7fc1c4100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 -- 172.21.15.3:0/2842771043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4104a70 msgr2=0x7fc1c41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4104a70 0x7fc1c41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c2a00640 1 -- 172.21.15.3:0/2842771043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4100b70 con 0x7fc1c4102140 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.915+0000 7fc1c3400640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1c41073a0 0x7fc1c4100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.916+0000 7fc1c2a00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4101ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4032380 tx=0x7fc1b4035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.916+0000 7fc1c8a00640 1 -- 172.21.15.3:0/2842771043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4039040 con 0x7fc1c4102140 2024-03-21T08:28:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.916+0000 7fc1c8a00640 1 -- 172.21.15.3:0/2842771043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1b4044070 con 0x7fc1c4102140 2024-03-21T08:28:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.916+0000 7fc1c8a00640 1 -- 172.21.15.3:0/2842771043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4004530 con 0x7fc1c4102140 2024-03-21T08:28:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.916+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c4100e00 con 0x7fc1c4102140 2024-03-21T08:28:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.916+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c41a6440 con 0x7fc1c4102140 2024-03-21T08:28:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.917+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40fd620 con 0x7fc1c4102140 2024-03-21T08:28:54.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.919+0000 7fc1c8a00640 1 -- 172.21.15.3:0/2842771043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fc1b40327c0 con 0x7fc1c4102140 2024-03-21T08:28:54.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.921+0000 7fc1c8a00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1a80a2f20 0x7fc1a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.921+0000 7fc1c8a00640 1 -- 172.21.15.3:0/2842771043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc1b40f1ea0 con 0x7fc1c4102140 2024-03-21T08:28:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.921+0000 7fc1c2000640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1a80a2f20 0x7fc1a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:54.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.922+0000 7fc1c2000640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1a80a2f20 0x7fc1a80a53e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0005fd0 tx=0x7fc1b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:54.925+0000 7fc1c8a00640 1 -- 172.21.15.3:0/2842771043 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc1b40bea90 con 0x7fc1c4102140 2024-03-21T08:28:55.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.129+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc1c40622a0 con 0x7fc1c4102140 2024-03-21T08:28:55.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.130+0000 7fc1c8a00640 1 -- 172.21.15.3:0/2842771043 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc1b403ee80 con 0x7fc1c4102140 2024-03-21T08:28:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1a80a2f20 msgr2=0x7fc1a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1a80a2f20 0x7fc1a80a53e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0005fd0 tx=0x7fc1b0005850 comp rx=0 tx=0).stop 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 msgr2=0x7fc1c4101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4101ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4032380 tx=0x7fc1b4035ce0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 shutdown_connections 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc1a80a2f20 0x7fc1a80a53e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1c41073a0 0x7fc1c4100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1c4104a70 0x7fc1c41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 --2- 172.21.15.3:0/2842771043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c4102140 0x7fc1c4101ab0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 >> 172.21.15.3:0/2842771043 conn(0x7fc1c40fbb30 msgr2=0x7fc1c4102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 shutdown_connections 2024-03-21T08:28:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.133+0000 7fc1c9e00640 1 -- 172.21.15.3:0/2842771043 wait complete. 2024-03-21T08:28:55.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T08:28:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.369+0000 7f4f55e00640 1 -- 172.21.15.3:0/4253532969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50101510 msgr2=0x7f4f50101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.369+0000 7f4f55e00640 1 --2- 172.21.15.3:0/4253532969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50101510 0x7f4f50101930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009900 tx=0x7f4f4002bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.370+0000 7f4f55e00640 1 -- 172.21.15.3:0/4253532969 shutdown_connections 2024-03-21T08:28:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.370+0000 7f4f55e00640 1 --2- 172.21.15.3:0/4253532969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f501030c0 0x7f4f50109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.370+0000 7f4f55e00640 1 --2- 172.21.15.3:0/4253532969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f50102700 0x7f4f50102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.370+0000 7f4f55e00640 1 --2- 172.21.15.3:0/4253532969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50101510 0x7f4f50101930 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.370+0000 7f4f55e00640 1 -- 172.21.15.3:0/4253532969 >> 172.21.15.3:0/4253532969 conn(0x7f4f500fca80 msgr2=0x7f4f500feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.370+0000 7f4f55e00640 1 -- 172.21.15.3:0/4253532969 shutdown_connections 2024-03-21T08:28:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.370+0000 7f4f55e00640 1 -- 172.21.15.3:0/4253532969 wait complete. 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.371+0000 7f4f55e00640 1 Processor -- start 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.371+0000 7f4f55e00640 1 -- start start 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.371+0000 7f4f55e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f50101510 0x7f4f50199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.371+0000 7f4f55e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50102700 0x7f4f50199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f55e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f501030c0 0x7f4f501a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f55e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5010d0a0 con 0x7f4f50102700 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f55e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5010cf20 con 0x7f4f50101510 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f55e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5010d220 con 0x7f4f501030c0 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50102700 0x7f4f50199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50102700 0x7f4f50199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60974/0 (socket says 172.21.15.3:60974) 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 -- 172.21.15.3:0/3143054283 learned_addr learned my addr 172.21.15.3:0/3143054283 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f54a00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f50101510 0x7f4f50199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f55400640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f501030c0 0x7f4f501a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 -- 172.21.15.3:0/3143054283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f501030c0 msgr2=0x7f4f501a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f501030c0 0x7f4f501a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 -- 172.21.15.3:0/3143054283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f50101510 msgr2=0x7f4f50199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f50101510 0x7f4f50199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.372+0000 7f4f4fe00640 1 -- 172.21.15.3:0/3143054283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f501a1580 con 0x7f4f50102700 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.373+0000 7f4f4fe00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50102700 0x7f4f50199ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800eab0 tx=0x7f4f3800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.373+0000 7f4f4d600640 1 -- 172.21.15.3:0/3143054283 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800ce40 con 0x7f4f50102700 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.373+0000 7f4f4d600640 1 -- 172.21.15.3:0/3143054283 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4f38011920 con 0x7f4f50102700 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.373+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f501a1870 con 0x7f4f50102700 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.373+0000 7f4f4d600640 1 -- 172.21.15.3:0/3143054283 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38010a00 con 0x7f4f50102700 2024-03-21T08:28:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.373+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f501a1e10 con 0x7f4f50102700 2024-03-21T08:28:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.375+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f501a20a0 con 0x7f4f50102700 2024-03-21T08:28:55.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.376+0000 7f4f4d600640 1 -- 172.21.15.3:0/3143054283 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f4f38011ad0 con 0x7f4f50102700 2024-03-21T08:28:55.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.378+0000 7f4f4d600640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f300a2fa0 0x7f4f300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.378+0000 7f4f4d600640 1 -- 172.21.15.3:0/3143054283 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4f380cfd20 con 0x7f4f50102700 2024-03-21T08:28:55.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.378+0000 7f4f54a00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f300a2fa0 0x7f4f300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.380+0000 7f4f54a00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f300a2fa0 0x7f4f300a5460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f4002c240 tx=0x7f4f40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:55.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.382+0000 7f4f4d600640 1 -- 172.21.15.3:0/3143054283 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f38016030 con 0x7f4f50102700 2024-03-21T08:28:55.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.600+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4f501a2270 con 0x7f4f50102700 2024-03-21T08:28:55.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.601+0000 7f4f4d600640 1 -- 172.21.15.3:0/3143054283 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) v1 ==== 81+0+581 (secure 0 0 0) 0x7f4f3809c8b0 con 0x7f4f50102700 2024-03-21T08:28:55.603 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2117697938 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:55.603 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2293101955 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/404577225 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1597279144 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4112229601 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2051354447 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f300a2fa0 msgr2=0x7f4f300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f300a2fa0 0x7f4f300a5460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f4002c240 tx=0x7f4f40005fd0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50102700 msgr2=0x7f4f50199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50102700 0x7f4f50199ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800eab0 tx=0x7f4f3800ef70 comp rx=0 tx=0).stop 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 shutdown_connections 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4f300a2fa0 0x7f4f300a5460 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f501030c0 0x7f4f501a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f50102700 0x7f4f50199ea0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 --2- 172.21.15.3:0/3143054283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f50101510 0x7f4f50199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 >> 172.21.15.3:0/3143054283 conn(0x7f4f500fca80 msgr2=0x7f4f500738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 shutdown_connections 2024-03-21T08:28:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.604+0000 7f4f55e00640 1 -- 172.21.15.3:0/3143054283 wait complete. 2024-03-21T08:28:55.624 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:28:55.624 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T08:28:55.679 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:28:55.762 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:55 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2463561705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:28:55.762 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:55 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2842771043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:28:55.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:55 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2463561705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:28:55.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:55 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2842771043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:28:55.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:55 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2463561705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:28:55.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:55 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2842771043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:28:55.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 -- 172.21.15.3:0/4218961520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 msgr2=0x7f58c4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 --2- 172.21.15.3:0/4218961520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4102da0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009900 tx=0x7f58b002bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:55.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 -- 172.21.15.3:0/4218961520 shutdown_connections 2024-03-21T08:28:55.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 --2- 172.21.15.3:0/4218961520 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58c41032e0 0x7f58c4109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 --2- 172.21.15.3:0/4218961520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4102da0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 --2- 172.21.15.3:0/4218961520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58c4101730 0x7f58c4101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 -- 172.21.15.3:0/4218961520 >> 172.21.15.3:0/4218961520 conn(0x7f58c40fcc80 msgr2=0x7f58c40ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:55.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 -- 172.21.15.3:0/4218961520 shutdown_connections 2024-03-21T08:28:55.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.901+0000 7f58cb400640 1 -- 172.21.15.3:0/4218961520 wait complete. 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 Processor -- start 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 -- start start 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58c4101730 0x7f58c4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58c41032e0 0x7f58c41a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c410d2a0 con 0x7f58c4102920 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c410d120 con 0x7f58c4101730 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.902+0000 7f58cb400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c410d420 con 0x7f58c41032e0 2024-03-21T08:28:55.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60994/0 (socket says 172.21.15.3:60994) 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 -- 172.21.15.3:0/818687683 learned_addr learned my addr 172.21.15.3:0/818687683 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c8a00640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58c4101730 0x7f58c4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c9400640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58c41032e0 0x7f58c41a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 -- 172.21.15.3:0/818687683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58c41032e0 msgr2=0x7f58c41a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58c41032e0 0x7f58c41a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 -- 172.21.15.3:0/818687683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58c4101730 msgr2=0x7f58c4199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58c4101730 0x7f58c4199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c3e00640 1 -- 172.21.15.3:0/818687683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c41a16c0 con 0x7f58c4102920 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.903+0000 7f58c8a00640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58c4101730 0x7f58c4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:28:55.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.904+0000 7f58c3e00640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4199fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000c00 tx=0x7f58b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:55.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.904+0000 7f58c1600640 1 -- 172.21.15.3:0/818687683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0043020 con 0x7f58c4102920 2024-03-21T08:28:55.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.904+0000 7f58c1600640 1 -- 172.21.15.3:0/818687683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58b00054c0 con 0x7f58c4102920 2024-03-21T08:28:55.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.904+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c41a1950 con 0x7f58c4102920 2024-03-21T08:28:55.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.904+0000 7f58c1600640 1 -- 172.21.15.3:0/818687683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0003d80 con 0x7f58c4102920 2024-03-21T08:28:55.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.904+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c41a1f20 con 0x7f58c4102920 2024-03-21T08:28:55.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.905+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4068970 con 0x7f58c4102920 2024-03-21T08:28:55.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.907+0000 7f58c1600640 1 -- 172.21.15.3:0/818687683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f58b0046020 con 0x7f58c4102920 2024-03-21T08:28:55.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.909+0000 7f58c1600640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f58ac0a2f20 0x7f58ac0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:55.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.909+0000 7f58c1600640 1 -- 172.21.15.3:0/818687683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f58b00f20b0 con 0x7f58c4102920 2024-03-21T08:28:55.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.909+0000 7f58c8a00640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f58ac0a2f20 0x7f58ac0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:55.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.910+0000 7f58c8a00640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f58ac0a2f20 0x7f58ac0a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58c4102780 tx=0x7f58b800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:55.911+0000 7f58c1600640 1 -- 172.21.15.3:0/818687683 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58b004d020 con 0x7f58c4102920 2024-03-21T08:28:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.120+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58c41a2300 con 0x7f58c4102920 2024-03-21T08:28:56.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.124+0000 7f58c1600640 1 -- 172.21.15.3:0/818687683 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24035 (secure 0 0 0) 0x7f58b00bed60 con 0x7f58c4102920 2024-03-21T08:28:56.126 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:28:56.127 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:28:41.154980+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:28:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f58ac0a2f20 msgr2=0x7f58ac0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f58ac0a2f20 0x7f58ac0a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58c4102780 tx=0x7f58b800a5e0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 msgr2=0x7f58c4199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4199fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000c00 tx=0x7f58b00042f0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 shutdown_connections 2024-03-21T08:28:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f58ac0a2f20 0x7f58ac0a53e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58c41032e0 0x7f58c41a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58c4102920 0x7f58c4199fe0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 --2- 172.21.15.3:0/818687683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58c4101730 0x7f58c4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 >> 172.21.15.3:0/818687683 conn(0x7f58c40fcc80 msgr2=0x7f58c40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 shutdown_connections 2024-03-21T08:28:56.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.127+0000 7f58cb400640 1 -- 172.21.15.3:0/818687683 wait complete. 2024-03-21T08:28:56.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:28:56.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:28:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.378+0000 7f9ce3200640 1 -- 172.21.15.3:0/1961180412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 msgr2=0x7f9ce4104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.378+0000 7f9ce3200640 1 --2- 172.21.15.3:0/1961180412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4104d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009900 tx=0x7f9ccc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.378+0000 7f9ce3200640 1 -- 172.21.15.3:0/1961180412 shutdown_connections 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.378+0000 7f9ce3200640 1 --2- 172.21.15.3:0/1961180412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ce4105290 0x7f9ce410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.378+0000 7f9ce3200640 1 --2- 172.21.15.3:0/1961180412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4104d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.378+0000 7f9ce3200640 1 --2- 172.21.15.3:0/1961180412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ce41036e0 0x7f9ce4103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.378+0000 7f9ce3200640 1 -- 172.21.15.3:0/1961180412 >> 172.21.15.3:0/1961180412 conn(0x7f9ce40fec70 msgr2=0x7f9ce4101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.379+0000 7f9ce3200640 1 -- 172.21.15.3:0/1961180412 shutdown_connections 2024-03-21T08:28:56.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.379+0000 7f9ce3200640 1 -- 172.21.15.3:0/1961180412 wait complete. 2024-03-21T08:28:56.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.379+0000 7f9ce3200640 1 Processor -- start 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce3200640 1 -- start start 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ce41036e0 0x7f9ce4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ce4105290 0x7f9ce41a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410f240 con 0x7f9ce41048d0 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410f0c0 con 0x7f9ce4105290 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410f3c0 con 0x7f9ce41036e0 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32788/0 (socket says 172.21.15.3:32788) 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.380+0000 7f9ce1400640 1 -- 172.21.15.3:0/577627632 learned_addr learned my addr 172.21.15.3:0/577627632 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce1e00640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ce41036e0 0x7f9ce4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce2800640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ce4105290 0x7f9ce41a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce1400640 1 -- 172.21.15.3:0/577627632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ce41036e0 msgr2=0x7f9ce4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce1400640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ce41036e0 0x7f9ce4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce1400640 1 -- 172.21.15.3:0/577627632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ce4105290 msgr2=0x7f9ce41a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce1400640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ce4105290 0x7f9ce41a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce1400640 1 -- 172.21.15.3:0/577627632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce41a1570 con 0x7f9ce41048d0 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9ce1400640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4199e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.381+0000 7f9cdaa00640 1 -- 172.21.15.3:0/577627632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc043020 con 0x7f9ce41048d0 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.382+0000 7f9cdaa00640 1 -- 172.21.15.3:0/577627632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ccc0054c0 con 0x7f9ce41048d0 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.382+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce41a1800 con 0x7f9ce41048d0 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.382+0000 7f9cdaa00640 1 -- 172.21.15.3:0/577627632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc003d70 con 0x7f9ce41048d0 2024-03-21T08:28:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.382+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce41a1d50 con 0x7f9ce41048d0 2024-03-21T08:28:56.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.383+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce410aaf0 con 0x7f9ce41048d0 2024-03-21T08:28:56.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.385+0000 7f9cdaa00640 1 -- 172.21.15.3:0/577627632 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f9ccc046020 con 0x7f9ce41048d0 2024-03-21T08:28:56.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.386+0000 7f9cdaa00640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9cc80a2f20 0x7f9cc80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.386+0000 7f9cdaa00640 1 -- 172.21.15.3:0/577627632 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9ccc0f1b30 con 0x7f9ce41048d0 2024-03-21T08:28:56.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.387+0000 7f9ce1e00640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9cc80a2f20 0x7f9cc80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.387+0000 7f9ce1e00640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9cc80a2f20 0x7f9cc80a53e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4104730 tx=0x7f9cd4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:56.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.390+0000 7f9cdaa00640 1 -- 172.21.15.3:0/577627632 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ccc04d020 con 0x7f9ce41048d0 2024-03-21T08:28:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.596+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ce40622a0 con 0x7f9ce41048d0 2024-03-21T08:28:56.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.597+0000 7f9cdaa00640 1 -- 172.21.15.3:0/577627632 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ccc0be720 con 0x7f9ce41048d0 2024-03-21T08:28:56.599 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:28:56.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9cc80a2f20 msgr2=0x7f9cc80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9cc80a2f20 0x7f9cc80a53e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4104730 tx=0x7f9cd4009870 comp rx=0 tx=0).stop 2024-03-21T08:28:56.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 msgr2=0x7f9ce4199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4199e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc0042f0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 shutdown_connections 2024-03-21T08:28:56.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ce4105290 0x7f9ce41a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ce41048d0 0x7f9ce4199e90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9cc80a2f20 0x7f9cc80a53e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 --2- 172.21.15.3:0/577627632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ce41036e0 0x7f9ce4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 >> 172.21.15.3:0/577627632 conn(0x7f9ce40fec70 msgr2=0x7f9ce4107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:56.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 shutdown_connections 2024-03-21T08:28:56.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.600+0000 7f9ce3200640 1 -- 172.21.15.3:0/577627632 wait complete. 2024-03-21T08:28:56.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:28:56.696 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:56 smithi003 ceph-mon[31675]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T08:28:56.696 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:56 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3143054283' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:28:56.696 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:56 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/818687683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 -- 172.21.15.3:0/3718666595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 msgr2=0x7fd9e8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3718666595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8102b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009900 tx=0x7fd9d802bc40 comp rx=0 tx=0).stop 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 -- 172.21.15.3:0/3718666595 shutdown_connections 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3718666595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9e81030c0 0x7fd9e8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3718666595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8102b80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3718666595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9e8101510 0x7fd9e8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 -- 172.21.15.3:0/3718666595 >> 172.21.15.3:0/3718666595 conn(0x7fd9e80fca80 msgr2=0x7fd9e80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 -- 172.21.15.3:0/3718666595 shutdown_connections 2024-03-21T08:28:56.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.834+0000 7fd9e7200640 1 -- 172.21.15.3:0/3718666595 wait complete. 2024-03-21T08:28:56.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.835+0000 7fd9e7200640 1 Processor -- start 2024-03-21T08:28:56.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e7200640 1 -- start start 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9e8101510 0x7fd9e8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9e81030c0 0x7fd9e8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e810d0a0 con 0x7fd9e8102700 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e810cf20 con 0x7fd9e8101510 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e810d220 con 0x7fd9e81030c0 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32808/0 (socket says 172.21.15.3:32808) 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e5400640 1 -- 172.21.15.3:0/3448943704 learned_addr learned my addr 172.21.15.3:0/3448943704 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.836+0000 7fd9e5e00640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9e8101510 0x7fd9e8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e6800640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9e81030c0 0x7fd9e8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e5400640 1 -- 172.21.15.3:0/3448943704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9e81030c0 msgr2=0x7fd9e8073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e5400640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9e81030c0 0x7fd9e8073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e5400640 1 -- 172.21.15.3:0/3448943704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9e8101510 msgr2=0x7fd9e8075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e5400640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9e8101510 0x7fd9e8075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e5400640 1 -- 172.21.15.3:0/3448943704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8074860 con 0x7fd9e8102700 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e6800640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9e81030c0 0x7fd9e8073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e5400640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8073a90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9dea00640 1 -- 172.21.15.3:0/3448943704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8043020 con 0x7fd9e8102700 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9dea00640 1 -- 172.21.15.3:0/3448943704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd9d80054c0 con 0x7fd9e8102700 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e81a1810 con 0x7fd9e8102700 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.837+0000 7fd9dea00640 1 -- 172.21.15.3:0/3448943704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8003d40 con 0x7fd9e8102700 2024-03-21T08:28:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.838+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e81a1db0 con 0x7fd9e8102700 2024-03-21T08:28:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.839+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e8108830 con 0x7fd9e8102700 2024-03-21T08:28:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.841+0000 7fd9dea00640 1 -- 172.21.15.3:0/3448943704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd9d8046020 con 0x7fd9e8102700 2024-03-21T08:28:56.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.842+0000 7fd9dea00640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9cc0a2fa0 0x7fd9cc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:56.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.842+0000 7fd9dea00640 1 -- 172.21.15.3:0/3448943704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd9d80f1a80 con 0x7fd9e8102700 2024-03-21T08:28:56.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.842+0000 7fd9e5e00640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9cc0a2fa0 0x7fd9cc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:56.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.843+0000 7fd9e5e00640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9cc0a2fa0 0x7fd9cc0a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8102560 tx=0x7fd9e0009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:56.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:56.846+0000 7fd9dea00640 1 -- 172.21.15.3:0/3448943704 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9d804d020 con 0x7fd9e8102700 2024-03-21T08:28:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:56 smithi084 ceph-mon[35477]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T08:28:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:56 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3143054283' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:28:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:56 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/818687683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:56 smithi171 ceph-mon[35388]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T08:28:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:56 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3143054283' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:28:56.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:56 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/818687683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:28:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.050+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9e80622a0 con 0x7fd9e8102700 2024-03-21T08:28:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.051+0000 7fd9dea00640 1 -- 172.21.15.3:0/3448943704 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9d80be670 con 0x7fd9e8102700 2024-03-21T08:28:57.053 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9cc0a2fa0 msgr2=0x7fd9cc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9cc0a2fa0 0x7fd9cc0a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8102560 tx=0x7fd9e0009870 comp rx=0 tx=0).stop 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 msgr2=0x7fd9e8073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8073a90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d80042f0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 shutdown_connections 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd9cc0a2fa0 0x7fd9cc0a5460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9e81030c0 0x7fd9e8073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9e8102700 0x7fd9e8073a90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 --2- 172.21.15.3:0/3448943704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9e8101510 0x7fd9e8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 >> 172.21.15.3:0/3448943704 conn(0x7fd9e80fca80 msgr2=0x7fd9e8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 shutdown_connections 2024-03-21T08:28:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.054+0000 7fd9e7200640 1 -- 172.21.15.3:0/3448943704 wait complete. 2024-03-21T08:28:57.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:28:57.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 -- 172.21.15.3:0/44246883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 msgr2=0x7fd3b4103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 --2- 172.21.15.3:0/44246883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4103b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009ca0 tx=0x7fd3a402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 -- 172.21.15.3:0/44246883 shutdown_connections 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 --2- 172.21.15.3:0/44246883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3b4105290 0x7fd3b410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 --2- 172.21.15.3:0/44246883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3b41048d0 0x7fd3b4104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 --2- 172.21.15.3:0/44246883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4103b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 -- 172.21.15.3:0/44246883 >> 172.21.15.3:0/44246883 conn(0x7fd3b40fec70 msgr2=0x7fd3b4101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 -- 172.21.15.3:0/44246883 shutdown_connections 2024-03-21T08:28:57.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.311+0000 7fd3ba600640 1 -- 172.21.15.3:0/44246883 wait complete. 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.312+0000 7fd3ba600640 1 Processor -- start 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3ba600640 1 -- start start 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3ba600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3ba600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3b41048d0 0x7fd3b4199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3ba600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3b4105290 0x7fd3b41a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3ba600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b410f200 con 0x7fd3b41036e0 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3ba600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b410f080 con 0x7fd3b41048d0 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3ba600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b410f380 con 0x7fd3b4105290 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3b9200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3b9200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32820/0 (socket says 172.21.15.3:32820) 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.313+0000 7fd3b9200640 1 -- 172.21.15.3:0/1490429660 learned_addr learned my addr 172.21.15.3:0/1490429660 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:57.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b3e00640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3b41048d0 0x7fd3b4199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b9c00640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3b4105290 0x7fd3b41a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b9200640 1 -- 172.21.15.3:0/1490429660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3b4105290 msgr2=0x7fd3b41a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b9200640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3b4105290 0x7fd3b41a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b9200640 1 -- 172.21.15.3:0/1490429660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3b41048d0 msgr2=0x7fd3b4199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b9200640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3b41048d0 0x7fd3b4199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b9200640 1 -- 172.21.15.3:0/1490429660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b41a1580 con 0x7fd3b41036e0 2024-03-21T08:28:57.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.314+0000 7fd3b9200640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4199960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4032380 tx=0x7fd3a4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:57.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.315+0000 7fd3b1600640 1 -- 172.21.15.3:0/1490429660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4041020 con 0x7fd3b41036e0 2024-03-21T08:28:57.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.315+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b41a1810 con 0x7fd3b41036e0 2024-03-21T08:28:57.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.315+0000 7fd3b1600640 1 -- 172.21.15.3:0/1490429660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd3a4003e20 con 0x7fd3b41036e0 2024-03-21T08:28:57.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.315+0000 7fd3b1600640 1 -- 172.21.15.3:0/1490429660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4039040 con 0x7fd3b41036e0 2024-03-21T08:28:57.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.315+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b41a1d50 con 0x7fd3b41036e0 2024-03-21T08:28:57.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.316+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b410ab40 con 0x7fd3b41036e0 2024-03-21T08:28:57.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.318+0000 7fd3b1600640 1 -- 172.21.15.3:0/1490429660 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd3a4044020 con 0x7fd3b41036e0 2024-03-21T08:28:57.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.319+0000 7fd3b1600640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3980a2f20 0x7fd3980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.320+0000 7fd3b1600640 1 -- 172.21.15.3:0/1490429660 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd3a40f19a0 con 0x7fd3b41036e0 2024-03-21T08:28:57.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.320+0000 7fd3b3e00640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3980a2f20 0x7fd3980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.321+0000 7fd3b3e00640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3980a2f20 0x7fd3980a53e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005fd0 tx=0x7fd3a0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:57.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.325+0000 7fd3b1600640 1 -- 172.21.15.3:0/1490429660 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd3a404b020 con 0x7fd3b41036e0 2024-03-21T08:28:57.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.530+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3b402d070 con 0x7fd3b41036e0 2024-03-21T08:28:57.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.531+0000 7fd3b1600640 1 -- 172.21.15.3:0/1490429660 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3a403b2f0 con 0x7fd3b41036e0 2024-03-21T08:28:57.533 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:28:57.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.533+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3980a2f20 msgr2=0x7fd3980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.533+0000 7fd3ba600640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3980a2f20 0x7fd3980a53e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005fd0 tx=0x7fd3a0005850 comp rx=0 tx=0).stop 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.533+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 msgr2=0x7fd3b4199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.533+0000 7fd3ba600640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4199960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4032380 tx=0x7fd3a4032820 comp rx=0 tx=0).stop 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 shutdown_connections 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3980a2f20 0x7fd3980a53e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3b4105290 0x7fd3b41a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3b41048d0 0x7fd3b4199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 --2- 172.21.15.3:0/1490429660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3b41036e0 0x7fd3b4199960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 >> 172.21.15.3:0/1490429660 conn(0x7fd3b40fec70 msgr2=0x7fd3b4107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 shutdown_connections 2024-03-21T08:28:57.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.534+0000 7fd3ba600640 1 -- 172.21.15.3:0/1490429660 wait complete. 2024-03-21T08:28:57.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:28:57.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 -- 172.21.15.3:0/1287546219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc107b00 msgr2=0x7f15fc107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 --2- 172.21.15.3:0/1287546219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc107b00 0x7f15fc107f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009900 tx=0x7f15f002bc30 comp rx=0 tx=0).stop 2024-03-21T08:28:57.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 -- 172.21.15.3:0/1287546219 shutdown_connections 2024-03-21T08:28:57.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 --2- 172.21.15.3:0/1287546219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc1096b0 0x7f15fc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 --2- 172.21.15.3:0/1287546219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15fc108cf0 0x7f15fc109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 --2- 172.21.15.3:0/1287546219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc107b00 0x7f15fc107f20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 -- 172.21.15.3:0/1287546219 >> 172.21.15.3:0/1287546219 conn(0x7f15fc076a90 msgr2=0x7f15fc078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 -- 172.21.15.3:0/1287546219 shutdown_connections 2024-03-21T08:28:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.784+0000 7f1602e00640 1 -- 172.21.15.3:0/1287546219 wait complete. 2024-03-21T08:28:57.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.785+0000 7f1602e00640 1 Processor -- start 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.785+0000 7f1602e00640 1 -- start start 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc107b00 0x7f15fc19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15fc108cf0 0x7f15fc19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc1096b0 0x7f15fc1a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc113650 con 0x7f15fc1096b0 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc1134d0 con 0x7f15fc107b00 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc1137d0 con 0x7f15fc108cf0 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc1096b0 0x7f15fc1a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc1096b0 0x7f15fc1a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32834/0 (socket says 172.21.15.3:32834) 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1602400640 1 -- 172.21.15.3:0/2394143361 learned_addr learned my addr 172.21.15.3:0/2394143361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.786+0000 7f1601a00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc107b00 0x7f15fc19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1601000640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15fc108cf0 0x7f15fc19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1602400640 1 -- 172.21.15.3:0/2394143361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15fc108cf0 msgr2=0x7f15fc19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1602400640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15fc108cf0 0x7f15fc19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1602400640 1 -- 172.21.15.3:0/2394143361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc107b00 msgr2=0x7f15fc19dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1602400640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc107b00 0x7f15fc19dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1602400640 1 -- 172.21.15.3:0/2394143361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc1a5920 con 0x7f15fc1096b0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1601000640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15fc108cf0 0x7f15fc19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1602400640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc1096b0 0x7f15fc1a5280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15f400cce0 tx=0x7f15f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f15fa000640 1 -- 172.21.15.3:0/2394143361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f400f450 con 0x7f15fc1096b0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc1a5c10 con 0x7f15fc1096b0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.787+0000 7f15fa000640 1 -- 172.21.15.3:0/2394143361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f15f400faa0 con 0x7f15fc1096b0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.788+0000 7f15fa000640 1 -- 172.21.15.3:0/2394143361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4018b10 con 0x7f15fc1096b0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.788+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc1a60e0 con 0x7f15fc1096b0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.789+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc10eea0 con 0x7f15fc1096b0 2024-03-21T08:28:57.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.791+0000 7f15fa000640 1 -- 172.21.15.3:0/2394143361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f15f4018cb0 con 0x7f15fc1096b0 2024-03-21T08:28:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.792+0000 7f15fa000640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f15e40a2f20 0x7f15e40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.792+0000 7f15fa000640 1 -- 172.21.15.3:0/2394143361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f15f40cfb30 con 0x7f15fc1096b0 2024-03-21T08:28:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.792+0000 7f1601a00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f15e40a2f20 0x7f15e40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.793+0000 7f1601a00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f15e40a2f20 0x7f15e40a53e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15f002c240 tx=0x7f15f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:57.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.796+0000 7f15fa000640 1 -- 172.21.15.3:0/2394143361 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f15f4014030 con 0x7f15fc1096b0 2024-03-21T08:28:57.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:57 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/577627632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:57.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:57 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3448943704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:28:57.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:57 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/577627632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:57.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:57 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3448943704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:28:57.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:57 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/577627632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:28:57.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:57 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3448943704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:28:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.997+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f15fc062340 con 0x7f15fc1096b0 2024-03-21T08:28:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:57.998+0000 7f15fa000640 1 -- 172.21.15.3:0/2394143361 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f15f409c6c0 con 0x7f15fc1096b0 2024-03-21T08:28:58.001 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:28:58.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f15e40a2f20 msgr2=0x7f15e40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:58.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f15e40a2f20 0x7f15e40a53e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15f002c240 tx=0x7f15f0000bc0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc1096b0 msgr2=0x7f15fc1a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc1096b0 0x7f15fc1a5280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15f400cce0 tx=0x7f15f400a620 comp rx=0 tx=0).stop 2024-03-21T08:28:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 shutdown_connections 2024-03-21T08:28:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15fc1096b0 0x7f15fc1a5280 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f15e40a2f20 0x7f15e40a53e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15fc108cf0 0x7f15fc19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 --2- 172.21.15.3:0/2394143361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc107b00 0x7f15fc19dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 >> 172.21.15.3:0/2394143361 conn(0x7f15fc076a90 msgr2=0x7f15fc10c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:58.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 shutdown_connections 2024-03-21T08:28:58.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.001+0000 7f1602e00640 1 -- 172.21.15.3:0/2394143361 wait complete. 2024-03-21T08:28:58.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T08:28:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.246+0000 7f3422600640 1 -- 172.21.15.3:0/188106659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c101510 msgr2=0x7f341c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.246+0000 7f3422600640 1 --2- 172.21.15.3:0/188106659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c101510 0x7f341c101930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f340c009ca0 tx=0x7f340c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.247+0000 7f3422600640 1 -- 172.21.15.3:0/188106659 shutdown_connections 2024-03-21T08:28:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.247+0000 7f3422600640 1 --2- 172.21.15.3:0/188106659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c1030c0 0x7f341c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.247+0000 7f3422600640 1 --2- 172.21.15.3:0/188106659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f341c102700 0x7f341c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.247+0000 7f3422600640 1 --2- 172.21.15.3:0/188106659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c101510 0x7f341c101930 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.247+0000 7f3422600640 1 -- 172.21.15.3:0/188106659 >> 172.21.15.3:0/188106659 conn(0x7f341c0fca80 msgr2=0x7f341c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.247+0000 7f3422600640 1 -- 172.21.15.3:0/188106659 shutdown_connections 2024-03-21T08:28:58.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.247+0000 7f3422600640 1 -- 172.21.15.3:0/188106659 wait complete. 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 Processor -- start 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 -- start start 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c101510 0x7f341c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c102700 0x7f341c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f341c1030c0 0x7f341c1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c10d050 con 0x7f341c102700 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c10ced0 con 0x7f341c101510 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f3422600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c10d1d0 con 0x7f341c1030c0 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.248+0000 7f341be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c102700 0x7f341c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c102700 0x7f341c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32856/0 (socket says 172.21.15.3:32856) 2024-03-21T08:28:58.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 -- 172.21.15.3:0/4134165719 learned_addr learned my addr 172.21.15.3:0/4134165719 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f3421200640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c101510 0x7f341c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f3421c00640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f341c1030c0 0x7f341c1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 -- 172.21.15.3:0/4134165719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f341c1030c0 msgr2=0x7f341c1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f341c1030c0 0x7f341c1a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 -- 172.21.15.3:0/4134165719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c101510 msgr2=0x7f341c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c101510 0x7f341c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 -- 172.21.15.3:0/4134165719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c1a1580 con 0x7f341c102700 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.249+0000 7f341be00640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c102700 0x7f341c199ea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f340400d950 tx=0x7f340400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.250+0000 7f3419600640 1 -- 172.21.15.3:0/4134165719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404017070 con 0x7f341c102700 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.250+0000 7f3419600640 1 -- 172.21.15.3:0/4134165719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3404009c30 con 0x7f341c102700 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.250+0000 7f3419600640 1 -- 172.21.15.3:0/4134165719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400f620 con 0x7f341c102700 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.250+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c1a1870 con 0x7f341c102700 2024-03-21T08:28:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.250+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c1a1e10 con 0x7f341c102700 2024-03-21T08:28:58.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.251+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c1a2040 con 0x7f341c102700 2024-03-21T08:28:58.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.253+0000 7f3419600640 1 -- 172.21.15.3:0/4134165719 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f34040109a0 con 0x7f341c102700 2024-03-21T08:28:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.254+0000 7f3419600640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f34000a2fa0 0x7f34000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:28:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.255+0000 7f3419600640 1 -- 172.21.15.3:0/4134165719 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f34040cf300 con 0x7f341c102700 2024-03-21T08:28:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.255+0000 7f3421200640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f34000a2fa0 0x7f34000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:28:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.255+0000 7f3421200640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f34000a2fa0 0x7f34000a5460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f340c02c630 tx=0x7f340c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:28:58.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.259+0000 7f3419600640 1 -- 172.21.15.3:0/4134165719 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3404009de0 con 0x7f341c102700 2024-03-21T08:28:58.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.477+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f341c1a2210 con 0x7f341c102700 2024-03-21T08:28:58.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.478+0000 7f3419600640 1 -- 172.21.15.3:0/4134165719 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) v1 ==== 81+0+581 (secure 0 0 0) 0x7f340409bef0 con 0x7f341c102700 2024-03-21T08:28:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2117697938 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2293101955 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/404577225 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:58.481 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1597279144 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:58.481 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4112229601 2024-03-22T08:18:30.424775+0000 2024-03-21T08:28:58.481 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2051354447 2024-03-22T08:19:05.157913+0000 2024-03-21T08:28:58.481 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f34000a2fa0 msgr2=0x7f34000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f34000a2fa0 0x7f34000a5460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f340c02c630 tx=0x7f340c000f40 comp rx=0 tx=0).stop 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c102700 msgr2=0x7f341c199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c102700 0x7f341c199ea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f340400d950 tx=0x7f340400de10 comp rx=0 tx=0).stop 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 shutdown_connections 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f34000a2fa0 0x7f34000a5460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f341c1030c0 0x7f341c1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f341c102700 0x7f341c199ea0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 --2- 172.21.15.3:0/4134165719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c101510 0x7f341c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:28:58.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 >> 172.21.15.3:0/4134165719 conn(0x7f341c0fca80 msgr2=0x7f341c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:28:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 shutdown_connections 2024-03-21T08:28:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:28:58.481+0000 7f3422600640 1 -- 172.21.15.3:0/4134165719 wait complete. 2024-03-21T08:28:58.501 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-03-21T08:28:58.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:28:58.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-21T08:28:58.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[75566]: fuse finished with error 0 and tester_r 0 2024-03-21T08:28:58.603 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T08:28:58.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:58 smithi084 ceph-mon[35477]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T08:28:58.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:58 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1490429660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:28:58.778 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:58 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2394143361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:28:58.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:58 smithi171 ceph-mon[35388]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T08:28:58.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:58 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1490429660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:28:58.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:58 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2394143361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:28:58.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:58 smithi003 ceph-mon[31675]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T08:28:58.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:58 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1490429660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:28:58.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:58 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2394143361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:28:59.804 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:28:59 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4134165719' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:28:59.817 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:28:59 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4134165719' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:28:59.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:28:59 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4134165719' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:29:00.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:00 smithi084 ceph-mon[35477]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:00.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:00 smithi171 ceph-mon[35388]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:00.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:00 smithi003 ceph-mon[31675]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:02.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:02 smithi084 ceph-mon[35477]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:02.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:02 smithi171 ceph-mon[35388]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:02.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:02 smithi003 ceph-mon[31675]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:04.704 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:29:04.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:29:04.705 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-21T08:29:04.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:29:04.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:04 smithi003 ceph-mon[31675]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:04.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:04 smithi084 ceph-mon[35477]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:04.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:04 smithi171 ceph-mon[35388]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 -- 172.21.15.3:0/2777357243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 msgr2=0x7f7548110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 --2- 172.21.15.3:0/2777357243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f7548110000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7540009900 tx=0x7f754002bc30 comp rx=0 tx=0).stop 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 -- 172.21.15.3:0/2777357243 shutdown_connections 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 --2- 172.21.15.3:0/2777357243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f7548110000 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 --2- 172.21.15.3:0/2777357243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7548108db0 0x7f7548109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 --2- 172.21.15.3:0/2777357243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7548107bc0 0x7f7548107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 -- 172.21.15.3:0/2777357243 >> 172.21.15.3:0/2777357243 conn(0x7f7548076b60 msgr2=0x7f7548078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 -- 172.21.15.3:0/2777357243 shutdown_connections 2024-03-21T08:29:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.012+0000 7f754dc00640 1 -- 172.21.15.3:0/2777357243 wait complete. 2024-03-21T08:29:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.013+0000 7f754dc00640 1 Processor -- start 2024-03-21T08:29:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.013+0000 7f754dc00640 1 -- start start 2024-03-21T08:29:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.013+0000 7f754dc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7548107bc0 0x7f754819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.013+0000 7f754dc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7548108db0 0x7f754819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f75481a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754dc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7548113740 con 0x7f7548109770 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754dc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481135c0 con 0x7f7548107bc0 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754dc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481138c0 con 0x7f7548108db0 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f75481a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f75481a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44046/0 (socket says 172.21.15.3:44046) 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 -- 172.21.15.3:0/388497458 learned_addr learned my addr 172.21.15.3:0/388497458 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f7547e00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7548107bc0 0x7f754819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 -- 172.21.15.3:0/388497458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7548108db0 msgr2=0x7f754819e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7548108db0 0x7f754819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 -- 172.21.15.3:0/388497458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7548107bc0 msgr2=0x7f754819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7548107bc0 0x7f754819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f754d200640 1 -- 172.21.15.3:0/388497458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75481a59b0 con 0x7f7548109770 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.014+0000 7f7547e00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7548107bc0 0x7f754819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.015+0000 7f754d200640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f75481a5310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7540036040 tx=0x7f754002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.015+0000 7f7544c00640 1 -- 172.21.15.3:0/388497458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540043020 con 0x7f7548109770 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.015+0000 7f7544c00640 1 -- 172.21.15.3:0/388497458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75400044c0 con 0x7f7548109770 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.015+0000 7f7544c00640 1 -- 172.21.15.3:0/388497458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540003bf0 con 0x7f7548109770 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.015+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75481a5c40 con 0x7f7548109770 2024-03-21T08:29:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.015+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75481a6140 con 0x7f7548109770 2024-03-21T08:29:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.016+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754810ee80 con 0x7f7548109770 2024-03-21T08:29:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.018+0000 7f7544c00640 1 -- 172.21.15.3:0/388497458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f7540046020 con 0x7f7548109770 2024-03-21T08:29:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.019+0000 7f7544c00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f752c0a2f10 0x7f752c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.020+0000 7f7544c00640 1 -- 172.21.15.3:0/388497458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f75400f1aa0 con 0x7f7548109770 2024-03-21T08:29:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.020+0000 7f7547e00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f752c0a2f10 0x7f752c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.021+0000 7f7547e00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f752c0a2f10 0x7f752c0a53d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f754819f270 tx=0x7f7538005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:05.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.024+0000 7f7544c00640 1 -- 172.21.15.3:0/388497458 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f754004d020 con 0x7f7548109770 2024-03-21T08:29:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.228+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f754802d2b0 con 0x7f7548109770 2024-03-21T08:29:05.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.232+0000 7f7544c00640 1 -- 172.21.15.3:0/388497458 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24035 (secure 0 0 0) 0x7f75400be630 con 0x7f7548109770 2024-03-21T08:29:05.235 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:28:41.154980+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:29:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f752c0a2f10 msgr2=0x7f752c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f752c0a2f10 0x7f752c0a53d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f754819f270 tx=0x7f7538005f90 comp rx=0 tx=0).stop 2024-03-21T08:29:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 msgr2=0x7f75481a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f75481a5310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7540036040 tx=0x7f754002cdd0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 shutdown_connections 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548109770 0x7f75481a5310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f752c0a2f10 0x7f752c0a53d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7548108db0 0x7f754819e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 --2- 172.21.15.3:0/388497458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7548107bc0 0x7f754819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 >> 172.21.15.3:0/388497458 conn(0x7f7548076b60 msgr2=0x7f754810c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.235+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 shutdown_connections 2024-03-21T08:29:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.236+0000 7f754dc00640 1 -- 172.21.15.3:0/388497458 wait complete. 2024-03-21T08:29:05.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:29:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.486+0000 7f6281600640 1 -- 172.21.15.3:0/3669712626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c107bc0 msgr2=0x7f627c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.486+0000 7f6281600640 1 --2- 172.21.15.3:0/3669712626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c107bc0 0x7f627c107fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f626c009900 tx=0x7f626c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.487+0000 7f6281600640 1 -- 172.21.15.3:0/3669712626 shutdown_connections 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.487+0000 7f6281600640 1 --2- 172.21.15.3:0/3669712626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627c109770 0x7f627c110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.487+0000 7f6281600640 1 --2- 172.21.15.3:0/3669712626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627c108db0 0x7f627c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.487+0000 7f6281600640 1 --2- 172.21.15.3:0/3669712626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c107bc0 0x7f627c107fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.487+0000 7f6281600640 1 -- 172.21.15.3:0/3669712626 >> 172.21.15.3:0/3669712626 conn(0x7f627c076b60 msgr2=0x7f627c078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.487+0000 7f6281600640 1 -- 172.21.15.3:0/3669712626 shutdown_connections 2024-03-21T08:29:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.487+0000 7f6281600640 1 -- 172.21.15.3:0/3669712626 wait complete. 2024-03-21T08:29:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.488+0000 7f6281600640 1 Processor -- start 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6281600640 1 -- start start 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6281600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627c107bc0 0x7f627c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6281600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c108db0 0x7f627c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6281600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627c109770 0x7f627c1a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6281600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c113790 con 0x7f627c108db0 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6281600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c113610 con 0x7f627c109770 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6281600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c113910 con 0x7f627c107bc0 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6273e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c108db0 0x7f627c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6273e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c108db0 0x7f627c19e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44076/0 (socket says 172.21.15.3:44076) 2024-03-21T08:29:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.489+0000 7f6273e00640 1 -- 172.21.15.3:0/1332944872 learned_addr learned my addr 172.21.15.3:0/1332944872 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f6280c00640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627c109770 0x7f627c1a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f627be00640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627c107bc0 0x7f627c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f6273e00640 1 -- 172.21.15.3:0/1332944872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627c107bc0 msgr2=0x7f627c19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f6273e00640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627c107bc0 0x7f627c19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f6273e00640 1 -- 172.21.15.3:0/1332944872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627c109770 msgr2=0x7f627c1a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f6273e00640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627c109770 0x7f627c1a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f6273e00640 1 -- 172.21.15.3:0/1332944872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c1a5a30 con 0x7f627c108db0 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.490+0000 7f6273e00640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c108db0 0x7f627c19e3a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.491+0000 7f6279600640 1 -- 172.21.15.3:0/1332944872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264017070 con 0x7f627c108db0 2024-03-21T08:29:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.491+0000 7f6279600640 1 -- 172.21.15.3:0/1332944872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6264009c30 con 0x7f627c108db0 2024-03-21T08:29:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.491+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c1a5d20 con 0x7f627c108db0 2024-03-21T08:29:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.491+0000 7f6279600640 1 -- 172.21.15.3:0/1332944872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626400f620 con 0x7f627c108db0 2024-03-21T08:29:05.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.491+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c1a6290 con 0x7f627c108db0 2024-03-21T08:29:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.492+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c10eee0 con 0x7f627c108db0 2024-03-21T08:29:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.494+0000 7f6279600640 1 -- 172.21.15.3:0/1332944872 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f62640109a0 con 0x7f627c108db0 2024-03-21T08:29:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.495+0000 7f6279600640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f62600a2f10 0x7f62600a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.496+0000 7f6279600640 1 -- 172.21.15.3:0/1332944872 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f62640cf350 con 0x7f627c108db0 2024-03-21T08:29:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.496+0000 7f627be00640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f62600a2f10 0x7f62600a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.496+0000 7f627be00640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f62600a2f10 0x7f62600a53d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f626c02c240 tx=0x7f626c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.500+0000 7f6279600640 1 -- 172.21.15.3:0/1332944872 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6264009de0 con 0x7f627c108db0 2024-03-21T08:29:05.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/388497458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:05.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-21T08:29:05.507 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-21T08:29:05.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-03-21T08:29:05.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.713+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f627c0622a0 con 0x7f627c108db0 2024-03-21T08:29:05.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.716+0000 7f6279600640 1 -- 172.21.15.3:0/1332944872 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24506 (secure 0 0 0) 0x7f626409bee0 con 0x7f627c108db0 2024-03-21T08:29:05.719 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:05.719 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":93,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:29:05.486964+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:29:05.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f62600a2f10 msgr2=0x7f62600a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f62600a2f10 0x7f62600a53d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f626c02c240 tx=0x7f626c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c108db0 msgr2=0x7f627c19e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c108db0 0x7f627c19e3a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).stop 2024-03-21T08:29:05.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 shutdown_connections 2024-03-21T08:29:05.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627c109770 0x7f627c1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627c108db0 0x7f627c19e3a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f62600a2f10 0x7f62600a53d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 --2- 172.21.15.3:0/1332944872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627c107bc0 0x7f627c19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 >> 172.21.15.3:0/1332944872 conn(0x7f627c076b60 msgr2=0x7f627c10c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:05.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 shutdown_connections 2024-03-21T08:29:05.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.719+0000 7f6281600640 1 -- 172.21.15.3:0/1332944872 wait complete. 2024-03-21T08:29:05.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-21T08:29:05.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/388497458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:05.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-21T08:29:05.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-03-21T08:29:05.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-03-21T08:29:05.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-21T08:29:05.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-21T08:29:05.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-21T08:29:05.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-03-21T08:29:05.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-21T08:29:05.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-21T08:29:05.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/388497458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-03-21T08:29:05.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-21T08:29:05.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-21T08:29:05.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.971+0000 7fc511a00640 1 -- 172.21.15.3:0/2378107619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 msgr2=0x7fc50c0744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.971+0000 7fc511a00640 1 --2- 172.21.15.3:0/2378107619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c0744e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009ca0 tx=0x7fc4f402c180 comp rx=0 tx=0).stop 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.972+0000 7fc511a00640 1 -- 172.21.15.3:0/2378107619 shutdown_connections 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.972+0000 7fc511a00640 1 --2- 172.21.15.3:0/2378107619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50c1093c0 0x7fc50c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.972+0000 7fc511a00640 1 --2- 172.21.15.3:0/2378107619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c0744e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.972+0000 7fc511a00640 1 --2- 172.21.15.3:0/2378107619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc50c075660 0x7fc50c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.972+0000 7fc511a00640 1 -- 172.21.15.3:0/2378107619 >> 172.21.15.3:0/2378107619 conn(0x7fc50c0fca80 msgr2=0x7fc50c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.972+0000 7fc511a00640 1 -- 172.21.15.3:0/2378107619 shutdown_connections 2024-03-21T08:29:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.972+0000 7fc511a00640 1 -- 172.21.15.3:0/2378107619 wait complete. 2024-03-21T08:29:05.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 Processor -- start 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 -- start start 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50c075660 0x7fc50c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc50c1093c0 0x7fc50c1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c10f250 con 0x7fc50c074080 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c10f0d0 con 0x7fc50c1093c0 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc511a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c10f3d0 con 0x7fc50c075660 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc50be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc50be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44106/0 (socket says 172.21.15.3:44106) 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.973+0000 7fc50be00640 1 -- 172.21.15.3:0/2677607267 learned_addr learned my addr 172.21.15.3:0/2677607267 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc50b400640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50c075660 0x7fc50c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc511000640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc50c1093c0 0x7fc50c1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc50be00640 1 -- 172.21.15.3:0/2677607267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50c075660 msgr2=0x7fc50c199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc50be00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50c075660 0x7fc50c199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc50be00640 1 -- 172.21.15.3:0/2677607267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc50c1093c0 msgr2=0x7fc50c1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc50be00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc50c1093c0 0x7fc50c1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc50be00640 1 -- 172.21.15.3:0/2677607267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c1a1570 con 0x7fc50c074080 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc50be00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c199950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00d950 tx=0x7fc4fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc508c00640 1 -- 172.21.15.3:0/2677607267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc017030 con 0x7fc50c074080 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc508c00640 1 -- 172.21.15.3:0/2677607267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc4fc004020 con 0x7fc50c074080 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c1a1860 con 0x7fc50c074080 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.974+0000 7fc508c00640 1 -- 172.21.15.3:0/2677607267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0047a0 con 0x7fc50c074080 2024-03-21T08:29:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.975+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c1a1d80 con 0x7fc50c074080 2024-03-21T08:29:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.976+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c108180 con 0x7fc50c074080 2024-03-21T08:29:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.978+0000 7fc508c00640 1 -- 172.21.15.3:0/2677607267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fc4fc0041d0 con 0x7fc50c074080 2024-03-21T08:29:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.979+0000 7fc508c00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc4f00a2f20 0x7fc4f00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.979+0000 7fc508c00640 1 -- 172.21.15.3:0/2677607267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fc4fc0cf360 con 0x7fc50c074080 2024-03-21T08:29:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.980+0000 7fc50b400640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc4f00a2f20 0x7fc4f00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:05.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.980+0000 7fc50b400640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc4f00a2f20 0x7fc4f00a53e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4006010 tx=0x7fc4f40325f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:05.984+0000 7fc508c00640 1 -- 172.21.15.3:0/2677607267 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4fc09bdd0 con 0x7fc50c074080 2024-03-21T08:29:06.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.231+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc50c0622a0 con 0x7fc50c074080 2024-03-21T08:29:06.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.523+0000 7fc508c00640 1 -- 172.21.15.3:0/2677607267 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc4fc09b6d0 con 0x7fc50c074080 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc4f00a2f20 msgr2=0x7fc4f00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc4f00a2f20 0x7fc4f00a53e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4006010 tx=0x7fc4f40325f0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 msgr2=0x7fc50c199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c199950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00d950 tx=0x7fc4fc00de10 comp rx=0 tx=0).stop 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 shutdown_connections 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc50c1093c0 0x7fc50c1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc4f00a2f20 0x7fc4f00a53e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50c075660 0x7fc50c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 --2- 172.21.15.3:0/2677607267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc50c074080 0x7fc50c199950 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.527+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 >> 172.21.15.3:0/2677607267 conn(0x7fc50c0fca80 msgr2=0x7fc50c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:06.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.528+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 shutdown_connections 2024-03-21T08:29:06.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.528+0000 7fc511a00640 1 -- 172.21.15.3:0/2677607267 wait complete. 2024-03-21T08:29:06.552 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-21T08:29:06.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-21T08:29:06.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 -- 172.21.15.3:0/2916697964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa08109000 msgr2=0x7faa0810f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:06.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 --2- 172.21.15.3:0/2916697964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa08109000 0x7faa0810f9e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009900 tx=0x7fa9f802bc30 comp rx=0 tx=0).stop 2024-03-21T08:29:06.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 -- 172.21.15.3:0/2916697964 shutdown_connections 2024-03-21T08:29:06.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 --2- 172.21.15.3:0/2916697964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0810ff20 0x7faa08112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 --2- 172.21.15.3:0/2916697964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa08109000 0x7faa0810f9e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 --2- 172.21.15.3:0/2916697964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa081086a0 0x7faa08108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 -- 172.21.15.3:0/2916697964 >> 172.21.15.3:0/2916697964 conn(0x7faa0806e880 msgr2=0x7faa08070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:06.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 -- 172.21.15.3:0/2916697964 shutdown_connections 2024-03-21T08:29:06.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.775+0000 7faa0f600640 1 -- 172.21.15.3:0/2916697964 wait complete. 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.776+0000 7faa0f600640 1 Processor -- start 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.776+0000 7faa0f600640 1 -- start start 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa0f600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa081086a0 0x7faa081a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa0f600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa0810ff20 0x7faa081a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa0f600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa081a7a80 0x7faa081a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa0f600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa081159b0 con 0x7faa0810ff20 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa0f600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa08115830 con 0x7faa081a7a80 2024-03-21T08:29:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa0f600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa08115b30 con 0x7faa081086a0 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa0810ff20 0x7faa081a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa0810ff20 0x7faa081a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44122/0 (socket says 172.21.15.3:44122) 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa07e00640 1 -- 172.21.15.3:0/102385330 learned_addr learned my addr 172.21.15.3:0/102385330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.777+0000 7faa0cc00640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa081086a0 0x7faa081a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa07e00640 1 -- 172.21.15.3:0/102385330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa081086a0 msgr2=0x7faa081a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa07e00640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa081086a0 0x7faa081a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa07e00640 1 -- 172.21.15.3:0/102385330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa081a7a80 msgr2=0x7faa081a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa07e00640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa081a7a80 0x7faa081a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa07e00640 1 -- 172.21.15.3:0/102385330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa081aa4e0 con 0x7faa0810ff20 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa0cc00640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa081086a0 0x7faa081a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa07e00640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa0810ff20 0x7faa081a2ac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8036040 tx=0x7fa9f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.778+0000 7faa05600640 1 -- 172.21.15.3:0/102385330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8043020 con 0x7faa0810ff20 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.779+0000 7faa05600640 1 -- 172.21.15.3:0/102385330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9f8004aa0 con 0x7faa0810ff20 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.779+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa081aa770 con 0x7faa0810ff20 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.779+0000 7faa05600640 1 -- 172.21.15.3:0/102385330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8003c80 con 0x7faa0810ff20 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.779+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa081aacc0 con 0x7faa0810ff20 2024-03-21T08:29:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.780+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa081aaec0 con 0x7faa0810ff20 2024-03-21T08:29:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.782+0000 7faa05600640 1 -- 172.21.15.3:0/102385330 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fa9f8046020 con 0x7faa0810ff20 2024-03-21T08:29:06.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.784+0000 7faa05600640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9f00a2fa0 0x7fa9f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:06.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.785+0000 7faa05600640 1 -- 172.21.15.3:0/102385330 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa9f80f1c50 con 0x7faa0810ff20 2024-03-21T08:29:06.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.785+0000 7faa0cc00640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9f00a2fa0 0x7fa9f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:06.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.786+0000 7faa0cc00640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9f00a2fa0 0x7fa9f00a5460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0058c0 tx=0x7fa9fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:06.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:06.789+0000 7faa05600640 1 -- 172.21.15.3:0/102385330 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa9f804d020 con 0x7faa0810ff20 2024-03-21T08:29:06.880 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:06.880 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-21T08:29:06.880 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-03-21T08:29:06.880 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1332944872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:06.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:06 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2677607267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T08:29:06.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:06.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-21T08:29:06.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1332944872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:06.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:06 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2677607267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T08:29:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T08:29:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-21T08:29:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-03-21T08:29:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1332944872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:06 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2677607267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T08:29:07.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.047+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7faa081ab090 con 0x7faa0810ff20 2024-03-21T08:29:07.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.292+0000 7faa05600640 1 -- 172.21.15.3:0/102385330 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fa9f80be6c0 con 0x7faa0810ff20 2024-03-21T08:29:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9f00a2fa0 msgr2=0x7fa9f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9f00a2fa0 0x7fa9f00a5460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0058c0 tx=0x7fa9fc005850 comp rx=0 tx=0).stop 2024-03-21T08:29:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa0810ff20 msgr2=0x7faa081a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa0810ff20 0x7faa081a2ac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8036040 tx=0x7fa9f8004270 comp rx=0 tx=0).stop 2024-03-21T08:29:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 shutdown_connections 2024-03-21T08:29:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa081a7a80 0x7faa081a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa0810ff20 0x7faa081a2ac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa9f00a2fa0 0x7fa9f00a5460 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 --2- 172.21.15.3:0/102385330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa081086a0 0x7faa081a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 >> 172.21.15.3:0/102385330 conn(0x7faa0806e880 msgr2=0x7faa08070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 shutdown_connections 2024-03-21T08:29:07.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.297+0000 7faa0f600640 1 -- 172.21.15.3:0/102385330 wait complete. 2024-03-21T08:29:07.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-21T08:29:07.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 -- 172.21.15.3:0/755393442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a80772a0 msgr2=0x7f12a8077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:07.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 --2- 172.21.15.3:0/755393442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a80772a0 0x7f12a8077700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1290009900 tx=0x7f129002bc30 comp rx=0 tx=0).stop 2024-03-21T08:29:07.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 -- 172.21.15.3:0/755393442 shutdown_connections 2024-03-21T08:29:07.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 --2- 172.21.15.3:0/755393442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a8077c40 0x7f12a80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 --2- 172.21.15.3:0/755393442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a80772a0 0x7f12a8077700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 --2- 172.21.15.3:0/755393442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12a8076060 0x7f12a8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 -- 172.21.15.3:0/755393442 >> 172.21.15.3:0/755393442 conn(0x7f12a80fee60 msgr2=0x7f12a8101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:07.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 -- 172.21.15.3:0/755393442 shutdown_connections 2024-03-21T08:29:07.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.557+0000 7f12ae200640 1 -- 172.21.15.3:0/755393442 wait complete. 2024-03-21T08:29:07.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.558+0000 7f12ae200640 1 Processor -- start 2024-03-21T08:29:07.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.558+0000 7f12ae200640 1 -- start start 2024-03-21T08:29:07.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12ae200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a8076060 0x7f12a819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:07.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12ae200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a80772a0 0x7f12a819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:07.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12ae200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12a8077c40 0x7f12a81a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:07.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12ae200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8113810 con 0x7f12a8076060 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12ae200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8113690 con 0x7f12a80772a0 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12ae200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8113990 con 0x7f12a8077c40 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a8076060 0x7f12a819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a8076060 0x7f12a819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44136/0 (socket says 172.21.15.3:44136) 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7400640 1 -- 172.21.15.3:0/1797847313 learned_addr learned my addr 172.21.15.3:0/1797847313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a6a00640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a80772a0 0x7f12a819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7e00640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12a8077c40 0x7f12a81a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7400640 1 -- 172.21.15.3:0/1797847313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12a8077c40 msgr2=0x7f12a81a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7400640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12a8077c40 0x7f12a81a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7400640 1 -- 172.21.15.3:0/1797847313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a80772a0 msgr2=0x7f12a819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.559+0000 7f12a7400640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a80772a0 0x7f12a819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.560+0000 7f12a7400640 1 -- 172.21.15.3:0/1797847313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a81a5af0 con 0x7f12a8076060 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.560+0000 7f12a7400640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a8076060 0x7f12a819ded0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f129800cce0 tx=0x7f129800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.560+0000 7f129fe00640 1 -- 172.21.15.3:0/1797847313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298004b20 con 0x7f12a8076060 2024-03-21T08:29:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.560+0000 7f129fe00640 1 -- 172.21.15.3:0/1797847313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1298004cc0 con 0x7f12a8076060 2024-03-21T08:29:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.560+0000 7f129fe00640 1 -- 172.21.15.3:0/1797847313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800f6b0 con 0x7f12a8076060 2024-03-21T08:29:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.560+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a81a5de0 con 0x7f12a8076060 2024-03-21T08:29:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.560+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a81a61f0 con 0x7f12a8076060 2024-03-21T08:29:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.561+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a8110410 con 0x7f12a8076060 2024-03-21T08:29:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.563+0000 7f129fe00640 1 -- 172.21.15.3:0/1797847313 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f129800f850 con 0x7f12a8076060 2024-03-21T08:29:07.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.565+0000 7f129fe00640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f128c0a2fa0 0x7f128c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:07.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.565+0000 7f129fe00640 1 -- 172.21.15.3:0/1797847313 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f12980cf850 con 0x7f12a8076060 2024-03-21T08:29:07.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.565+0000 7f12a6a00640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f128c0a2fa0 0x7f128c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:07.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.566+0000 7f12a6a00640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f128c0a2fa0 0x7f128c0a5460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12a819f3e0 tx=0x7f1290005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:07.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.569+0000 7f129fe00640 1 -- 172.21.15.3:0/1797847313 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1298014030 con 0x7f12a8076060 2024-03-21T08:29:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T08:29:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2677607267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T08:29:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:29:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/102385330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T08:29:07.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: daemon mds.e assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T08:29:07.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T08:29:07.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/102385330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-21T08:29:07.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: fsmap cephfs:2 {0=h=up:active,1=e=up:creating} 10 up:standby 2024-03-21T08:29:07.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:07 smithi003 ceph-mon[31675]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-03-21T08:29:07.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:07.818+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f12a8061ae0 con 0x7f12a8076060 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2677607267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/102385330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: daemon mds.e assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/102385330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: fsmap cephfs:2 {0=h=up:active,1=e=up:creating} 10 up:standby 2024-03-21T08:29:07.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:07 smithi084 ceph-mon[35477]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2677607267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/102385330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: daemon mds.e assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/102385330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-21T08:29:07.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: fsmap cephfs:2 {0=h=up:active,1=e=up:creating} 10 up:standby 2024-03-21T08:29:07.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:07 smithi171 ceph-mon[35388]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-03-21T08:29:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.304+0000 7f129fe00640 1 -- 172.21.15.3:0/1797847313 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f129809c2c0 con 0x7f12a8076060 2024-03-21T08:29:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f128c0a2fa0 msgr2=0x7f128c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f128c0a2fa0 0x7f128c0a5460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12a819f3e0 tx=0x7f1290005be0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a8076060 msgr2=0x7f12a819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:08.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a8076060 0x7f12a819ded0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f129800cce0 tx=0x7f129800a620 comp rx=0 tx=0).stop 2024-03-21T08:29:08.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 shutdown_connections 2024-03-21T08:29:08.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f128c0a2fa0 0x7f128c0a5460 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12a8077c40 0x7f12a81a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12a80772a0 0x7f12a819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 --2- 172.21.15.3:0/1797847313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a8076060 0x7f12a819ded0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 >> 172.21.15.3:0/1797847313 conn(0x7f12a80fee60 msgr2=0x7f12a81005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 shutdown_connections 2024-03-21T08:29:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.308+0000 7f12ae200640 1 -- 172.21.15.3:0/1797847313 wait complete. 2024-03-21T08:29:08.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-21T08:29:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 -- 172.21.15.3:0/2885352927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c077c40 msgr2=0x7f7f1c0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 --2- 172.21.15.3:0/2885352927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c077c40 0x7f7f1c0784e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009ca0 tx=0x7f7f1402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 -- 172.21.15.3:0/2885352927 shutdown_connections 2024-03-21T08:29:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 --2- 172.21.15.3:0/2885352927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c077c40 0x7f7f1c0784e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 --2- 172.21.15.3:0/2885352927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f1c0772a0 0x7f7f1c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 --2- 172.21.15.3:0/2885352927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f1c076060 0x7f7f1c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 -- 172.21.15.3:0/2885352927 >> 172.21.15.3:0/2885352927 conn(0x7f7f1c0fee50 msgr2=0x7f7f1c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:08.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 -- 172.21.15.3:0/2885352927 shutdown_connections 2024-03-21T08:29:08.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.548+0000 7f7f21800640 1 -- 172.21.15.3:0/2885352927 wait complete. 2024-03-21T08:29:08.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.549+0000 7f7f21800640 1 Processor -- start 2024-03-21T08:29:08.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f21800640 1 -- start start 2024-03-21T08:29:08.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f21800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c076060 0x7f7f1c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f21800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f1c0772a0 0x7f7f1c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f21800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f1c077c40 0x7f7f1c1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f21800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c113800 con 0x7f7f1c076060 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f21800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c113680 con 0x7f7f1c0772a0 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f21800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c113980 con 0x7f7f1c077c40 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f1aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c076060 0x7f7f1c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f1aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c076060 0x7f7f1c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44160/0 (socket says 172.21.15.3:44160) 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f1aa00640 1 -- 172.21.15.3:0/1065504014 learned_addr learned my addr 172.21.15.3:0/1065504014 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.550+0000 7f7f1a000640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f1c0772a0 0x7f7f1c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f1b400640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f1c077c40 0x7f7f1c1a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f1aa00640 1 -- 172.21.15.3:0/1065504014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f1c077c40 msgr2=0x7f7f1c1a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f1aa00640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f1c077c40 0x7f7f1c1a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f1aa00640 1 -- 172.21.15.3:0/1065504014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f1c0772a0 msgr2=0x7f7f1c19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:08.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f1aa00640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f1c0772a0 0x7f7f1c19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f1aa00640 1 -- 172.21.15.3:0/1065504014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c1a5a90 con 0x7f7f1c076060 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f1aa00640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c076060 0x7f7f1c19de70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9f0 tx=0x7f7f0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.551+0000 7f7f13400640 1 -- 172.21.15.3:0/1065504014 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00cd80 con 0x7f7f1c076060 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.552+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c1a5d80 con 0x7f7f1c076060 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.552+0000 7f7f13400640 1 -- 172.21.15.3:0/1065504014 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f0c0108e0 con 0x7f7f1c076060 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.552+0000 7f7f13400640 1 -- 172.21.15.3:0/1065504014 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c018920 con 0x7f7f1c076060 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.552+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c1a6210 con 0x7f7f1c076060 2024-03-21T08:29:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.553+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c110230 con 0x7f7f1c076060 2024-03-21T08:29:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.555+0000 7f7f13400640 1 -- 172.21.15.3:0/1065504014 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f7f0c010a90 con 0x7f7f1c076060 2024-03-21T08:29:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.556+0000 7f7f13400640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7f000a2f20 0x7f7f000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.557+0000 7f7f13400640 1 -- 172.21.15.3:0/1065504014 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7f0c0cfa80 con 0x7f7f1c076060 2024-03-21T08:29:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.557+0000 7f7f1a000640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7f000a2f20 0x7f7f000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.558+0000 7f7f1a000640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7f000a2f20 0x7f7f000a53e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c19f380 tx=0x7f7f0800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.560+0000 7f7f13400640 1 -- 172.21.15.3:0/1065504014 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f0c0d1e20 con 0x7f7f1c076060 2024-03-21T08:29:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: pgmap v385: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:29:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1797847313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T08:29:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1797847313' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T08:29:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] up:active 2024-03-21T08:29:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: daemon mds.k assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T08:29:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T08:29:08.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: Cluster is now healthy 2024-03-21T08:29:08.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: fsmap cephfs:2 {0=h=up:active,1=e=up:active} 10 up:standby 2024-03-21T08:29:08.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:creating} 9 up:standby 2024-03-21T08:29:08.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:08 smithi003 ceph-mon[31675]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-03-21T08:29:08.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:08.815+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f7f1c02d070 con 0x7f7f1c076060 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: pgmap v385: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1797847313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1797847313' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] up:active 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: daemon mds.k assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: Cluster is now healthy 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: fsmap cephfs:2 {0=h=up:active,1=e=up:active} 10 up:standby 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:creating} 9 up:standby 2024-03-21T08:29:08.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:08 smithi084 ceph-mon[35477]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: pgmap v385: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1797847313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1797847313' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] up:active 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: daemon mds.k assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: Cluster is now healthy 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: fsmap cephfs:2 {0=h=up:active,1=e=up:active} 10 up:standby 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:creating} 9 up:standby 2024-03-21T08:29:08.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:08 smithi171 ceph-mon[35388]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-03-21T08:29:09.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.316+0000 7f7f13400640 1 -- 172.21.15.3:0/1065504014 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7f0c09bea0 con 0x7f7f1c076060 2024-03-21T08:29:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.319+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7f000a2f20 msgr2=0x7f7f000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.319+0000 7f7f21800640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7f000a2f20 0x7f7f000a53e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c19f380 tx=0x7f7f0800b390 comp rx=0 tx=0).stop 2024-03-21T08:29:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.319+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c076060 msgr2=0x7f7f1c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.319+0000 7f7f21800640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c076060 0x7f7f1c19de70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9f0 tx=0x7f7f0c00eeb0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 shutdown_connections 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7f000a2f20 0x7f7f000a53e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f1c077c40 0x7f7f1c1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f1c0772a0 0x7f7f1c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 --2- 172.21.15.3:0/1065504014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f1c076060 0x7f7f1c19de70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 >> 172.21.15.3:0/1065504014 conn(0x7f7f1c0fee50 msgr2=0x7f7f1c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 shutdown_connections 2024-03-21T08:29:09.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.320+0000 7f7f21800640 1 -- 172.21.15.3:0/1065504014 wait complete. 2024-03-21T08:29:09.343 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-21T08:29:09.344 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-21T08:29:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.569+0000 7fd44b200640 1 -- 172.21.15.3:0/49682341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c109690 msgr2=0x7fd44c10ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.569+0000 7fd44b200640 1 --2- 172.21.15.3:0/49682341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c109690 0x7fd44c10ff20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd444009ca0 tx=0x7fd44402c100 comp rx=0 tx=0).stop 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.570+0000 7fd44b200640 1 -- 172.21.15.3:0/49682341 shutdown_connections 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.570+0000 7fd44b200640 1 --2- 172.21.15.3:0/49682341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c109690 0x7fd44c10ff20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.570+0000 7fd44b200640 1 --2- 172.21.15.3:0/49682341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c108cd0 0x7fd44c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.570+0000 7fd44b200640 1 --2- 172.21.15.3:0/49682341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd44c107ae0 0x7fd44c107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.570+0000 7fd44b200640 1 -- 172.21.15.3:0/49682341 >> 172.21.15.3:0/49682341 conn(0x7fd44c076a90 msgr2=0x7fd44c078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.570+0000 7fd44b200640 1 -- 172.21.15.3:0/49682341 shutdown_connections 2024-03-21T08:29:09.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.570+0000 7fd44b200640 1 -- 172.21.15.3:0/49682341 wait complete. 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 Processor -- start 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 -- start start 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c107ae0 0x7fd44c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c108cd0 0x7fd44c19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd44c109690 0x7fd44c1a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c1135f0 con 0x7fd44c108cd0 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c113470 con 0x7fd44c109690 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd44b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c113770 con 0x7fd44c107ae0 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.571+0000 7fd449400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c108cd0 0x7fd44c19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c108cd0 0x7fd44c19e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44174/0 (socket says 172.21.15.3:44174) 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 -- 172.21.15.3:0/978907497 learned_addr learned my addr 172.21.15.3:0/978907497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:09.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449e00640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c107ae0 0x7fd44c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd44a800640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd44c109690 0x7fd44c1a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 -- 172.21.15.3:0/978907497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c107ae0 msgr2=0x7fd44c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c107ae0 0x7fd44c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 -- 172.21.15.3:0/978907497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd44c109690 msgr2=0x7fd44c1a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd44c109690 0x7fd44c1a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 -- 172.21.15.3:0/978907497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c1a5890 con 0x7fd44c108cd0 2024-03-21T08:29:09.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.572+0000 7fd449400640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c108cd0 0x7fd44c19e1c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c890 tx=0x7fd43800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.573+0000 7fd442a00640 1 -- 172.21.15.3:0/978907497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438004220 con 0x7fd44c108cd0 2024-03-21T08:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.573+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c1a5b80 con 0x7fd44c108cd0 2024-03-21T08:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.573+0000 7fd442a00640 1 -- 172.21.15.3:0/978907497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd4380043c0 con 0x7fd44c108cd0 2024-03-21T08:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.573+0000 7fd442a00640 1 -- 172.21.15.3:0/978907497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438004a00 con 0x7fd44c108cd0 2024-03-21T08:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.573+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c1a60d0 con 0x7fd44c108cd0 2024-03-21T08:29:09.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.574+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c10ee80 con 0x7fd44c108cd0 2024-03-21T08:29:09.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.578+0000 7fd442a00640 1 -- 172.21.15.3:0/978907497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd438004ba0 con 0x7fd44c108cd0 2024-03-21T08:29:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.579+0000 7fd442a00640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4300a2f10 0x7fd4300a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.579+0000 7fd442a00640 1 -- 172.21.15.3:0/978907497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd4380cf540 con 0x7fd44c108cd0 2024-03-21T08:29:09.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.580+0000 7fd449e00640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4300a2f10 0x7fd4300a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:09.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.580+0000 7fd449e00640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4300a2f10 0x7fd4300a53d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd44c19f150 tx=0x7fd43c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:09.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.583+0000 7fd442a00640 1 -- 172.21.15.3:0/978907497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd438014030 con 0x7fd44c108cd0 2024-03-21T08:29:09.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1065504014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T08:29:09.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1065504014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T08:29:09.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:active 2024-03-21T08:29:09.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 9 up:standby 2024-03-21T08:29:09.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:29:09.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.819+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd44c061ae0 con 0x7fd4300a2f10 2024-03-21T08:29:09.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.868+0000 7fd442a00640 1 -- 172.21.15.3:0/978907497 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd44c061ae0 con 0x7fd4300a2f10 2024-03-21T08:29:09.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.870+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4300a2f10 msgr2=0x7fd4300a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:09.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.870+0000 7fd44b200640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4300a2f10 0x7fd4300a53d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd44c19f150 tx=0x7fd43c005f90 comp rx=0 tx=0).stop 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.870+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c108cd0 msgr2=0x7fd44c19e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.870+0000 7fd44b200640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c108cd0 0x7fd44c19e1c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c890 tx=0x7fd43800cd50 comp rx=0 tx=0).stop 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.870+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 shutdown_connections 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.870+0000 7fd44b200640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd44c109690 0x7fd44c1a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.871+0000 7fd44b200640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd44c108cd0 0x7fd44c19e1c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.871+0000 7fd44b200640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd4300a2f10 0x7fd4300a53d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.871+0000 7fd44b200640 1 --2- 172.21.15.3:0/978907497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c107ae0 0x7fd44c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.871+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 >> 172.21.15.3:0/978907497 conn(0x7fd44c076a90 msgr2=0x7fd44c10c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.871+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 shutdown_connections 2024-03-21T08:29:09.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:09.871+0000 7fd44b200640 1 -- 172.21.15.3:0/978907497 wait complete. 2024-03-21T08:29:09.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:09 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1065504014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T08:29:09.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:09 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1065504014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T08:29:09.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:09 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:active 2024-03-21T08:29:09.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:09 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 9 up:standby 2024-03-21T08:29:09.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:09 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:29:09.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:09 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1065504014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T08:29:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:09 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1065504014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T08:29:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:09 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:active 2024-03-21T08:29:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:09 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 9 up:standby 2024-03-21T08:29:09.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:09 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:29:09.887 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:29:09.829+0000 7f4674400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:29:09.887 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:29:09.829+0000 7f4674400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:29:09.888 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:29:09.829+0000 7f4674400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:29:09.888 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:29:09.829+0000 7f4674400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:29:09.888 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:29:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:29:09.829+0000 7f4674400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:29:09.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-21T08:29:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 -- 172.21.15.3:0/3154659998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01016d0 msgr2=0x7f2aa0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/3154659998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01016d0 0x7f2aa0101af0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009ca0 tx=0x7f2a9002c100 comp rx=0 tx=0).stop 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 -- 172.21.15.3:0/3154659998 shutdown_connections 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/3154659998 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2aa0103280 0x7f2aa0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/3154659998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa01028c0 0x7f2aa0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/3154659998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01016d0 0x7f2aa0101af0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 -- 172.21.15.3:0/3154659998 >> 172.21.15.3:0/3154659998 conn(0x7f2aa00fcc60 msgr2=0x7f2aa00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.089+0000 7f2aa4a00640 1 -- 172.21.15.3:0/3154659998 shutdown_connections 2024-03-21T08:29:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.090+0000 7f2aa4a00640 1 -- 172.21.15.3:0/3154659998 wait complete. 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.090+0000 7f2aa4a00640 1 Processor -- start 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2aa4a00640 1 -- start start 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2aa4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2aa01016d0 0x7f2aa0199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2aa4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01028c0 0x7f2aa019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2aa4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0103280 0x7f2aa01a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2aa4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa010d200 con 0x7f2aa01028c0 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2aa4a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa010d080 con 0x7f2aa0103280 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2aa4a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa010d380 con 0x7f2aa01016d0 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2a95400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01028c0 0x7f2aa019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2a95400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01028c0 0x7f2aa019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50982/0 (socket says 172.21.15.3:50982) 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.091+0000 7f2a95400640 1 -- 172.21.15.3:0/1488984063 learned_addr learned my addr 172.21.15.3:0/1488984063 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a9de00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2aa01016d0 0x7f2aa0199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a9e800640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0103280 0x7f2aa01a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a95400640 1 -- 172.21.15.3:0/1488984063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2aa01016d0 msgr2=0x7f2aa0199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a95400640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2aa01016d0 0x7f2aa0199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a95400640 1 -- 172.21.15.3:0/1488984063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0103280 msgr2=0x7f2aa01a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a95400640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0103280 0x7f2aa01a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a95400640 1 -- 172.21.15.3:0/1488984063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa01a1780 con 0x7f2aa01028c0 2024-03-21T08:29:10.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a9de00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2aa01016d0 0x7f2aa0199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:29:10.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a9e800640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0103280 0x7f2aa01a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:29:10.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a95400640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01028c0 0x7f2aa019a0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e9f0 tx=0x7f2a8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:10.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.092+0000 7f2a97400640 1 -- 172.21.15.3:0/1488984063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800cd80 con 0x7f2aa01028c0 2024-03-21T08:29:10.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.093+0000 7f2a97400640 1 -- 172.21.15.3:0/1488984063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a880108e0 con 0x7f2aa01028c0 2024-03-21T08:29:10.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.093+0000 7f2a97400640 1 -- 172.21.15.3:0/1488984063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88018920 con 0x7f2aa01028c0 2024-03-21T08:29:10.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.093+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa01a1a70 con 0x7f2aa01028c0 2024-03-21T08:29:10.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.093+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa01a1f70 con 0x7f2aa01028c0 2024-03-21T08:29:10.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.094+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa01a2260 con 0x7f2aa01028c0 2024-03-21T08:29:10.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.097+0000 7f2a97400640 1 -- 172.21.15.3:0/1488984063 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f2a88010a90 con 0x7f2aa01028c0 2024-03-21T08:29:10.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.099+0000 7f2a97400640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a840a2fa0 0x7f2a840a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.099+0000 7f2a97400640 1 -- 172.21.15.3:0/1488984063 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2a880cfb50 con 0x7f2aa01028c0 2024-03-21T08:29:10.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.099+0000 7f2a9de00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a840a2fa0 0x7f2a840a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.100+0000 7f2a9de00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a840a2fa0 0x7f2a840a5460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2a9002c710 tx=0x7f2a900324c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:10.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.103+0000 7f2a97400640 1 -- 172.21.15.3:0/1488984063 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a8809c5c0 con 0x7f2aa01028c0 2024-03-21T08:29:10.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.338+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aa0074ff0 con 0x7f2a840a2fa0 2024-03-21T08:29:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.376+0000 7f2a97400640 1 -- 172.21.15.3:0/1488984063 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2aa0074ff0 con 0x7f2a840a2fa0 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.378+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a840a2fa0 msgr2=0x7f2a840a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a840a2fa0 0x7f2a840a5460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2a9002c710 tx=0x7f2a900324c0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01028c0 msgr2=0x7f2aa019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01028c0 0x7f2aa019a0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e9f0 tx=0x7f2a8800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 shutdown_connections 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2aa0103280 0x7f2aa01a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa01028c0 0x7f2aa019a0a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2a840a2fa0 0x7f2a840a5460 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 --2- 172.21.15.3:0/1488984063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2aa01016d0 0x7f2aa0199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 >> 172.21.15.3:0/1488984063 conn(0x7f2aa00fcc60 msgr2=0x7f2aa00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 shutdown_connections 2024-03-21T08:29:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.379+0000 7f2aa4a00640 1 -- 172.21.15.3:0/1488984063 wait complete. 2024-03-21T08:29:10.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 -- 172.21.15.3:0/352511196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1030a0 msgr2=0x7f9e8c109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 --2- 172.21.15.3:0/352511196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1030a0 0x7f9e8c109930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009ca0 tx=0x7f9e8402c100 comp rx=0 tx=0).stop 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 -- 172.21.15.3:0/352511196 shutdown_connections 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 --2- 172.21.15.3:0/352511196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1030a0 0x7f9e8c109930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 --2- 172.21.15.3:0/352511196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1026e0 0x7f9e8c102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 --2- 172.21.15.3:0/352511196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e8c1014f0 0x7f9e8c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 -- 172.21.15.3:0/352511196 >> 172.21.15.3:0/352511196 conn(0x7f9e8c0fca60 msgr2=0x7f9e8c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 -- 172.21.15.3:0/352511196 shutdown_connections 2024-03-21T08:29:10.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.611+0000 7f9e91800640 1 -- 172.21.15.3:0/352511196 wait complete. 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.612+0000 7f9e91800640 1 Processor -- start 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.612+0000 7f9e91800640 1 -- start start 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.612+0000 7f9e91800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1014f0 0x7f9e8c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.612+0000 7f9e91800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1026e0 0x7f9e8c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e91800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e8c1030a0 0x7f9e8c1a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e91800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c10d030 con 0x7f9e8c1014f0 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e91800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c10ceb0 con 0x7f9e8c1026e0 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e91800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c10d1b0 con 0x7f9e8c1030a0 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1014f0 0x7f9e8c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1014f0 0x7f9e8c199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50994/0 (socket says 172.21.15.3:50994) 2024-03-21T08:29:10.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 -- 172.21.15.3:0/2737791015 learned_addr learned my addr 172.21.15.3:0/2737791015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:10.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e83e00640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1026e0 0x7f9e8c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 -- 172.21.15.3:0/2737791015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e8c1030a0 msgr2=0x7f9e8c1a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e90e00640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e8c1030a0 0x7f9e8c1a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e8c1030a0 0x7f9e8c1a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 -- 172.21.15.3:0/2737791015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1026e0 msgr2=0x7f9e8c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1026e0 0x7f9e8c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.613+0000 7f9e8be00640 1 -- 172.21.15.3:0/2737791015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c1a14b0 con 0x7f9e8c1014f0 2024-03-21T08:29:10.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.614+0000 7f9e8be00640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1014f0 0x7f9e8c199890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00e9c0 tx=0x7f9e7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:10.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.614+0000 7f9e89600640 1 -- 172.21.15.3:0/2737791015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00cd80 con 0x7f9e8c1014f0 2024-03-21T08:29:10.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.614+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c1a17a0 con 0x7f9e8c1014f0 2024-03-21T08:29:10.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.614+0000 7f9e89600640 1 -- 172.21.15.3:0/2737791015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e7c0035f0 con 0x7f9e8c1014f0 2024-03-21T08:29:10.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.614+0000 7f9e89600640 1 -- 172.21.15.3:0/2737791015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c010690 con 0x7f9e8c1014f0 2024-03-21T08:29:10.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.614+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c1a1cf0 con 0x7f9e8c1014f0 2024-03-21T08:29:10.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.615+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c1a1fb0 con 0x7f9e8c1014f0 2024-03-21T08:29:10.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.617+0000 7f9e89600640 1 -- 172.21.15.3:0/2737791015 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f9e7c0037a0 con 0x7f9e8c1014f0 2024-03-21T08:29:10.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.619+0000 7f9e89600640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e700a2f20 0x7f9e700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:10.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.619+0000 7f9e89600640 1 -- 172.21.15.3:0/2737791015 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9e7c0cfd60 con 0x7f9e8c1014f0 2024-03-21T08:29:10.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.619+0000 7f9e83e00640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e700a2f20 0x7f9e700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:10.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.620+0000 7f9e83e00640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e700a2f20 0x7f9e700a53e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9e740095f0 tx=0x7f9e74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:10.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.623+0000 7f9e89600640 1 -- 172.21.15.3:0/2737791015 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9e7c09c7d0 con 0x7f9e8c1014f0 2024-03-21T08:29:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.852+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e8c074ff0 con 0x7f9e700a2f20 2024-03-21T08:29:10.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.857+0000 7f9e89600640 1 -- 172.21.15.3:0/2737791015 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f9e8c074ff0 con 0x7f9e700a2f20 2024-03-21T08:29:10.859 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393 2024-03-21T08:29:10.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.859+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e700a2f20 msgr2=0x7f9e700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.859+0000 7f9e91800640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e700a2f20 0x7f9e700a53e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9e740095f0 tx=0x7f9e74005f90 comp rx=0 tx=0).stop 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.859+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1014f0 msgr2=0x7f9e8c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.859+0000 7f9e91800640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1014f0 0x7f9e8c199890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00e9c0 tx=0x7f9e7c00ee80 comp rx=0 tx=0).stop 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 shutdown_connections 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e700a2f20 0x7f9e700a53e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e8c1030a0 0x7f9e8c1a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e8c1026e0 0x7f9e8c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 --2- 172.21.15.3:0/2737791015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e8c1014f0 0x7f9e8c199890 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 >> 172.21.15.3:0/2737791015 conn(0x7f9e8c0fca60 msgr2=0x7f9e8c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 shutdown_connections 2024-03-21T08:29:10.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:10.860+0000 7f9e91800640 1 -- 172.21.15.3:0/2737791015 wait complete. 2024-03-21T08:29:10.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:10 smithi084 ceph-mon[35477]: pgmap v386: 129 pgs: 1 activating, 1 peering, 127 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-03-21T08:29:10.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:10 smithi084 ceph-mon[35477]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:10.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:10 smithi003 ceph-mon[31675]: pgmap v386: 129 pgs: 1 activating, 1 peering, 127 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-03-21T08:29:10.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:10 smithi003 ceph-mon[31675]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:10.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-21T08:29:10.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:10 smithi171 ceph-mon[35388]: pgmap v386: 129 pgs: 1 activating, 1 peering, 127 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-03-21T08:29:10.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:10 smithi171 ceph-mon[35388]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 -- 172.21.15.3:0/3707390338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 msgr2=0x7f03c8101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3707390338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8101920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009900 tx=0x7f03b802bc40 comp rx=0 tx=0).stop 2024-03-21T08:29:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 -- 172.21.15.3:0/3707390338 shutdown_connections 2024-03-21T08:29:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3707390338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03c81030b0 0x7f03c8109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3707390338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c81026f0 0x7f03c8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3707390338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8101920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 -- 172.21.15.3:0/3707390338 >> 172.21.15.3:0/3707390338 conn(0x7f03c80fcab0 msgr2=0x7f03c80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 -- 172.21.15.3:0/3707390338 shutdown_connections 2024-03-21T08:29:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.086+0000 7f03ccc00640 1 -- 172.21.15.3:0/3707390338 wait complete. 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.087+0000 7f03ccc00640 1 Processor -- start 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03ccc00640 1 -- start start 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c81026f0 0x7f03c8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03c81030b0 0x7f03c8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03ccc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c810d0d0 con 0x7f03c8101500 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03ccc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c810cf50 con 0x7f03c81030b0 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03ccc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c810d250 con 0x7f03c81026f0 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51012/0 (socket says 172.21.15.3:51012) 2024-03-21T08:29:11.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.088+0000 7f03c7400640 1 -- 172.21.15.3:0/3393941522 learned_addr learned my addr 172.21.15.3:0/3393941522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c7e00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03c81030b0 0x7f03c8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03bfe00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c81026f0 0x7f03c8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c7400640 1 -- 172.21.15.3:0/3393941522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c81026f0 msgr2=0x7f03c8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c7400640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c81026f0 0x7f03c8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c7400640 1 -- 172.21.15.3:0/3393941522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03c81030b0 msgr2=0x7f03c8073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c7400640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03c81030b0 0x7f03c8073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c7400640 1 -- 172.21.15.3:0/3393941522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8074860 con 0x7f03c8101500 2024-03-21T08:29:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c7400640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8075450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005ab0 tx=0x7f03b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.089+0000 7f03c4c00640 1 -- 172.21.15.3:0/3393941522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8043020 con 0x7f03c8101500 2024-03-21T08:29:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.090+0000 7f03c4c00640 1 -- 172.21.15.3:0/3393941522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f03b80054c0 con 0x7f03c8101500 2024-03-21T08:29:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.090+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c81a18f0 con 0x7f03c8101500 2024-03-21T08:29:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.090+0000 7f03c4c00640 1 -- 172.21.15.3:0/3393941522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8003d10 con 0x7f03c8101500 2024-03-21T08:29:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.090+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c81a1ec0 con 0x7f03c8101500 2024-03-21T08:29:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.091+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c8108890 con 0x7f03c8101500 2024-03-21T08:29:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.093+0000 7f03c4c00640 1 -- 172.21.15.3:0/3393941522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f03b8046020 con 0x7f03c8101500 2024-03-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.095+0000 7f03c4c00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03ac0a2f20 0x7f03ac0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.095+0000 7f03c4c00640 1 -- 172.21.15.3:0/3393941522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f03b80f1ba0 con 0x7f03c8101500 2024-03-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.095+0000 7f03bfe00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03ac0a2f20 0x7f03ac0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.095+0000 7f03bfe00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03ac0a2f20 0x7f03ac0a53e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03c80751c0 tx=0x7f03b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:11.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.099+0000 7f03c4c00640 1 -- 172.21.15.3:0/3393941522 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f03b804d020 con 0x7f03c8101500 2024-03-21T08:29:11.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.323+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f03c8061b70 con 0x7f03ac0a2f20 2024-03-21T08:29:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.363+0000 7f03c4c00640 1 -- 172.21.15.3:0/3393941522 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f03c8061b70 con 0x7f03ac0a2f20 2024-03-21T08:29:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.366+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03ac0a2f20 msgr2=0x7f03ac0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.366+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03ac0a2f20 0x7f03ac0a53e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03c80751c0 tx=0x7f03b0008040 comp rx=0 tx=0).stop 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.366+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 msgr2=0x7f03c8075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.366+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8075450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005ab0 tx=0x7f03b80042f0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 shutdown_connections 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03c81030b0 0x7f03c8073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f03ac0a2f20 0x7f03ac0a53e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c81026f0 0x7f03c8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 --2- 172.21.15.3:0/3393941522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03c8101500 0x7f03c8075450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 >> 172.21.15.3:0/3393941522 conn(0x7f03c80fcab0 msgr2=0x7f03c8105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 shutdown_connections 2024-03-21T08:29:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.367+0000 7f03ccc00640 1 -- 172.21.15.3:0/3393941522 wait complete. 2024-03-21T08:29:11.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-21T08:29:11.530 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:29:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h[65129]: 2024-03-21T08:29:11.145+0000 7f8c7ec00700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-21T08:29:11.530 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:29:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h[65129]: 2024-03-21T08:29:11.145+0000 7f8c7ec00700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.609+0000 7f027f400640 1 -- 172.21.15.3:0/2203055683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 msgr2=0x7f02780784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.609+0000 7f027f400640 1 --2- 172.21.15.3:0/2203055683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02780784e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0270009860 tx=0x7f027002bb70 comp rx=0 tx=0).stop 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.610+0000 7f027f400640 1 -- 172.21.15.3:0/2203055683 shutdown_connections 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.610+0000 7f027f400640 1 --2- 172.21.15.3:0/2203055683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02780784e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.610+0000 7f027f400640 1 --2- 172.21.15.3:0/2203055683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02780772a0 0x7f0278077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.610+0000 7f027f400640 1 --2- 172.21.15.3:0/2203055683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0278076060 0x7f0278076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.610+0000 7f027f400640 1 -- 172.21.15.3:0/2203055683 >> 172.21.15.3:0/2203055683 conn(0x7f02780fee50 msgr2=0x7f0278101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.610+0000 7f027f400640 1 -- 172.21.15.3:0/2203055683 shutdown_connections 2024-03-21T08:29:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.610+0000 7f027f400640 1 -- 172.21.15.3:0/2203055683 wait complete. 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 Processor -- start 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 -- start start 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0278076060 0x7f027819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02780772a0 0x7f027819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02781a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278113800 con 0x7f0278077c40 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278113680 con 0x7f02780772a0 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.611+0000 7f027f400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278113980 con 0x7f0278076060 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02781a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02781a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51022/0 (socket says 172.21.15.3:51022) 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 -- 172.21.15.3:0/3507231765 learned_addr learned my addr 172.21.15.3:0/3507231765 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f0277e00640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02780772a0 0x7f027819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027ca00640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0278076060 0x7f027819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 -- 172.21.15.3:0/3507231765 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0278076060 msgr2=0x7f027819de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0278076060 0x7f027819de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 -- 172.21.15.3:0/3507231765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02780772a0 msgr2=0x7f027819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02780772a0 0x7f027819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 -- 172.21.15.3:0/3507231765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02781a5aa0 con 0x7f0278077c40 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.612+0000 7f027d400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02781a5430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f027002c180 tx=0x7f0270004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.613+0000 7f0275600640 1 -- 172.21.15.3:0/3507231765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270041020 con 0x7f0278077c40 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.613+0000 7f0275600640 1 -- 172.21.15.3:0/3507231765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f027002c9c0 con 0x7f0278077c40 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.613+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02781a5d30 con 0x7f0278077c40 2024-03-21T08:29:11.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.613+0000 7f0275600640 1 -- 172.21.15.3:0/3507231765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270039040 con 0x7f0278077c40 2024-03-21T08:29:11.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.613+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781a61d0 con 0x7f0278077c40 2024-03-21T08:29:11.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.614+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0278110230 con 0x7f0278077c40 2024-03-21T08:29:11.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.616+0000 7f0275600640 1 -- 172.21.15.3:0/3507231765 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f0270044020 con 0x7f0278077c40 2024-03-21T08:29:11.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.617+0000 7f0275600640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f025c0a2fb0 0x7f025c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:11.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.618+0000 7f0275600640 1 -- 172.21.15.3:0/3507231765 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f02700f1340 con 0x7f0278077c40 2024-03-21T08:29:11.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.618+0000 7f027ca00640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f025c0a2fb0 0x7f025c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:11.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.618+0000 7f027ca00640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f025c0a2fb0 0x7f025c0a5470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0268005fd0 tx=0x7f0268009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:11.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.622+0000 7f0275600640 1 -- 172.21.15.3:0/3507231765 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f027004b020 con 0x7f0278077c40 2024-03-21T08:29:11.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.859+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0278061b70 con 0x7f025c0a2fb0 2024-03-21T08:29:11.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.869+0000 7f0275600640 1 -- 172.21.15.3:0/3507231765 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f0278061b70 con 0x7f025c0a2fb0 2024-03-21T08:29:11.871 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_1/c4aad153-0af7-474f-a10e-1f6c8db61a5e 2024-03-21T08:29:11.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.871+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f025c0a2fb0 msgr2=0x7f025c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.871+0000 7f027f400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f025c0a2fb0 0x7f025c0a5470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0268005fd0 tx=0x7f0268009870 comp rx=0 tx=0).stop 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 msgr2=0x7f02781a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02781a5430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f027002c180 tx=0x7f0270004270 comp rx=0 tx=0).stop 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 shutdown_connections 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0278077c40 0x7f02781a5430 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02780772a0 0x7f027819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f025c0a2fb0 0x7f025c0a5470 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 --2- 172.21.15.3:0/3507231765 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0278076060 0x7f027819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 >> 172.21.15.3:0/3507231765 conn(0x7f02780fee50 msgr2=0x7f0278102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 shutdown_connections 2024-03-21T08:29:11.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:11.872+0000 7f027f400640 1 -- 172.21.15.3:0/3507231765 wait complete. 2024-03-21T08:29:11.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:11 smithi084 ceph-mon[35477]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:11.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:11 smithi084 ceph-mon[35477]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:11.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:11 smithi171 ceph-mon[35388]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:11.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:11 smithi171 ceph-mon[35388]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:11.889 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:11 smithi003 ceph-mon[31675]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:11.889 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:11 smithi003 ceph-mon[31675]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:11.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:29:12.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 -- 172.21.15.3:0/1268447900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1049b0 msgr2=0x7f9a3c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 --2- 172.21.15.3:0/1268447900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1049b0 0x7f9a3c104e30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009900 tx=0x7f9a2402bc40 comp rx=0 tx=0).stop 2024-03-21T08:29:12.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 -- 172.21.15.3:0/1268447900 shutdown_connections 2024-03-21T08:29:12.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 --2- 172.21.15.3:0/1268447900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a3c105370 0x7f9a3c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 --2- 172.21.15.3:0/1268447900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1049b0 0x7f9a3c104e30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 --2- 172.21.15.3:0/1268447900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a3c1037c0 0x7f9a3c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 -- 172.21.15.3:0/1268447900 >> 172.21.15.3:0/1268447900 conn(0x7f9a3c0fed50 msgr2=0x7f9a3c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:12.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 -- 172.21.15.3:0/1268447900 shutdown_connections 2024-03-21T08:29:12.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.107+0000 7f9a3be00640 1 -- 172.21.15.3:0/1268447900 wait complete. 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 Processor -- start 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 -- start start 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1037c0 0x7f9a3c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a3c1049b0 0x7f9a3c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a3c105370 0x7f9a3c1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c10f320 con 0x7f9a3c1037c0 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c10f1a0 con 0x7f9a3c105370 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.108+0000 7f9a3be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c10f4a0 con 0x7f9a3c1049b0 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1037c0 0x7f9a3c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1037c0 0x7f9a3c199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51046/0 (socket says 172.21.15.3:51046) 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 -- 172.21.15.3:0/3459725147 learned_addr learned my addr 172.21.15.3:0/3459725147 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:12.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3b400640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a3c105370 0x7f9a3c1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a32000640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a3c1049b0 0x7f9a3c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 -- 172.21.15.3:0/3459725147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a3c1049b0 msgr2=0x7f9a3c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a3c1049b0 0x7f9a3c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 -- 172.21.15.3:0/3459725147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a3c105370 msgr2=0x7f9a3c1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a3c105370 0x7f9a3c1a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 -- 172.21.15.3:0/3459725147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c1a1580 con 0x7f9a3c1037c0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.109+0000 7f9a3aa00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1037c0 0x7f9a3c199960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c970 tx=0x7f9a2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.110+0000 7f9a40a00640 1 -- 172.21.15.3:0/3459725147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c007c10 con 0x7f9a3c1037c0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.110+0000 7f9a40a00640 1 -- 172.21.15.3:0/3459725147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a2c007db0 con 0x7f9a3c1037c0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.110+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c1a1870 con 0x7f9a3c1037c0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.110+0000 7f9a40a00640 1 -- 172.21.15.3:0/3459725147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00f870 con 0x7f9a3c1037c0 2024-03-21T08:29:12.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.110+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c1a1de0 con 0x7f9a3c1037c0 2024-03-21T08:29:12.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.111+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c10aa80 con 0x7f9a3c1037c0 2024-03-21T08:29:12.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.113+0000 7f9a40a00640 1 -- 172.21.15.3:0/3459725147 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f9a2c00fa10 con 0x7f9a3c1037c0 2024-03-21T08:29:12.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.114+0000 7f9a40a00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a200a3020 0x7f9a200a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.114+0000 7f9a40a00640 1 -- 172.21.15.3:0/3459725147 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9a2c0cfcd0 con 0x7f9a3c1037c0 2024-03-21T08:29:12.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.115+0000 7f9a32000640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a200a3020 0x7f9a200a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.115+0000 7f9a32000640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a200a3020 0x7f9a200a54e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9a24006010 tx=0x7f9a24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:12.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.119+0000 7f9a40a00640 1 -- 172.21.15.3:0/3459725147 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9a2c09c670 con 0x7f9a3c1037c0 2024-03-21T08:29:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.353+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a3c02d070 con 0x7f9a3c1037c0 2024-03-21T08:29:12.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.355+0000 7f9a40a00640 1 -- 172.21.15.3:0/3459725147 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f9a2c09bf70 con 0x7f9a3c1037c0 2024-03-21T08:29:12.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:12.357 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:29:09.323523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34409,34472]},"id":1}]} 2024-03-21T08:29:12.358 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a200a3020 msgr2=0x7f9a200a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a200a3020 0x7f9a200a54e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9a24006010 tx=0x7f9a24000bc0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1037c0 msgr2=0x7f9a3c199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1037c0 0x7f9a3c199960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c970 tx=0x7f9a2c00ce30 comp rx=0 tx=0).stop 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 shutdown_connections 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a3c105370 0x7f9a3c1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a200a3020 0x7f9a200a54e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a3c1049b0 0x7f9a3c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 --2- 172.21.15.3:0/3459725147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a3c1037c0 0x7f9a3c199960 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 >> 172.21.15.3:0/3459725147 conn(0x7f9a3c0fed50 msgr2=0x7f9a3c107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.358+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 shutdown_connections 2024-03-21T08:29:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.359+0000 7f9a3be00640 1 -- 172.21.15.3:0/3459725147 wait complete. 2024-03-21T08:29:12.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:29:12.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 -- 172.21.15.3:0/501061631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c40756e0 msgr2=0x7fa6c4073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 --2- 172.21.15.3:0/501061631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c40756e0 0x7fa6c4073b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009900 tx=0x7fa6b402bc30 comp rx=0 tx=0).stop 2024-03-21T08:29:12.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 -- 172.21.15.3:0/501061631 shutdown_connections 2024-03-21T08:29:12.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 --2- 172.21.15.3:0/501061631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c4109460 0x7fa6c410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 --2- 172.21.15.3:0/501061631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c4074100 0x7fa6c4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 --2- 172.21.15.3:0/501061631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c40756e0 0x7fa6c4073b30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 -- 172.21.15.3:0/501061631 >> 172.21.15.3:0/501061631 conn(0x7fa6c40fcb30 msgr2=0x7fa6c40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 -- 172.21.15.3:0/501061631 shutdown_connections 2024-03-21T08:29:12.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.598+0000 7fa6c9400640 1 -- 172.21.15.3:0/501061631 wait complete. 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.599+0000 7fa6c9400640 1 Processor -- start 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.599+0000 7fa6c9400640 1 -- start start 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c4074100 0x7fa6c41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c9400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c40756e0 0x7fa6c4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c4109460 0x7fa6c41a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c9400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c410f2b0 con 0x7fa6c4074100 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c9400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c410f130 con 0x7fa6c4109460 2024-03-21T08:29:12.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c9400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c410f430 con 0x7fa6c40756e0 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c4074100 0x7fa6c41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c4074100 0x7fa6c41999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51062/0 (socket says 172.21.15.3:51062) 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 -- 172.21.15.3:0/3382867919 learned_addr learned my addr 172.21.15.3:0/3382867919 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 -- 172.21.15.3:0/3382867919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c40756e0 msgr2=0x7fa6c4199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6bb400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c40756e0 0x7fa6c4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c8a00640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c4109460 0x7fa6c41a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c40756e0 0x7fa6c4199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 -- 172.21.15.3:0/3382867919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c4109460 msgr2=0x7fa6c41a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c4109460 0x7fa6c41a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6c3e00640 1 -- 172.21.15.3:0/3382867919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c41a15d0 con 0x7fa6c4074100 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.600+0000 7fa6bb400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c40756e0 0x7fa6c4199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.601+0000 7fa6c3e00640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c4074100 0x7fa6c41999b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4006010 tx=0x7fa6b4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.601+0000 7fa6c1600640 1 -- 172.21.15.3:0/3382867919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4043020 con 0x7fa6c4074100 2024-03-21T08:29:12.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.601+0000 7fa6c1600640 1 -- 172.21.15.3:0/3382867919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa6b40043d0 con 0x7fa6c4074100 2024-03-21T08:29:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.601+0000 7fa6c1600640 1 -- 172.21.15.3:0/3382867919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4003b80 con 0x7fa6c4074100 2024-03-21T08:29:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.601+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c41a1860 con 0x7fa6c4074100 2024-03-21T08:29:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.601+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c41a1db0 con 0x7fa6c4074100 2024-03-21T08:29:12.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.603+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c4108190 con 0x7fa6c4074100 2024-03-21T08:29:12.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.604+0000 7fa6c1600640 1 -- 172.21.15.3:0/3382867919 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fa6b4046020 con 0x7fa6c4074100 2024-03-21T08:29:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.606+0000 7fa6c1600640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6a80a2fa0 0x7fa6a80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.606+0000 7fa6c1600640 1 -- 172.21.15.3:0/3382867919 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa6b40f1b40 con 0x7fa6c4074100 2024-03-21T08:29:12.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.606+0000 7fa6bb400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6a80a2fa0 0x7fa6a80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:12.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.607+0000 7fa6bb400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6a80a2fa0 0x7fa6a80a5460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6c419aec0 tx=0x7fa6ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:12.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.610+0000 7fa6c1600640 1 -- 172.21.15.3:0/3382867919 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa6b404d020 con 0x7fa6c4074100 2024-03-21T08:29:12.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:12 smithi003 ceph-mon[31675]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:12.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:12 smithi003 ceph-mon[31675]: pgmap v387: 129 pgs: 1 activating, 128 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-03-21T08:29:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:12 smithi003 ceph-mon[31675]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-21T08:29:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:12 smithi003 ceph-mon[31675]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:12.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:12 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3459725147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:29:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.811+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c40622a0 con 0x7fa6c4074100 2024-03-21T08:29:12.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.815+0000 7fa6c1600640 1 -- 172.21.15.3:0/3382867919 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24506 (secure 0 0 0) 0x7fa6b40be560 con 0x7fa6c4074100 2024-03-21T08:29:12.818 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:12.818 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":94,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:29:06.501461+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6a80a2fa0 msgr2=0x7fa6a80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6a80a2fa0 0x7fa6a80a5460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6c419aec0 tx=0x7fa6ac005f90 comp rx=0 tx=0).stop 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c4074100 msgr2=0x7fa6c41999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c4074100 0x7fa6c41999b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4006010 tx=0x7fa6b4005e40 comp rx=0 tx=0).stop 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 shutdown_connections 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c4109460 0x7fa6c41a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6a80a2fa0 0x7fa6a80a5460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6c40756e0 0x7fa6c4199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 --2- 172.21.15.3:0/3382867919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c4074100 0x7fa6c41999b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 >> 172.21.15.3:0/3382867919 conn(0x7fa6c40fcb30 msgr2=0x7fa6c4105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 shutdown_connections 2024-03-21T08:29:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:12.818+0000 7fa6c9400640 1 -- 172.21.15.3:0/3382867919 wait complete. 2024-03-21T08:29:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-21T08:29:12.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T08:29:12.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:12 smithi084 ceph-mon[35477]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:12.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:12 smithi084 ceph-mon[35477]: pgmap v387: 129 pgs: 1 activating, 128 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-03-21T08:29:12.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:12 smithi084 ceph-mon[35477]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-21T08:29:12.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:12 smithi084 ceph-mon[35477]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:12.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:12 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3459725147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:29:12.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:12 smithi171 ceph-mon[35388]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:12.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:12 smithi171 ceph-mon[35388]: pgmap v387: 129 pgs: 1 activating, 128 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-03-21T08:29:12.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:12 smithi171 ceph-mon[35388]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-21T08:29:12.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:12 smithi171 ceph-mon[35388]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:29:12.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:12 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3459725147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:29:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 -- 172.21.15.3:0/677847268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc107ce0 msgr2=0x7f4cfc108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 --2- 172.21.15.3:0/677847268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc107ce0 0x7f4cfc108100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009900 tx=0x7f4cec02bc30 comp rx=0 tx=0).stop 2024-03-21T08:29:13.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 -- 172.21.15.3:0/677847268 shutdown_connections 2024-03-21T08:29:13.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 --2- 172.21.15.3:0/677847268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cfc109890 0x7f4cfc110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 --2- 172.21.15.3:0/677847268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc108ed0 0x7f4cfc109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 --2- 172.21.15.3:0/677847268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc107ce0 0x7f4cfc108100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 -- 172.21.15.3:0/677847268 >> 172.21.15.3:0/677847268 conn(0x7f4cfc076ca0 msgr2=0x7f4cfc0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:13.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 -- 172.21.15.3:0/677847268 shutdown_connections 2024-03-21T08:29:13.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.055+0000 7f4d02e00640 1 -- 172.21.15.3:0/677847268 wait complete. 2024-03-21T08:29:13.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.056+0000 7f4d02e00640 1 Processor -- start 2024-03-21T08:29:13.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.056+0000 7f4d02e00640 1 -- start start 2024-03-21T08:29:13.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d02e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cfc107ce0 0x7f4cfc19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d02e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc108ed0 0x7f4cfc19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d02e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc109890 0x7f4cfc1a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d02e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc113850 con 0x7f4cfc109890 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d02e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc1136d0 con 0x7f4cfc107ce0 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d02e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc1139d0 con 0x7f4cfc108ed0 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d00e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc109890 0x7f4cfc1a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d00e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc109890 0x7f4cfc1a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51080/0 (socket says 172.21.15.3:51080) 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d00e00640 1 -- 172.21.15.3:0/1723422430 learned_addr learned my addr 172.21.15.3:0/1723422430 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4cfbe00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cfc107ce0 0x7f4cfc19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4cf3e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc108ed0 0x7f4cfc19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.057+0000 7f4d00e00640 1 -- 172.21.15.3:0/1723422430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc108ed0 msgr2=0x7f4cfc19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4d00e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc108ed0 0x7f4cfc19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4d00e00640 1 -- 172.21.15.3:0/1723422430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cfc107ce0 msgr2=0x7f4cfc19df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4d00e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cfc107ce0 0x7f4cfc19df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4d00e00640 1 -- 172.21.15.3:0/1723422430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc1a5b40 con 0x7f4cfc109890 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4cfbe00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cfc107ce0 0x7f4cfc19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:29:13.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4d00e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc109890 0x7f4cfc1a54d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400eb10 tx=0x7f4cf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:13.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4cf9600640 1 -- 172.21.15.3:0/1723422430 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4004b20 con 0x7f4cfc109890 2024-03-21T08:29:13.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4cf9600640 1 -- 172.21.15.3:0/1723422430 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4cf4004cc0 con 0x7f4cfc109890 2024-03-21T08:29:13.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc1a5e30 con 0x7f4cfc109890 2024-03-21T08:29:13.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4cf9600640 1 -- 172.21.15.3:0/1723422430 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4010690 con 0x7f4cfc109890 2024-03-21T08:29:13.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.058+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc1a62a0 con 0x7f4cfc109890 2024-03-21T08:29:13.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.059+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc10f0d0 con 0x7f4cfc109890 2024-03-21T08:29:13.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.061+0000 7f4cf9600640 1 -- 172.21.15.3:0/1723422430 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f4cf4005020 con 0x7f4cfc109890 2024-03-21T08:29:13.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.063+0000 7f4cf9600640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4ce00a2fb0 0x7f4ce00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.063+0000 7f4cf9600640 1 -- 172.21.15.3:0/1723422430 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4cf40cffe0 con 0x7f4cfc109890 2024-03-21T08:29:13.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.063+0000 7f4cfbe00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4ce00a2fb0 0x7f4ce00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.064+0000 7f4cfbe00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4ce00a2fb0 0x7f4ce00a5470 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4cec02c240 tx=0x7f4cec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:13.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.067+0000 7f4cf9600640 1 -- 172.21.15.3:0/1723422430 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4cf409c910 con 0x7f4cfc109890 2024-03-21T08:29:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.267+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4cfc061ae0 con 0x7f4ce00a2fb0 2024-03-21T08:29:13.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.303+0000 7f4cf9600640 1 -- 172.21.15.3:0/1723422430 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340423 (secure 0 0 0) 0x7f4cfc061ae0 con 0x7f4ce00a2fb0 2024-03-21T08:29:13.310 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-03-21T08:29:13.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4ce00a2fb0 msgr2=0x7f4ce00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4ce00a2fb0 0x7f4ce00a5470 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4cec02c240 tx=0x7f4cec005fd0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc109890 msgr2=0x7f4cfc1a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc109890 0x7f4cfc1a54d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400eb10 tx=0x7f4cf400c2d0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 shutdown_connections 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cfc109890 0x7f4cfc1a54d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f4ce00a2fb0 0x7f4ce00a5470 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc108ed0 0x7f4cfc19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 --2- 172.21.15.3:0/1723422430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cfc107ce0 0x7f4cfc19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 >> 172.21.15.3:0/1723422430 conn(0x7f4cfc076ca0 msgr2=0x7f4cfc10c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.309+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 shutdown_connections 2024-03-21T08:29:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.310+0000 7f4d02e00640 1 -- 172.21.15.3:0/1723422430 wait complete. 2024-03-21T08:29:13.332 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-03-21T08:29:11.344042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":146,"num_read_kb":128,"num_write":309,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371712,"kb_used_data":14216,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570112,"statfs":{"total":1151940427776,"available":1151559794688,"internally_reserved":0,"allocated":14557184,"data_stored":7915900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2812,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":79,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003311"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.525299+0000","last_change":"2024-03-21T08:28:36.667851+0000","last_active":"2024-03-21T08:29:06.525299+0000","last_peered":"2024-03-21T08:29:06.525299+0000","last_clean":"2024-03-21T08:29:06.525299+0000","last_became_active":"2024-03-21T08:28:36.665713+0000","last_became_peered":"2024-03-21T08:28:36.665713+0000","last_unstale":"2024-03-21T08:29:06.525299+0000","last_undegraded":"2024-03-21T08:29:06.525299+0000","last_fullsized":"2024-03-21T08:29:06.525299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:02.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526623+0000","last_change":"2024-03-21T08:28:36.650986+0000","last_active":"2024-03-21T08:29:06.526623+0000","last_peered":"2024-03-21T08:29:06.526623+0000","last_clean":"2024-03-21T08:29:06.526623+0000","last_became_active":"2024-03-21T08:28:36.650633+0000","last_became_peered":"2024-03-21T08:28:36.650633+0000","last_unstale":"2024-03-21T08:29:06.526623+0000","last_undegraded":"2024-03-21T08:29:06.526623+0000","last_fullsized":"2024-03-21T08:29:06.526623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:40:20.801814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.524980+0000","last_change":"2024-03-21T08:28:36.662984+0000","last_active":"2024-03-21T08:29:06.524980+0000","last_peered":"2024-03-21T08:29:06.524980+0000","last_clean":"2024-03-21T08:29:06.524980+0000","last_became_active":"2024-03-21T08:28:36.662657+0000","last_became_peered":"2024-03-21T08:28:36.662657+0000","last_unstale":"2024-03-21T08:29:06.524980+0000","last_undegraded":"2024-03-21T08:29:06.524980+0000","last_fullsized":"2024-03-21T08:29:06.524980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:41.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.782243+0000","last_change":"2024-03-21T08:28:36.665958+0000","last_active":"2024-03-21T08:29:06.782243+0000","last_peered":"2024-03-21T08:29:06.782243+0000","last_clean":"2024-03-21T08:29:06.782243+0000","last_became_active":"2024-03-21T08:28:36.664309+0000","last_became_peered":"2024-03-21T08:28:36.664309+0000","last_unstale":"2024-03-21T08:29:06.782243+0000","last_undegraded":"2024-03-21T08:29:06.782243+0000","last_fullsized":"2024-03-21T08:29:06.782243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:16.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526265+0000","last_change":"2024-03-21T08:28:36.657767+0000","last_active":"2024-03-21T08:29:06.526265+0000","last_peered":"2024-03-21T08:29:06.526265+0000","last_clean":"2024-03-21T08:29:06.526265+0000","last_became_active":"2024-03-21T08:28:36.657350+0000","last_became_peered":"2024-03-21T08:28:36.657350+0000","last_unstale":"2024-03-21T08:29:06.526265+0000","last_undegraded":"2024-03-21T08:29:06.526265+0000","last_fullsized":"2024-03-21T08:29:06.526265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:47.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916971+0000","last_change":"2024-03-21T08:29:06.916971+0000","last_active":"2024-03-21T08:29:06.916971+0000","last_peered":"2024-03-21T08:29:06.916971+0000","last_clean":"2024-03-21T08:29:06.916971+0000","last_became_active":"2024-03-21T08:29:06.915997+0000","last_became_peered":"2024-03-21T08:29:06.915997+0000","last_unstale":"2024-03-21T08:29:06.916971+0000","last_undegraded":"2024-03-21T08:29:06.916971+0000","last_fullsized":"2024-03-21T08:29:06.916971+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:02.525190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.027701+0000","last_change":"2024-03-21T08:28:36.667787+0000","last_active":"2024-03-21T08:29:07.027701+0000","last_peered":"2024-03-21T08:29:07.027701+0000","last_clean":"2024-03-21T08:29:07.027701+0000","last_became_active":"2024-03-21T08:28:36.667451+0000","last_became_peered":"2024-03-21T08:28:36.667451+0000","last_unstale":"2024-03-21T08:29:07.027701+0000","last_undegraded":"2024-03-21T08:29:07.027701+0000","last_fullsized":"2024-03-21T08:29:07.027701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:01.874897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511084+0000","last_change":"2024-03-21T08:28:36.662445+0000","last_active":"2024-03-21T08:29:06.511084+0000","last_peered":"2024-03-21T08:29:06.511084+0000","last_clean":"2024-03-21T08:29:06.511084+0000","last_became_active":"2024-03-21T08:28:36.662113+0000","last_became_peered":"2024-03-21T08:28:36.662113+0000","last_unstale":"2024-03-21T08:29:06.511084+0000","last_undegraded":"2024-03-21T08:29:06.511084+0000","last_fullsized":"2024-03-21T08:29:06.511084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:21.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.520184+0000","last_change":"2024-03-21T08:28:36.666084+0000","last_active":"2024-03-21T08:29:06.520184+0000","last_peered":"2024-03-21T08:29:06.520184+0000","last_clean":"2024-03-21T08:29:06.520184+0000","last_became_active":"2024-03-21T08:28:36.665354+0000","last_became_peered":"2024-03-21T08:28:36.665354+0000","last_unstale":"2024-03-21T08:29:06.520184+0000","last_undegraded":"2024-03-21T08:29:06.520184+0000","last_fullsized":"2024-03-21T08:29:06.520184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:02:50.160765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.919231+0000","last_change":"2024-03-21T08:29:06.919231+0000","last_active":"2024-03-21T08:29:06.919231+0000","last_peered":"2024-03-21T08:29:06.919231+0000","last_clean":"2024-03-21T08:29:06.919231+0000","last_became_active":"2024-03-21T08:29:06.918199+0000","last_became_peered":"2024-03-21T08:29:06.918199+0000","last_unstale":"2024-03-21T08:29:06.919231+0000","last_undegraded":"2024-03-21T08:29:06.919231+0000","last_fullsized":"2024-03-21T08:29:06.919231+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:44.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029760+0000","last_change":"2024-03-21T08:28:36.656385+0000","last_active":"2024-03-21T08:29:07.029760+0000","last_peered":"2024-03-21T08:29:07.029760+0000","last_clean":"2024-03-21T08:29:07.029760+0000","last_became_active":"2024-03-21T08:28:36.656014+0000","last_became_peered":"2024-03-21T08:28:36.656014+0000","last_unstale":"2024-03-21T08:29:07.029760+0000","last_undegraded":"2024-03-21T08:29:07.029760+0000","last_fullsized":"2024-03-21T08:29:07.029760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:46.000495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.781932+0000","last_change":"2024-03-21T08:28:36.666254+0000","last_active":"2024-03-21T08:29:06.781932+0000","last_peered":"2024-03-21T08:29:06.781932+0000","last_clean":"2024-03-21T08:29:06.781932+0000","last_became_active":"2024-03-21T08:28:36.664698+0000","last_became_peered":"2024-03-21T08:28:36.664698+0000","last_unstale":"2024-03-21T08:29:06.781932+0000","last_undegraded":"2024-03-21T08:29:06.781932+0000","last_fullsized":"2024-03-21T08:29:06.781932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:03:08.358159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.919930+0000","last_change":"2024-03-21T08:29:06.919930+0000","last_active":"2024-03-21T08:29:06.919930+0000","last_peered":"2024-03-21T08:29:06.919930+0000","last_clean":"2024-03-21T08:29:06.919930+0000","last_became_active":"2024-03-21T08:29:06.917852+0000","last_became_peered":"2024-03-21T08:29:06.917852+0000","last_unstale":"2024-03-21T08:29:06.919930+0000","last_undegraded":"2024-03-21T08:29:06.919930+0000","last_fullsized":"2024-03-21T08:29:06.919930+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:05:01.185388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.523780+0000","last_change":"2024-03-21T08:28:36.655229+0000","last_active":"2024-03-21T08:29:06.523780+0000","last_peered":"2024-03-21T08:29:06.523780+0000","last_clean":"2024-03-21T08:29:06.523780+0000","last_became_active":"2024-03-21T08:28:36.653949+0000","last_became_peered":"2024-03-21T08:28:36.653949+0000","last_unstale":"2024-03-21T08:29:06.523780+0000","last_undegraded":"2024-03-21T08:29:06.523780+0000","last_fullsized":"2024-03-21T08:29:06.523780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:57:38.370897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.523277+0000","last_change":"2024-03-21T08:28:36.664894+0000","last_active":"2024-03-21T08:29:06.523277+0000","last_peered":"2024-03-21T08:29:06.523277+0000","last_clean":"2024-03-21T08:29:06.523277+0000","last_became_active":"2024-03-21T08:28:36.664523+0000","last_became_peered":"2024-03-21T08:28:36.664523+0000","last_unstale":"2024-03-21T08:29:06.523277+0000","last_undegraded":"2024-03-21T08:29:06.523277+0000","last_fullsized":"2024-03-21T08:29:06.523277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:52:54.147459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.517632+0000","last_change":"2024-03-21T08:28:36.662374+0000","last_active":"2024-03-21T08:29:06.517632+0000","last_peered":"2024-03-21T08:29:06.517632+0000","last_clean":"2024-03-21T08:29:06.517632+0000","last_became_active":"2024-03-21T08:28:36.662066+0000","last_became_peered":"2024-03-21T08:28:36.662066+0000","last_unstale":"2024-03-21T08:29:06.517632+0000","last_undegraded":"2024-03-21T08:29:06.517632+0000","last_fullsized":"2024-03-21T08:29:06.517632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:06.082925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.524487+0000","last_change":"2024-03-21T08:28:36.672172+0000","last_active":"2024-03-21T08:29:06.524487+0000","last_peered":"2024-03-21T08:29:06.524487+0000","last_clean":"2024-03-21T08:29:06.524487+0000","last_became_active":"2024-03-21T08:28:36.671657+0000","last_became_peered":"2024-03-21T08:28:36.671657+0000","last_unstale":"2024-03-21T08:29:06.524487+0000","last_undegraded":"2024-03-21T08:29:06.524487+0000","last_fullsized":"2024-03-21T08:29:06.524487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:56.892818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.918218+0000","last_change":"2024-03-21T08:28:36.666973+0000","last_active":"2024-03-21T08:29:06.918218+0000","last_peered":"2024-03-21T08:29:06.918218+0000","last_clean":"2024-03-21T08:29:06.918218+0000","last_became_active":"2024-03-21T08:28:36.663770+0000","last_became_peered":"2024-03-21T08:28:36.663770+0000","last_unstale":"2024-03-21T08:29:06.918218+0000","last_undegraded":"2024-03-21T08:29:06.918218+0000","last_fullsized":"2024-03-21T08:29:06.918218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:54:26.701009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514090+0000","last_change":"2024-03-21T08:28:36.668793+0000","last_active":"2024-03-21T08:29:06.514090+0000","last_peered":"2024-03-21T08:29:06.514090+0000","last_clean":"2024-03-21T08:29:06.514090+0000","last_became_active":"2024-03-21T08:28:36.668452+0000","last_became_peered":"2024-03-21T08:28:36.668452+0000","last_unstale":"2024-03-21T08:29:06.514090+0000","last_undegraded":"2024-03-21T08:29:06.514090+0000","last_fullsized":"2024-03-21T08:29:06.514090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:45.191849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513391+0000","last_change":"2024-03-21T08:28:40.687618+0000","last_active":"2024-03-21T08:29:06.513391+0000","last_peered":"2024-03-21T08:29:06.513391+0000","last_clean":"2024-03-21T08:29:06.513391+0000","last_became_active":"2024-03-21T08:28:40.687155+0000","last_became_peered":"2024-03-21T08:28:40.687155+0000","last_unstale":"2024-03-21T08:29:06.513391+0000","last_undegraded":"2024-03-21T08:29:06.513391+0000","last_fullsized":"2024-03-21T08:29:06.513391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:08.014790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.336746+0000","last_change":"2024-03-21T08:28:35.679236+0000","last_active":"2024-03-21T08:29:08.336746+0000","last_peered":"2024-03-21T08:29:08.336746+0000","last_clean":"2024-03-21T08:29:08.336746+0000","last_became_active":"2024-03-21T08:28:35.678838+0000","last_became_peered":"2024-03-21T08:28:35.678838+0000","last_unstale":"2024-03-21T08:29:08.336746+0000","last_undegraded":"2024-03-21T08:29:08.336746+0000","last_fullsized":"2024-03-21T08:29:08.336746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:15.310068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512989+0000","last_change":"2024-03-21T08:28:36.670325+0000","last_active":"2024-03-21T08:29:06.512989+0000","last_peered":"2024-03-21T08:29:06.512989+0000","last_clean":"2024-03-21T08:29:06.512989+0000","last_became_active":"2024-03-21T08:28:36.669783+0000","last_became_peered":"2024-03-21T08:28:36.669783+0000","last_unstale":"2024-03-21T08:29:06.512989+0000","last_undegraded":"2024-03-21T08:29:06.512989+0000","last_fullsized":"2024-03-21T08:29:06.512989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:24.166567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.779634+0000","last_change":"2024-03-21T08:28:40.689849+0000","last_active":"2024-03-21T08:29:06.779634+0000","last_peered":"2024-03-21T08:29:06.779634+0000","last_clean":"2024-03-21T08:29:06.779634+0000","last_became_active":"2024-03-21T08:28:40.689307+0000","last_became_peered":"2024-03-21T08:28:40.689307+0000","last_unstale":"2024-03-21T08:29:06.779634+0000","last_undegraded":"2024-03-21T08:29:06.779634+0000","last_fullsized":"2024-03-21T08:29:06.779634+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:34:05.660317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513463+0000","last_change":"2024-03-21T08:28:35.674685+0000","last_active":"2024-03-21T08:29:06.513463+0000","last_peered":"2024-03-21T08:29:06.513463+0000","last_clean":"2024-03-21T08:29:06.513463+0000","last_became_active":"2024-03-21T08:28:35.674312+0000","last_became_peered":"2024-03-21T08:28:35.674312+0000","last_unstale":"2024-03-21T08:29:06.513463+0000","last_undegraded":"2024-03-21T08:29:06.513463+0000","last_fullsized":"2024-03-21T08:29:06.513463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:27.644111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526724+0000","last_change":"2024-03-21T08:28:36.667744+0000","last_active":"2024-03-21T08:29:06.526724+0000","last_peered":"2024-03-21T08:29:06.526724+0000","last_clean":"2024-03-21T08:29:06.526724+0000","last_became_active":"2024-03-21T08:28:36.667279+0000","last_became_peered":"2024-03-21T08:28:36.667279+0000","last_unstale":"2024-03-21T08:29:06.526724+0000","last_undegraded":"2024-03-21T08:29:06.526724+0000","last_fullsized":"2024-03-21T08:29:06.526724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:14:24.974683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.780335+0000","last_change":"2024-03-21T08:28:40.688864+0000","last_active":"2024-03-21T08:29:06.780335+0000","last_peered":"2024-03-21T08:29:06.780335+0000","last_clean":"2024-03-21T08:29:06.780335+0000","last_became_active":"2024-03-21T08:28:40.688029+0000","last_became_peered":"2024-03-21T08:28:40.688029+0000","last_unstale":"2024-03-21T08:29:06.780335+0000","last_undegraded":"2024-03-21T08:29:06.780335+0000","last_fullsized":"2024-03-21T08:29:06.780335+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:39.235807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.333838+0000","last_change":"2024-03-21T08:28:35.687682+0000","last_active":"2024-03-21T08:29:08.333838+0000","last_peered":"2024-03-21T08:29:08.333838+0000","last_clean":"2024-03-21T08:29:08.333838+0000","last_became_active":"2024-03-21T08:28:35.686473+0000","last_became_peered":"2024-03-21T08:28:35.686473+0000","last_unstale":"2024-03-21T08:29:08.333838+0000","last_undegraded":"2024-03-21T08:29:08.333838+0000","last_fullsized":"2024-03-21T08:29:08.333838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:30.618673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519265+0000","last_change":"2024-03-21T08:28:36.667244+0000","last_active":"2024-03-21T08:29:06.519265+0000","last_peered":"2024-03-21T08:29:06.519265+0000","last_clean":"2024-03-21T08:29:06.519265+0000","last_became_active":"2024-03-21T08:28:36.666829+0000","last_became_peered":"2024-03-21T08:28:36.666829+0000","last_unstale":"2024-03-21T08:29:06.519265+0000","last_undegraded":"2024-03-21T08:29:06.519265+0000","last_fullsized":"2024-03-21T08:29:06.519265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:12:56.842084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.779575+0000","last_change":"2024-03-21T08:28:40.692621+0000","last_active":"2024-03-21T08:29:06.779575+0000","last_peered":"2024-03-21T08:29:06.779575+0000","last_clean":"2024-03-21T08:29:06.779575+0000","last_became_active":"2024-03-21T08:28:40.691984+0000","last_became_peered":"2024-03-21T08:28:40.691984+0000","last_unstale":"2024-03-21T08:29:06.779575+0000","last_undegraded":"2024-03-21T08:29:06.779575+0000","last_fullsized":"2024-03-21T08:29:06.779575+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:29.330702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.529238+0000","last_change":"2024-03-21T08:28:35.676421+0000","last_active":"2024-03-21T08:29:06.529238+0000","last_peered":"2024-03-21T08:29:06.529238+0000","last_clean":"2024-03-21T08:29:06.529238+0000","last_became_active":"2024-03-21T08:28:35.674957+0000","last_became_peered":"2024-03-21T08:28:35.674957+0000","last_unstale":"2024-03-21T08:29:06.529238+0000","last_undegraded":"2024-03-21T08:29:06.529238+0000","last_fullsized":"2024-03-21T08:29:06.529238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:01:38.397757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877668+0000","last_change":"2024-03-21T08:28:36.664662+0000","last_active":"2024-03-21T08:29:06.877668+0000","last_peered":"2024-03-21T08:29:06.877668+0000","last_clean":"2024-03-21T08:29:06.877668+0000","last_became_active":"2024-03-21T08:28:36.664324+0000","last_became_peered":"2024-03-21T08:28:36.664324+0000","last_unstale":"2024-03-21T08:29:06.877668+0000","last_undegraded":"2024-03-21T08:29:06.877668+0000","last_fullsized":"2024-03-21T08:29:06.877668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:57.325869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513597+0000","last_change":"2024-03-21T08:28:40.676297+0000","last_active":"2024-03-21T08:29:06.513597+0000","last_peered":"2024-03-21T08:29:06.513597+0000","last_clean":"2024-03-21T08:29:06.513597+0000","last_became_active":"2024-03-21T08:28:40.675337+0000","last_became_peered":"2024-03-21T08:28:40.675337+0000","last_unstale":"2024-03-21T08:29:06.513597+0000","last_undegraded":"2024-03-21T08:29:06.513597+0000","last_fullsized":"2024-03-21T08:29:06.513597+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:05.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.326702+0000","last_change":"2024-03-21T08:28:35.687651+0000","last_active":"2024-03-21T08:29:07.326702+0000","last_peered":"2024-03-21T08:29:07.326702+0000","last_clean":"2024-03-21T08:29:07.326702+0000","last_became_active":"2024-03-21T08:28:35.686284+0000","last_became_peered":"2024-03-21T08:28:35.686284+0000","last_unstale":"2024-03-21T08:29:07.326702+0000","last_undegraded":"2024-03-21T08:29:07.326702+0000","last_fullsized":"2024-03-21T08:29:07.326702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:56.121798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.033436+0000","last_change":"2024-03-21T08:29:07.033436+0000","last_active":"2024-03-21T08:29:07.033436+0000","last_peered":"2024-03-21T08:29:07.033436+0000","last_clean":"2024-03-21T08:29:07.033436+0000","last_became_active":"2024-03-21T08:29:07.032012+0000","last_became_peered":"2024-03-21T08:29:07.032012+0000","last_unstale":"2024-03-21T08:29:07.033436+0000","last_undegraded":"2024-03-21T08:29:07.033436+0000","last_fullsized":"2024-03-21T08:29:07.033436+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:44:47.009078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.882449+0000","last_change":"2024-03-21T08:29:06.882449+0000","last_active":"2024-03-21T08:29:06.882449+0000","last_peered":"2024-03-21T08:29:06.882449+0000","last_clean":"2024-03-21T08:29:06.882449+0000","last_became_active":"2024-03-21T08:29:06.880682+0000","last_became_peered":"2024-03-21T08:29:06.880682+0000","last_unstale":"2024-03-21T08:29:06.882449+0000","last_undegraded":"2024-03-21T08:29:06.882449+0000","last_fullsized":"2024-03-21T08:29:06.882449+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:09.464388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334899+0000","last_change":"2024-03-21T08:28:35.679252+0000","last_active":"2024-03-21T08:29:08.334899+0000","last_peered":"2024-03-21T08:29:08.334899+0000","last_clean":"2024-03-21T08:29:08.334899+0000","last_became_active":"2024-03-21T08:28:35.678848+0000","last_became_peered":"2024-03-21T08:28:35.678848+0000","last_unstale":"2024-03-21T08:29:08.334899+0000","last_undegraded":"2024-03-21T08:29:08.334899+0000","last_fullsized":"2024-03-21T08:29:08.334899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:19:11.261660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878657+0000","last_change":"2024-03-21T08:28:36.670176+0000","last_active":"2024-03-21T08:29:06.878657+0000","last_peered":"2024-03-21T08:29:06.878657+0000","last_clean":"2024-03-21T08:29:06.878657+0000","last_became_active":"2024-03-21T08:28:36.669477+0000","last_became_peered":"2024-03-21T08:28:36.669477+0000","last_unstale":"2024-03-21T08:29:06.878657+0000","last_undegraded":"2024-03-21T08:29:06.878657+0000","last_fullsized":"2024-03-21T08:29:06.878657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:10:21.289484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.784996+0000","last_change":"2024-03-21T08:29:06.784996+0000","last_active":"2024-03-21T08:29:06.784996+0000","last_peered":"2024-03-21T08:29:06.784996+0000","last_clean":"2024-03-21T08:29:06.784996+0000","last_became_active":"2024-03-21T08:29:06.783403+0000","last_became_peered":"2024-03-21T08:29:06.783403+0000","last_unstale":"2024-03-21T08:29:06.784996+0000","last_undegraded":"2024-03-21T08:29:06.784996+0000","last_fullsized":"2024-03-21T08:29:06.784996+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:18:59.912956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334707+0000","last_change":"2024-03-21T08:28:35.667502+0000","last_active":"2024-03-21T08:29:08.334707+0000","last_peered":"2024-03-21T08:29:08.334707+0000","last_clean":"2024-03-21T08:29:08.334707+0000","last_became_active":"2024-03-21T08:28:35.666667+0000","last_became_peered":"2024-03-21T08:28:35.666667+0000","last_unstale":"2024-03-21T08:29:08.334707+0000","last_undegraded":"2024-03-21T08:29:08.334707+0000","last_fullsized":"2024-03-21T08:29:08.334707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:42.196361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511542+0000","last_change":"2024-03-21T08:28:36.670855+0000","last_active":"2024-03-21T08:29:06.511542+0000","last_peered":"2024-03-21T08:29:06.511542+0000","last_clean":"2024-03-21T08:29:06.511542+0000","last_became_active":"2024-03-21T08:28:36.666012+0000","last_became_peered":"2024-03-21T08:28:36.666012+0000","last_unstale":"2024-03-21T08:29:06.511542+0000","last_undegraded":"2024-03-21T08:29:06.511542+0000","last_fullsized":"2024-03-21T08:29:06.511542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:19.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526807+0000","last_change":"2024-03-21T08:28:40.700037+0000","last_active":"2024-03-21T08:29:06.526807+0000","last_peered":"2024-03-21T08:29:06.526807+0000","last_clean":"2024-03-21T08:29:06.526807+0000","last_became_active":"2024-03-21T08:28:40.698127+0000","last_became_peered":"2024-03-21T08:28:40.698127+0000","last_unstale":"2024-03-21T08:29:06.526807+0000","last_undegraded":"2024-03-21T08:29:06.526807+0000","last_fullsized":"2024-03-21T08:29:06.526807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:48:57.135690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877020+0000","last_change":"2024-03-21T08:28:35.677071+0000","last_active":"2024-03-21T08:29:06.877020+0000","last_peered":"2024-03-21T08:29:06.877020+0000","last_clean":"2024-03-21T08:29:06.877020+0000","last_became_active":"2024-03-21T08:28:35.676652+0000","last_became_peered":"2024-03-21T08:28:35.676652+0000","last_unstale":"2024-03-21T08:29:06.877020+0000","last_undegraded":"2024-03-21T08:29:06.877020+0000","last_fullsized":"2024-03-21T08:29:06.877020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:24:54.097952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.915114+0000","last_change":"2024-03-21T08:28:36.677981+0000","last_active":"2024-03-21T08:29:06.915114+0000","last_peered":"2024-03-21T08:29:06.915114+0000","last_clean":"2024-03-21T08:29:06.915114+0000","last_became_active":"2024-03-21T08:28:36.677512+0000","last_became_peered":"2024-03-21T08:28:36.677512+0000","last_unstale":"2024-03-21T08:29:06.915114+0000","last_undegraded":"2024-03-21T08:29:06.915114+0000","last_fullsized":"2024-03-21T08:29:06.915114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:50.742928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512688+0000","last_change":"2024-03-21T08:28:40.692672+0000","last_active":"2024-03-21T08:29:06.512688+0000","last_peered":"2024-03-21T08:29:06.512688+0000","last_clean":"2024-03-21T08:29:06.512688+0000","last_became_active":"2024-03-21T08:28:40.688940+0000","last_became_peered":"2024-03-21T08:28:40.688940+0000","last_unstale":"2024-03-21T08:29:06.512688+0000","last_undegraded":"2024-03-21T08:29:06.512688+0000","last_fullsized":"2024-03-21T08:29:06.512688+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:39.569709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.325701+0000","last_change":"2024-03-21T08:28:35.681970+0000","last_active":"2024-03-21T08:29:07.325701+0000","last_peered":"2024-03-21T08:29:07.325701+0000","last_clean":"2024-03-21T08:29:07.325701+0000","last_became_active":"2024-03-21T08:28:35.681252+0000","last_became_peered":"2024-03-21T08:28:35.681252+0000","last_unstale":"2024-03-21T08:29:07.325701+0000","last_undegraded":"2024-03-21T08:29:07.325701+0000","last_fullsized":"2024-03-21T08:29:07.325701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:46.345359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519859+0000","last_change":"2024-03-21T08:28:36.675419+0000","last_active":"2024-03-21T08:29:06.519859+0000","last_peered":"2024-03-21T08:29:06.519859+0000","last_clean":"2024-03-21T08:29:06.519859+0000","last_became_active":"2024-03-21T08:28:36.674039+0000","last_became_peered":"2024-03-21T08:28:36.674039+0000","last_unstale":"2024-03-21T08:29:06.519859+0000","last_undegraded":"2024-03-21T08:29:06.519859+0000","last_fullsized":"2024-03-21T08:29:06.519859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:09.667634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.915099+0000","last_change":"2024-03-21T08:28:40.690076+0000","last_active":"2024-03-21T08:29:06.915099+0000","last_peered":"2024-03-21T08:29:06.915099+0000","last_clean":"2024-03-21T08:29:06.915099+0000","last_became_active":"2024-03-21T08:28:40.683817+0000","last_became_peered":"2024-03-21T08:28:40.683817+0000","last_unstale":"2024-03-21T08:29:06.915099+0000","last_undegraded":"2024-03-21T08:29:06.915099+0000","last_fullsized":"2024-03-21T08:29:06.915099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:33:00.948879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.321229+0000","last_change":"2024-03-21T08:28:35.690693+0000","last_active":"2024-03-21T08:29:07.321229+0000","last_peered":"2024-03-21T08:29:07.321229+0000","last_clean":"2024-03-21T08:29:07.321229+0000","last_became_active":"2024-03-21T08:28:35.690250+0000","last_became_peered":"2024-03-21T08:28:35.690250+0000","last_unstale":"2024-03-21T08:29:07.321229+0000","last_undegraded":"2024-03-21T08:29:07.321229+0000","last_fullsized":"2024-03-21T08:29:07.321229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:44:21.307772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526656+0000","last_change":"2024-03-21T08:28:36.661872+0000","last_active":"2024-03-21T08:29:06.526656+0000","last_peered":"2024-03-21T08:29:06.526656+0000","last_clean":"2024-03-21T08:29:06.526656+0000","last_became_active":"2024-03-21T08:28:36.661483+0000","last_became_peered":"2024-03-21T08:28:36.661483+0000","last_unstale":"2024-03-21T08:29:06.526656+0000","last_undegraded":"2024-03-21T08:29:06.526656+0000","last_fullsized":"2024-03-21T08:29:06.526656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:03:35.630141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.780990+0000","last_change":"2024-03-21T08:28:40.682753+0000","last_active":"2024-03-21T08:29:06.780990+0000","last_peered":"2024-03-21T08:29:06.780990+0000","last_clean":"2024-03-21T08:29:06.780990+0000","last_became_active":"2024-03-21T08:28:40.682278+0000","last_became_peered":"2024-03-21T08:28:40.682278+0000","last_unstale":"2024-03-21T08:29:06.780990+0000","last_undegraded":"2024-03-21T08:29:06.780990+0000","last_fullsized":"2024-03-21T08:29:06.780990+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:20.280442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878296+0000","last_change":"2024-03-21T08:28:35.672524+0000","last_active":"2024-03-21T08:29:06.878296+0000","last_peered":"2024-03-21T08:29:06.878296+0000","last_clean":"2024-03-21T08:29:06.878296+0000","last_became_active":"2024-03-21T08:28:35.672051+0000","last_became_peered":"2024-03-21T08:28:35.672051+0000","last_unstale":"2024-03-21T08:29:06.878296+0000","last_undegraded":"2024-03-21T08:29:06.878296+0000","last_fullsized":"2024-03-21T08:29:06.878296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:53.336320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.532172+0000","last_change":"2024-03-21T08:29:06.532172+0000","last_active":"2024-03-21T08:29:06.532172+0000","last_peered":"2024-03-21T08:29:06.532172+0000","last_clean":"2024-03-21T08:29:06.532172+0000","last_became_active":"2024-03-21T08:29:06.531439+0000","last_became_peered":"2024-03-21T08:29:06.531439+0000","last_unstale":"2024-03-21T08:29:06.532172+0000","last_undegraded":"2024-03-21T08:29:06.532172+0000","last_fullsized":"2024-03-21T08:29:06.532172+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:27.222359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.915105+0000","last_change":"2024-03-21T08:28:40.682294+0000","last_active":"2024-03-21T08:29:06.915105+0000","last_peered":"2024-03-21T08:29:06.915105+0000","last_clean":"2024-03-21T08:29:06.915105+0000","last_became_active":"2024-03-21T08:28:40.681855+0000","last_became_peered":"2024-03-21T08:28:40.681855+0000","last_unstale":"2024-03-21T08:29:06.915105+0000","last_undegraded":"2024-03-21T08:29:06.915105+0000","last_fullsized":"2024-03-21T08:29:06.915105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:49.756322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'12","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.319825+0000","last_change":"2024-03-21T08:28:35.682947+0000","last_active":"2024-03-21T08:29:07.319825+0000","last_peered":"2024-03-21T08:29:07.319825+0000","last_clean":"2024-03-21T08:29:07.319825+0000","last_became_active":"2024-03-21T08:28:35.682395+0000","last_became_peered":"2024-03-21T08:28:35.682395+0000","last_unstale":"2024-03-21T08:29:07.319825+0000","last_undegraded":"2024-03-21T08:29:07.319825+0000","last_fullsized":"2024-03-21T08:29:07.319825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:07:48.072988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.527327+0000","last_change":"2024-03-21T08:28:36.651140+0000","last_active":"2024-03-21T08:29:06.527327+0000","last_peered":"2024-03-21T08:29:06.527327+0000","last_clean":"2024-03-21T08:29:06.527327+0000","last_became_active":"2024-03-21T08:28:36.650551+0000","last_became_peered":"2024-03-21T08:28:36.650551+0000","last_unstale":"2024-03-21T08:29:06.527327+0000","last_undegraded":"2024-03-21T08:29:06.527327+0000","last_fullsized":"2024-03-21T08:29:06.527327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:53.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878806+0000","last_change":"2024-03-21T08:28:40.691814+0000","last_active":"2024-03-21T08:29:06.878806+0000","last_peered":"2024-03-21T08:29:06.878806+0000","last_clean":"2024-03-21T08:29:06.878806+0000","last_became_active":"2024-03-21T08:28:40.691372+0000","last_became_peered":"2024-03-21T08:28:40.691372+0000","last_unstale":"2024-03-21T08:29:06.878806+0000","last_undegraded":"2024-03-21T08:29:06.878806+0000","last_fullsized":"2024-03-21T08:29:06.878806+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:21:42.612037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.332492+0000","last_change":"2024-03-21T08:28:35.687178+0000","last_active":"2024-03-21T08:29:08.332492+0000","last_peered":"2024-03-21T08:29:08.332492+0000","last_clean":"2024-03-21T08:29:08.332492+0000","last_became_active":"2024-03-21T08:28:35.686771+0000","last_became_peered":"2024-03-21T08:28:35.686771+0000","last_unstale":"2024-03-21T08:29:08.332492+0000","last_undegraded":"2024-03-21T08:29:08.332492+0000","last_fullsized":"2024-03-21T08:29:08.332492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:54.665866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518258+0000","last_change":"2024-03-21T08:28:36.662952+0000","last_active":"2024-03-21T08:29:06.518258+0000","last_peered":"2024-03-21T08:29:06.518258+0000","last_clean":"2024-03-21T08:29:06.518258+0000","last_became_active":"2024-03-21T08:28:36.662468+0000","last_became_peered":"2024-03-21T08:28:36.662468+0000","last_unstale":"2024-03-21T08:29:06.518258+0000","last_undegraded":"2024-03-21T08:29:06.518258+0000","last_fullsized":"2024-03-21T08:29:06.518258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:40.486998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.521486+0000","last_change":"2024-03-21T08:28:40.693999+0000","last_active":"2024-03-21T08:29:06.521486+0000","last_peered":"2024-03-21T08:29:06.521486+0000","last_clean":"2024-03-21T08:29:06.521486+0000","last_became_active":"2024-03-21T08:28:40.693507+0000","last_became_peered":"2024-03-21T08:28:40.693507+0000","last_unstale":"2024-03-21T08:29:06.521486+0000","last_undegraded":"2024-03-21T08:29:06.521486+0000","last_fullsized":"2024-03-21T08:29:06.521486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:14.159379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.325489+0000","last_change":"2024-03-21T08:28:35.677536+0000","last_active":"2024-03-21T08:29:07.325489+0000","last_peered":"2024-03-21T08:29:07.325489+0000","last_clean":"2024-03-21T08:29:07.325489+0000","last_became_active":"2024-03-21T08:28:35.677100+0000","last_became_peered":"2024-03-21T08:28:35.677100+0000","last_unstale":"2024-03-21T08:29:07.325489+0000","last_undegraded":"2024-03-21T08:29:07.325489+0000","last_fullsized":"2024-03-21T08:29:07.325489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:52:47.275869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.028502+0000","last_change":"2024-03-21T08:28:36.667190+0000","last_active":"2024-03-21T08:29:07.028502+0000","last_peered":"2024-03-21T08:29:07.028502+0000","last_clean":"2024-03-21T08:29:07.028502+0000","last_became_active":"2024-03-21T08:28:36.665336+0000","last_became_peered":"2024-03-21T08:28:36.665336+0000","last_unstale":"2024-03-21T08:29:07.028502+0000","last_undegraded":"2024-03-21T08:29:07.028502+0000","last_fullsized":"2024-03-21T08:29:07.028502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:21.888426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916441+0000","last_change":"2024-03-21T08:28:40.696582+0000","last_active":"2024-03-21T08:29:06.916441+0000","last_peered":"2024-03-21T08:29:06.916441+0000","last_clean":"2024-03-21T08:29:06.916441+0000","last_became_active":"2024-03-21T08:28:40.689583+0000","last_became_peered":"2024-03-21T08:28:40.689583+0000","last_unstale":"2024-03-21T08:29:06.916441+0000","last_undegraded":"2024-03-21T08:29:06.916441+0000","last_fullsized":"2024-03-21T08:29:06.916441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:57.329533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:09.342354+0000","last_change":"2024-03-21T08:28:35.675560+0000","last_active":"2024-03-21T08:29:09.342354+0000","last_peered":"2024-03-21T08:29:09.342354+0000","last_clean":"2024-03-21T08:29:09.342354+0000","last_became_active":"2024-03-21T08:28:35.673496+0000","last_became_peered":"2024-03-21T08:28:35.673496+0000","last_unstale":"2024-03-21T08:29:09.342354+0000","last_undegraded":"2024-03-21T08:29:09.342354+0000","last_fullsized":"2024-03-21T08:29:09.342354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:41:27.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.027450+0000","last_change":"2024-03-21T08:28:36.660869+0000","last_active":"2024-03-21T08:29:07.027450+0000","last_peered":"2024-03-21T08:29:07.027450+0000","last_clean":"2024-03-21T08:29:07.027450+0000","last_became_active":"2024-03-21T08:28:36.659291+0000","last_became_peered":"2024-03-21T08:28:36.659291+0000","last_unstale":"2024-03-21T08:29:07.027450+0000","last_undegraded":"2024-03-21T08:29:07.027450+0000","last_fullsized":"2024-03-21T08:29:07.027450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:06.534214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519649+0000","last_change":"2024-03-21T08:28:40.701351+0000","last_active":"2024-03-21T08:29:06.519649+0000","last_peered":"2024-03-21T08:29:06.519649+0000","last_clean":"2024-03-21T08:29:06.519649+0000","last_became_active":"2024-03-21T08:28:40.700989+0000","last_became_peered":"2024-03-21T08:28:40.700989+0000","last_unstale":"2024-03-21T08:29:06.519649+0000","last_undegraded":"2024-03-21T08:29:06.519649+0000","last_fullsized":"2024-03-21T08:29:06.519649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:35.697013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.335679+0000","last_change":"2024-03-21T08:28:35.685024+0000","last_active":"2024-03-21T08:29:08.335679+0000","last_peered":"2024-03-21T08:29:08.335679+0000","last_clean":"2024-03-21T08:29:08.335679+0000","last_became_active":"2024-03-21T08:28:35.684597+0000","last_became_peered":"2024-03-21T08:28:35.684597+0000","last_unstale":"2024-03-21T08:29:08.335679+0000","last_undegraded":"2024-03-21T08:29:08.335679+0000","last_fullsized":"2024-03-21T08:29:08.335679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:09:46.262782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.876971+0000","last_change":"2024-03-21T08:28:36.664688+0000","last_active":"2024-03-21T08:29:06.876971+0000","last_peered":"2024-03-21T08:29:06.876971+0000","last_clean":"2024-03-21T08:29:06.876971+0000","last_became_active":"2024-03-21T08:28:36.663990+0000","last_became_peered":"2024-03-21T08:28:36.663990+0000","last_unstale":"2024-03-21T08:29:06.876971+0000","last_undegraded":"2024-03-21T08:29:06.876971+0000","last_fullsized":"2024-03-21T08:29:06.876971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:39.656290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.876815+0000","last_change":"2024-03-21T08:28:40.695269+0000","last_active":"2024-03-21T08:29:06.876815+0000","last_peered":"2024-03-21T08:29:06.876815+0000","last_clean":"2024-03-21T08:29:06.876815+0000","last_became_active":"2024-03-21T08:28:40.694820+0000","last_became_peered":"2024-03-21T08:28:40.694820+0000","last_unstale":"2024-03-21T08:29:06.876815+0000","last_undegraded":"2024-03-21T08:29:06.876815+0000","last_fullsized":"2024-03-21T08:29:06.876815+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:47:25.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526439+0000","last_change":"2024-03-21T08:28:35.677377+0000","last_active":"2024-03-21T08:29:06.526439+0000","last_peered":"2024-03-21T08:29:06.526439+0000","last_clean":"2024-03-21T08:29:06.526439+0000","last_became_active":"2024-03-21T08:28:35.673250+0000","last_became_peered":"2024-03-21T08:28:35.673250+0000","last_unstale":"2024-03-21T08:29:06.526439+0000","last_undegraded":"2024-03-21T08:29:06.526439+0000","last_fullsized":"2024-03-21T08:29:06.526439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:28.616353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.522542+0000","last_change":"2024-03-21T08:28:36.667605+0000","last_active":"2024-03-21T08:29:06.522542+0000","last_peered":"2024-03-21T08:29:06.522542+0000","last_clean":"2024-03-21T08:29:06.522542+0000","last_became_active":"2024-03-21T08:28:36.667300+0000","last_became_peered":"2024-03-21T08:28:36.667300+0000","last_unstale":"2024-03-21T08:29:06.522542+0000","last_undegraded":"2024-03-21T08:29:06.522542+0000","last_fullsized":"2024-03-21T08:29:06.522542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:49.268935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.529247+0000","last_change":"2024-03-21T08:28:40.884964+0000","last_active":"2024-03-21T08:29:06.529247+0000","last_peered":"2024-03-21T08:29:06.529247+0000","last_clean":"2024-03-21T08:29:06.529247+0000","last_became_active":"2024-03-21T08:28:40.883445+0000","last_became_peered":"2024-03-21T08:28:40.883445+0000","last_unstale":"2024-03-21T08:29:06.529247+0000","last_undegraded":"2024-03-21T08:29:06.529247+0000","last_fullsized":"2024-03-21T08:29:06.529247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:36:54.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916126+0000","last_change":"2024-03-21T08:28:35.682581+0000","last_active":"2024-03-21T08:29:06.916126+0000","last_peered":"2024-03-21T08:29:06.916126+0000","last_clean":"2024-03-21T08:29:06.916126+0000","last_became_active":"2024-03-21T08:28:35.682150+0000","last_became_peered":"2024-03-21T08:28:35.682150+0000","last_unstale":"2024-03-21T08:29:06.916126+0000","last_undegraded":"2024-03-21T08:29:06.916126+0000","last_fullsized":"2024-03-21T08:29:06.916126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:08.729710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511564+0000","last_change":"2024-03-21T08:28:36.666632+0000","last_active":"2024-03-21T08:29:06.511564+0000","last_peered":"2024-03-21T08:29:06.511564+0000","last_clean":"2024-03-21T08:29:06.511564+0000","last_became_active":"2024-03-21T08:28:36.666305+0000","last_became_peered":"2024-03-21T08:28:36.666305+0000","last_unstale":"2024-03-21T08:29:06.511564+0000","last_undegraded":"2024-03-21T08:29:06.511564+0000","last_fullsized":"2024-03-21T08:29:06.511564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:15.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519052+0000","last_change":"2024-03-21T08:28:40.700382+0000","last_active":"2024-03-21T08:29:06.519052+0000","last_peered":"2024-03-21T08:29:06.519052+0000","last_clean":"2024-03-21T08:29:06.519052+0000","last_became_active":"2024-03-21T08:28:40.699762+0000","last_became_peered":"2024-03-21T08:28:40.699762+0000","last_unstale":"2024-03-21T08:29:06.519052+0000","last_undegraded":"2024-03-21T08:29:06.519052+0000","last_fullsized":"2024-03-21T08:29:06.519052+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:53.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029010+0000","last_change":"2024-03-21T08:28:35.680310+0000","last_active":"2024-03-21T08:29:07.029010+0000","last_peered":"2024-03-21T08:29:07.029010+0000","last_clean":"2024-03-21T08:29:07.029010+0000","last_became_active":"2024-03-21T08:28:35.678634+0000","last_became_peered":"2024-03-21T08:28:35.678634+0000","last_unstale":"2024-03-21T08:29:07.029010+0000","last_undegraded":"2024-03-21T08:29:07.029010+0000","last_fullsized":"2024-03-21T08:29:07.029010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:56:42.872521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511087+0000","last_change":"2024-03-21T08:28:36.669527+0000","last_active":"2024-03-21T08:29:06.511087+0000","last_peered":"2024-03-21T08:29:06.511087+0000","last_clean":"2024-03-21T08:29:06.511087+0000","last_became_active":"2024-03-21T08:28:36.669196+0000","last_became_peered":"2024-03-21T08:28:36.669196+0000","last_unstale":"2024-03-21T08:29:06.511087+0000","last_undegraded":"2024-03-21T08:29:06.511087+0000","last_fullsized":"2024-03-21T08:29:06.511087+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:07:34.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519996+0000","last_change":"2024-03-21T08:28:40.882949+0000","last_active":"2024-03-21T08:29:06.519996+0000","last_peered":"2024-03-21T08:29:06.519996+0000","last_clean":"2024-03-21T08:29:06.519996+0000","last_became_active":"2024-03-21T08:28:40.882432+0000","last_became_peered":"2024-03-21T08:28:40.882432+0000","last_unstale":"2024-03-21T08:29:06.519996+0000","last_undegraded":"2024-03-21T08:29:06.519996+0000","last_fullsized":"2024-03-21T08:29:06.519996+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:51.171365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877181+0000","last_change":"2024-03-21T08:28:36.665164+0000","last_active":"2024-03-21T08:29:06.877181+0000","last_peered":"2024-03-21T08:29:06.877181+0000","last_clean":"2024-03-21T08:29:06.877181+0000","last_became_active":"2024-03-21T08:28:36.664794+0000","last_became_peered":"2024-03-21T08:28:36.664794+0000","last_unstale":"2024-03-21T08:29:06.877181+0000","last_undegraded":"2024-03-21T08:29:06.877181+0000","last_fullsized":"2024-03-21T08:29:06.877181+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:25:39.355633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334636+0000","last_change":"2024-03-21T08:28:35.671754+0000","last_active":"2024-03-21T08:29:08.334636+0000","last_peered":"2024-03-21T08:29:08.334636+0000","last_clean":"2024-03-21T08:29:08.334636+0000","last_became_active":"2024-03-21T08:28:35.671363+0000","last_became_peered":"2024-03-21T08:28:35.671363+0000","last_unstale":"2024-03-21T08:29:08.334636+0000","last_undegraded":"2024-03-21T08:29:08.334636+0000","last_fullsized":"2024-03-21T08:29:08.334636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:22:11.147325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029587+0000","last_change":"2024-03-21T08:28:40.689038+0000","last_active":"2024-03-21T08:29:07.029587+0000","last_peered":"2024-03-21T08:29:07.029587+0000","last_clean":"2024-03-21T08:29:07.029587+0000","last_became_active":"2024-03-21T08:28:40.688538+0000","last_became_peered":"2024-03-21T08:28:40.688538+0000","last_unstale":"2024-03-21T08:29:07.029587+0000","last_undegraded":"2024-03-21T08:29:07.029587+0000","last_fullsized":"2024-03-21T08:29:07.029587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:14.833594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511367+0000","last_change":"2024-03-21T08:28:36.667725+0000","last_active":"2024-03-21T08:29:06.511367+0000","last_peered":"2024-03-21T08:29:06.511367+0000","last_clean":"2024-03-21T08:29:06.511367+0000","last_became_active":"2024-03-21T08:28:36.667303+0000","last_became_peered":"2024-03-21T08:28:36.667303+0000","last_unstale":"2024-03-21T08:29:06.511367+0000","last_undegraded":"2024-03-21T08:29:06.511367+0000","last_fullsized":"2024-03-21T08:29:06.511367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:32:18.990943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:09.339335+0000","last_change":"2024-03-21T08:28:35.673497+0000","last_active":"2024-03-21T08:29:09.339335+0000","last_peered":"2024-03-21T08:29:09.339335+0000","last_clean":"2024-03-21T08:29:09.339335+0000","last_became_active":"2024-03-21T08:28:35.673045+0000","last_became_peered":"2024-03-21T08:28:35.673045+0000","last_unstale":"2024-03-21T08:29:09.339335+0000","last_undegraded":"2024-03-21T08:29:09.339335+0000","last_fullsized":"2024-03-21T08:29:09.339335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:23.972672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.027114+0000","last_change":"2024-03-21T08:26:58.139094+0000","last_active":"2024-03-21T08:29:07.027114+0000","last_peered":"2024-03-21T08:29:07.027114+0000","last_clean":"2024-03-21T08:29:07.027114+0000","last_became_active":"2024-03-21T08:26:58.137648+0000","last_became_peered":"2024-03-21T08:26:58.137648+0000","last_unstale":"2024-03-21T08:29:07.027114+0000","last_undegraded":"2024-03-21T08:29:07.027114+0000","last_fullsized":"2024-03-21T08:29:07.027114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_clean_scrub_stamp":"2024-03-21T08:22:17.977314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:57.950111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878595+0000","last_change":"2024-03-21T08:28:40.887865+0000","last_active":"2024-03-21T08:29:06.878595+0000","last_peered":"2024-03-21T08:29:06.878595+0000","last_clean":"2024-03-21T08:29:06.878595+0000","last_became_active":"2024-03-21T08:28:40.885538+0000","last_became_peered":"2024-03-21T08:28:40.885538+0000","last_unstale":"2024-03-21T08:29:06.878595+0000","last_undegraded":"2024-03-21T08:29:06.878595+0000","last_fullsized":"2024-03-21T08:29:06.878595+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:26.532005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518547+0000","last_change":"2024-03-21T08:28:36.668600+0000","last_active":"2024-03-21T08:29:06.518547+0000","last_peered":"2024-03-21T08:29:06.518547+0000","last_clean":"2024-03-21T08:29:06.518547+0000","last_became_active":"2024-03-21T08:28:36.668188+0000","last_became_peered":"2024-03-21T08:28:36.668188+0000","last_unstale":"2024-03-21T08:29:06.518547+0000","last_undegraded":"2024-03-21T08:29:06.518547+0000","last_fullsized":"2024-03-21T08:29:06.518547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:13.536003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334248+0000","last_change":"2024-03-21T08:28:35.674594+0000","last_active":"2024-03-21T08:29:08.334248+0000","last_peered":"2024-03-21T08:29:08.334248+0000","last_clean":"2024-03-21T08:29:08.334248+0000","last_became_active":"2024-03-21T08:28:35.674257+0000","last_became_peered":"2024-03-21T08:28:35.674257+0000","last_unstale":"2024-03-21T08:29:08.334248+0000","last_undegraded":"2024-03-21T08:29:08.334248+0000","last_fullsized":"2024-03-21T08:29:08.334248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:14:51.585631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.032622+0000","last_change":"2024-03-21T08:29:07.032622+0000","last_active":"2024-03-21T08:29:07.032622+0000","last_peered":"2024-03-21T08:29:07.032622+0000","last_clean":"2024-03-21T08:29:07.032622+0000","last_became_active":"2024-03-21T08:29:07.030884+0000","last_became_peered":"2024-03-21T08:29:07.030884+0000","last_unstale":"2024-03-21T08:29:07.032622+0000","last_undegraded":"2024-03-21T08:29:07.032622+0000","last_fullsized":"2024-03-21T08:29:07.032622+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:47.615776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.527720+0000","last_change":"2024-03-21T08:28:35.672948+0000","last_active":"2024-03-21T08:29:06.527720+0000","last_peered":"2024-03-21T08:29:06.527720+0000","last_clean":"2024-03-21T08:29:06.527720+0000","last_became_active":"2024-03-21T08:28:35.671576+0000","last_became_peered":"2024-03-21T08:28:35.671576+0000","last_unstale":"2024-03-21T08:29:06.527720+0000","last_undegraded":"2024-03-21T08:29:06.527720+0000","last_fullsized":"2024-03-21T08:29:06.527720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:36:25.620392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518463+0000","last_change":"2024-03-21T08:28:36.671173+0000","last_active":"2024-03-21T08:29:06.518463+0000","last_peered":"2024-03-21T08:29:06.518463+0000","last_clean":"2024-03-21T08:29:06.518463+0000","last_became_active":"2024-03-21T08:28:36.666264+0000","last_became_peered":"2024-03-21T08:28:36.666264+0000","last_unstale":"2024-03-21T08:29:06.518463+0000","last_undegraded":"2024-03-21T08:29:06.518463+0000","last_fullsized":"2024-03-21T08:29:06.518463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:35.081487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511119+0000","last_change":"2024-03-21T08:28:40.883801+0000","last_active":"2024-03-21T08:29:06.511119+0000","last_peered":"2024-03-21T08:29:06.511119+0000","last_clean":"2024-03-21T08:29:06.511119+0000","last_became_active":"2024-03-21T08:28:40.882556+0000","last_became_peered":"2024-03-21T08:28:40.882556+0000","last_unstale":"2024-03-21T08:29:06.511119+0000","last_undegraded":"2024-03-21T08:29:06.511119+0000","last_fullsized":"2024-03-21T08:29:06.511119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:24.211625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.335741+0000","last_change":"2024-03-21T08:28:35.668657+0000","last_active":"2024-03-21T08:29:08.335741+0000","last_peered":"2024-03-21T08:29:08.335741+0000","last_clean":"2024-03-21T08:29:08.335741+0000","last_became_active":"2024-03-21T08:28:35.668284+0000","last_became_peered":"2024-03-21T08:28:35.668284+0000","last_unstale":"2024-03-21T08:29:08.335741+0000","last_undegraded":"2024-03-21T08:29:08.335741+0000","last_fullsized":"2024-03-21T08:29:08.335741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:26:07.791388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512373+0000","last_change":"2024-03-21T08:28:36.652383+0000","last_active":"2024-03-21T08:29:06.512373+0000","last_peered":"2024-03-21T08:29:06.512373+0000","last_clean":"2024-03-21T08:29:06.512373+0000","last_became_active":"2024-03-21T08:28:36.652046+0000","last_became_peered":"2024-03-21T08:28:36.652046+0000","last_unstale":"2024-03-21T08:29:06.512373+0000","last_undegraded":"2024-03-21T08:29:06.512373+0000","last_fullsized":"2024-03-21T08:29:06.512373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:11:08.699391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512994+0000","last_change":"2024-03-21T08:28:40.686122+0000","last_active":"2024-03-21T08:29:06.512994+0000","last_peered":"2024-03-21T08:29:06.512994+0000","last_clean":"2024-03-21T08:29:06.512994+0000","last_became_active":"2024-03-21T08:28:40.685736+0000","last_became_peered":"2024-03-21T08:28:40.685736+0000","last_unstale":"2024-03-21T08:29:06.512994+0000","last_undegraded":"2024-03-21T08:29:06.512994+0000","last_fullsized":"2024-03-21T08:29:06.512994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:52.941896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.317246+0000","last_change":"2024-03-21T08:28:35.675197+0000","last_active":"2024-03-21T08:29:07.317246+0000","last_peered":"2024-03-21T08:29:07.317246+0000","last_clean":"2024-03-21T08:29:07.317246+0000","last_became_active":"2024-03-21T08:28:35.674688+0000","last_became_peered":"2024-03-21T08:28:35.674688+0000","last_unstale":"2024-03-21T08:29:07.317246+0000","last_undegraded":"2024-03-21T08:29:07.317246+0000","last_fullsized":"2024-03-21T08:29:07.317246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:01:11.489294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526862+0000","last_change":"2024-03-21T08:28:36.668160+0000","last_active":"2024-03-21T08:29:06.526862+0000","last_peered":"2024-03-21T08:29:06.526862+0000","last_clean":"2024-03-21T08:29:06.526862+0000","last_became_active":"2024-03-21T08:28:36.665152+0000","last_became_peered":"2024-03-21T08:28:36.665152+0000","last_unstale":"2024-03-21T08:29:06.526862+0000","last_undegraded":"2024-03-21T08:29:06.526862+0000","last_fullsized":"2024-03-21T08:29:06.526862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:43:41.803822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.781105+0000","last_change":"2024-03-21T08:28:40.884365+0000","last_active":"2024-03-21T08:29:06.781105+0000","last_peered":"2024-03-21T08:29:06.781105+0000","last_clean":"2024-03-21T08:29:06.781105+0000","last_became_active":"2024-03-21T08:28:40.883654+0000","last_became_peered":"2024-03-21T08:28:40.883654+0000","last_unstale":"2024-03-21T08:29:06.781105+0000","last_undegraded":"2024-03-21T08:29:06.781105+0000","last_fullsized":"2024-03-21T08:29:06.781105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:46:43.888121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.322708+0000","last_change":"2024-03-21T08:28:35.682548+0000","last_active":"2024-03-21T08:29:07.322708+0000","last_peered":"2024-03-21T08:29:07.322708+0000","last_clean":"2024-03-21T08:29:07.322708+0000","last_became_active":"2024-03-21T08:28:35.682133+0000","last_became_peered":"2024-03-21T08:28:35.682133+0000","last_unstale":"2024-03-21T08:29:07.322708+0000","last_undegraded":"2024-03-21T08:29:07.322708+0000","last_fullsized":"2024-03-21T08:29:07.322708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:59:13.935159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526167+0000","last_change":"2024-03-21T08:28:36.653797+0000","last_active":"2024-03-21T08:29:06.526167+0000","last_peered":"2024-03-21T08:29:06.526167+0000","last_clean":"2024-03-21T08:29:06.526167+0000","last_became_active":"2024-03-21T08:28:36.651872+0000","last_became_peered":"2024-03-21T08:28:36.651872+0000","last_unstale":"2024-03-21T08:29:06.526167+0000","last_undegraded":"2024-03-21T08:29:06.526167+0000","last_fullsized":"2024-03-21T08:29:06.526167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:09.467573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916574+0000","last_change":"2024-03-21T08:28:40.692996+0000","last_active":"2024-03-21T08:29:06.916574+0000","last_peered":"2024-03-21T08:29:06.916574+0000","last_clean":"2024-03-21T08:29:06.916574+0000","last_became_active":"2024-03-21T08:28:40.687598+0000","last_became_peered":"2024-03-21T08:28:40.687598+0000","last_unstale":"2024-03-21T08:29:06.916574+0000","last_undegraded":"2024-03-21T08:29:06.916574+0000","last_fullsized":"2024-03-21T08:29:06.916574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:09.557272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.323477+0000","last_change":"2024-03-21T08:28:35.672567+0000","last_active":"2024-03-21T08:29:07.323477+0000","last_peered":"2024-03-21T08:29:07.323477+0000","last_clean":"2024-03-21T08:29:07.323477+0000","last_became_active":"2024-03-21T08:28:35.671849+0000","last_became_peered":"2024-03-21T08:28:35.671849+0000","last_unstale":"2024-03-21T08:29:07.323477+0000","last_undegraded":"2024-03-21T08:29:07.323477+0000","last_fullsized":"2024-03-21T08:29:07.323477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:13.050227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.520556+0000","last_change":"2024-03-21T08:28:36.664110+0000","last_active":"2024-03-21T08:29:06.520556+0000","last_peered":"2024-03-21T08:29:06.520556+0000","last_clean":"2024-03-21T08:29:06.520556+0000","last_became_active":"2024-03-21T08:28:36.663297+0000","last_became_peered":"2024-03-21T08:28:36.663297+0000","last_unstale":"2024-03-21T08:29:06.520556+0000","last_undegraded":"2024-03-21T08:29:06.520556+0000","last_fullsized":"2024-03-21T08:29:06.520556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:55.450399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.779846+0000","last_change":"2024-03-21T08:28:40.682371+0000","last_active":"2024-03-21T08:29:06.779846+0000","last_peered":"2024-03-21T08:29:06.779846+0000","last_clean":"2024-03-21T08:29:06.779846+0000","last_became_active":"2024-03-21T08:28:40.681786+0000","last_became_peered":"2024-03-21T08:28:40.681786+0000","last_unstale":"2024-03-21T08:29:06.779846+0000","last_undegraded":"2024-03-21T08:29:06.779846+0000","last_fullsized":"2024-03-21T08:29:06.779846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:15:38.130807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514277+0000","last_change":"2024-03-21T08:28:35.685312+0000","last_active":"2024-03-21T08:29:06.514277+0000","last_peered":"2024-03-21T08:29:06.514277+0000","last_clean":"2024-03-21T08:29:06.514277+0000","last_became_active":"2024-03-21T08:28:35.684327+0000","last_became_peered":"2024-03-21T08:28:35.684327+0000","last_unstale":"2024-03-21T08:29:06.514277+0000","last_undegraded":"2024-03-21T08:29:06.514277+0000","last_fullsized":"2024-03-21T08:29:06.514277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:12.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513310+0000","last_change":"2024-03-21T08:28:36.671186+0000","last_active":"2024-03-21T08:29:06.513310+0000","last_peered":"2024-03-21T08:29:06.513310+0000","last_clean":"2024-03-21T08:29:06.513310+0000","last_became_active":"2024-03-21T08:28:36.662726+0000","last_became_peered":"2024-03-21T08:28:36.662726+0000","last_unstale":"2024-03-21T08:29:06.513310+0000","last_undegraded":"2024-03-21T08:29:06.513310+0000","last_fullsized":"2024-03-21T08:29:06.513310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:29.129014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.780982+0000","last_change":"2024-03-21T08:28:40.885203+0000","last_active":"2024-03-21T08:29:06.780982+0000","last_peered":"2024-03-21T08:29:06.780982+0000","last_clean":"2024-03-21T08:29:06.780982+0000","last_became_active":"2024-03-21T08:28:40.883810+0000","last_became_peered":"2024-03-21T08:28:40.883810+0000","last_unstale":"2024-03-21T08:29:06.780982+0000","last_undegraded":"2024-03-21T08:29:06.780982+0000","last_fullsized":"2024-03-21T08:29:06.780982+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:12:58.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.324911+0000","last_change":"2024-03-21T08:28:35.690494+0000","last_active":"2024-03-21T08:29:07.324911+0000","last_peered":"2024-03-21T08:29:07.324911+0000","last_clean":"2024-03-21T08:29:07.324911+0000","last_became_active":"2024-03-21T08:28:35.690071+0000","last_became_peered":"2024-03-21T08:28:35.690071+0000","last_unstale":"2024-03-21T08:29:07.324911+0000","last_undegraded":"2024-03-21T08:29:07.324911+0000","last_fullsized":"2024-03-21T08:29:07.324911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:04:46.713006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.028651+0000","last_change":"2024-03-21T08:28:36.665135+0000","last_active":"2024-03-21T08:29:07.028651+0000","last_peered":"2024-03-21T08:29:07.028651+0000","last_clean":"2024-03-21T08:29:07.028651+0000","last_became_active":"2024-03-21T08:28:36.663700+0000","last_became_peered":"2024-03-21T08:28:36.663700+0000","last_unstale":"2024-03-21T08:29:07.028651+0000","last_undegraded":"2024-03-21T08:29:07.028651+0000","last_fullsized":"2024-03-21T08:29:07.028651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:50:00.604958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.517790+0000","last_change":"2024-03-21T08:28:40.696076+0000","last_active":"2024-03-21T08:29:06.517790+0000","last_peered":"2024-03-21T08:29:06.517790+0000","last_clean":"2024-03-21T08:29:06.517790+0000","last_became_active":"2024-03-21T08:28:40.695556+0000","last_became_peered":"2024-03-21T08:28:40.695556+0000","last_unstale":"2024-03-21T08:29:06.517790+0000","last_undegraded":"2024-03-21T08:29:06.517790+0000","last_fullsized":"2024-03-21T08:29:06.517790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:42.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.335341+0000","last_change":"2024-03-21T08:28:35.685545+0000","last_active":"2024-03-21T08:29:08.335341+0000","last_peered":"2024-03-21T08:29:08.335341+0000","last_clean":"2024-03-21T08:29:08.335341+0000","last_became_active":"2024-03-21T08:28:35.684795+0000","last_became_peered":"2024-03-21T08:28:35.684795+0000","last_unstale":"2024-03-21T08:29:08.335341+0000","last_undegraded":"2024-03-21T08:29:08.335341+0000","last_fullsized":"2024-03-21T08:29:08.335341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:40.308014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029794+0000","last_change":"2024-03-21T08:28:36.656276+0000","last_active":"2024-03-21T08:29:07.029794+0000","last_peered":"2024-03-21T08:29:07.029794+0000","last_clean":"2024-03-21T08:29:07.029794+0000","last_became_active":"2024-03-21T08:28:36.655871+0000","last_became_peered":"2024-03-21T08:28:36.655871+0000","last_unstale":"2024-03-21T08:29:07.029794+0000","last_undegraded":"2024-03-21T08:29:07.029794+0000","last_fullsized":"2024-03-21T08:29:07.029794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:22.362165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":94,"state":"activating","last_fresh":"2024-03-21T08:29:06.526345+0000","last_change":"2024-03-21T08:29:06.524943+0000","last_active":"2024-03-21T08:29:05.517122+0000","last_peered":"2024-03-21T08:28:41.608538+0000","last_clean":"2024-03-21T08:28:41.608538+0000","last_became_active":"2024-03-21T08:28:40.698603+0000","last_became_peered":"2024-03-21T08:28:40.698603+0000","last_unstale":"2024-03-21T08:29:06.526345+0000","last_undegraded":"2024-03-21T08:29:06.526345+0000","last_fullsized":"2024-03-21T08:29:06.526345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":["0(0)","1(2)","4(3)","8(1)","9(1)"],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877980+0000","last_change":"2024-03-21T08:28:35.672783+0000","last_active":"2024-03-21T08:29:06.877980+0000","last_peered":"2024-03-21T08:29:06.877980+0000","last_clean":"2024-03-21T08:29:06.877980+0000","last_became_active":"2024-03-21T08:28:35.672161+0000","last_became_peered":"2024-03-21T08:28:35.672161+0000","last_unstale":"2024-03-21T08:29:06.877980+0000","last_undegraded":"2024-03-21T08:29:06.877980+0000","last_fullsized":"2024-03-21T08:29:06.877980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:37.180625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.028739+0000","last_change":"2024-03-21T08:28:36.659447+0000","last_active":"2024-03-21T08:29:07.028739+0000","last_peered":"2024-03-21T08:29:07.028739+0000","last_clean":"2024-03-21T08:29:07.028739+0000","last_became_active":"2024-03-21T08:28:36.657773+0000","last_became_peered":"2024-03-21T08:28:36.657773+0000","last_unstale":"2024-03-21T08:29:07.028739+0000","last_undegraded":"2024-03-21T08:29:07.028739+0000","last_fullsized":"2024-03-21T08:29:07.028739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:05.324498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.876555+0000","last_change":"2024-03-21T08:28:40.679806+0000","last_active":"2024-03-21T08:29:06.876555+0000","last_peered":"2024-03-21T08:29:06.876555+0000","last_clean":"2024-03-21T08:29:06.876555+0000","last_became_active":"2024-03-21T08:28:40.679329+0000","last_became_peered":"2024-03-21T08:28:40.679329+0000","last_unstale":"2024-03-21T08:29:06.876555+0000","last_undegraded":"2024-03-21T08:29:06.876555+0000","last_fullsized":"2024-03-21T08:29:06.876555+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:30:58.297581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519294+0000","last_change":"2024-03-21T08:28:35.690117+0000","last_active":"2024-03-21T08:29:06.519294+0000","last_peered":"2024-03-21T08:29:06.519294+0000","last_clean":"2024-03-21T08:29:06.519294+0000","last_became_active":"2024-03-21T08:28:35.689530+0000","last_became_peered":"2024-03-21T08:28:35.689530+0000","last_unstale":"2024-03-21T08:29:06.519294+0000","last_undegraded":"2024-03-21T08:29:06.519294+0000","last_fullsized":"2024-03-21T08:29:06.519294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:24:02.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029422+0000","last_change":"2024-03-21T08:28:36.649194+0000","last_active":"2024-03-21T08:29:07.029422+0000","last_peered":"2024-03-21T08:29:07.029422+0000","last_clean":"2024-03-21T08:29:07.029422+0000","last_became_active":"2024-03-21T08:28:36.648663+0000","last_became_peered":"2024-03-21T08:28:36.648663+0000","last_unstale":"2024-03-21T08:29:07.029422+0000","last_undegraded":"2024-03-21T08:29:07.029422+0000","last_fullsized":"2024-03-21T08:29:07.029422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:16.296886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.521057+0000","last_change":"2024-03-21T08:28:36.664473+0000","last_active":"2024-03-21T08:29:06.521057+0000","last_peered":"2024-03-21T08:29:06.521057+0000","last_clean":"2024-03-21T08:29:06.521057+0000","last_became_active":"2024-03-21T08:28:36.663785+0000","last_became_peered":"2024-03-21T08:28:36.663785+0000","last_unstale":"2024-03-21T08:29:06.521057+0000","last_undegraded":"2024-03-21T08:29:06.521057+0000","last_fullsized":"2024-03-21T08:29:06.521057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:00.409224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514656+0000","last_change":"2024-03-21T08:28:36.668372+0000","last_active":"2024-03-21T08:29:06.514656+0000","last_peered":"2024-03-21T08:29:06.514656+0000","last_clean":"2024-03-21T08:29:06.514656+0000","last_became_active":"2024-03-21T08:28:36.666227+0000","last_became_peered":"2024-03-21T08:28:36.666227+0000","last_unstale":"2024-03-21T08:29:06.514656+0000","last_undegraded":"2024-03-21T08:29:06.514656+0000","last_fullsized":"2024-03-21T08:29:06.514656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:40:56.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513642+0000","last_change":"2024-03-21T08:28:36.670770+0000","last_active":"2024-03-21T08:29:06.513642+0000","last_peered":"2024-03-21T08:29:06.513642+0000","last_clean":"2024-03-21T08:29:06.513642+0000","last_became_active":"2024-03-21T08:28:36.670467+0000","last_became_peered":"2024-03-21T08:28:36.670467+0000","last_unstale":"2024-03-21T08:29:06.513642+0000","last_undegraded":"2024-03-21T08:29:06.513642+0000","last_fullsized":"2024-03-21T08:29:06.513642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:48.075189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526197+0000","last_change":"2024-03-21T08:28:36.669395+0000","last_active":"2024-03-21T08:29:06.526197+0000","last_peered":"2024-03-21T08:29:06.526197+0000","last_clean":"2024-03-21T08:29:06.526197+0000","last_became_active":"2024-03-21T08:28:36.668788+0000","last_became_peered":"2024-03-21T08:28:36.668788+0000","last_unstale":"2024-03-21T08:29:06.526197+0000","last_undegraded":"2024-03-21T08:29:06.526197+0000","last_fullsized":"2024-03-21T08:29:06.526197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:03.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518162+0000","last_change":"2024-03-21T08:28:36.669305+0000","last_active":"2024-03-21T08:29:06.518162+0000","last_peered":"2024-03-21T08:29:06.518162+0000","last_clean":"2024-03-21T08:29:06.518162+0000","last_became_active":"2024-03-21T08:28:36.668919+0000","last_became_peered":"2024-03-21T08:28:36.668919+0000","last_unstale":"2024-03-21T08:29:06.518162+0000","last_undegraded":"2024-03-21T08:29:06.518162+0000","last_fullsized":"2024-03-21T08:29:06.518162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:29:19.440425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.522611+0000","last_change":"2024-03-21T08:28:36.666836+0000","last_active":"2024-03-21T08:29:06.522611+0000","last_peered":"2024-03-21T08:29:06.522611+0000","last_clean":"2024-03-21T08:29:06.522611+0000","last_became_active":"2024-03-21T08:28:36.666538+0000","last_became_peered":"2024-03-21T08:28:36.666538+0000","last_unstale":"2024-03-21T08:29:06.522611+0000","last_undegraded":"2024-03-21T08:29:06.522611+0000","last_fullsized":"2024-03-21T08:29:06.522611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:43.353269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514660+0000","last_change":"2024-03-21T08:28:36.667823+0000","last_active":"2024-03-21T08:29:06.514660+0000","last_peered":"2024-03-21T08:29:06.514660+0000","last_clean":"2024-03-21T08:29:06.514660+0000","last_became_active":"2024-03-21T08:28:36.666096+0000","last_became_peered":"2024-03-21T08:28:36.666096+0000","last_unstale":"2024-03-21T08:29:06.514660+0000","last_undegraded":"2024-03-21T08:29:06.514660+0000","last_fullsized":"2024-03-21T08:29:06.514660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:58.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512865+0000","last_change":"2024-03-21T08:28:36.672164+0000","last_active":"2024-03-21T08:29:06.512865+0000","last_peered":"2024-03-21T08:29:06.512865+0000","last_clean":"2024-03-21T08:29:06.512865+0000","last_became_active":"2024-03-21T08:28:36.671812+0000","last_became_peered":"2024-03-21T08:28:36.671812+0000","last_unstale":"2024-03-21T08:29:06.512865+0000","last_undegraded":"2024-03-21T08:29:06.512865+0000","last_fullsized":"2024-03-21T08:29:06.512865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:24:19.687108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877000+0000","last_change":"2024-03-21T08:28:36.675056+0000","last_active":"2024-03-21T08:29:06.877000+0000","last_peered":"2024-03-21T08:29:06.877000+0000","last_clean":"2024-03-21T08:29:06.877000+0000","last_became_active":"2024-03-21T08:28:36.674586+0000","last_became_peered":"2024-03-21T08:28:36.674586+0000","last_unstale":"2024-03-21T08:29:06.877000+0000","last_undegraded":"2024-03-21T08:29:06.877000+0000","last_fullsized":"2024-03-21T08:29:06.877000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:04:51.207066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511171+0000","last_change":"2024-03-21T08:28:36.666370+0000","last_active":"2024-03-21T08:29:06.511171+0000","last_peered":"2024-03-21T08:29:06.511171+0000","last_clean":"2024-03-21T08:29:06.511171+0000","last_became_active":"2024-03-21T08:28:36.665950+0000","last_became_peered":"2024-03-21T08:28:36.665950+0000","last_unstale":"2024-03-21T08:29:06.511171+0000","last_undegraded":"2024-03-21T08:29:06.511171+0000","last_fullsized":"2024-03-21T08:29:06.511171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:49:31.945890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.032862+0000","last_change":"2024-03-21T08:29:07.032862+0000","last_active":"2024-03-21T08:29:07.032862+0000","last_peered":"2024-03-21T08:29:07.032862+0000","last_clean":"2024-03-21T08:29:07.032862+0000","last_became_active":"2024-03-21T08:29:07.031784+0000","last_became_peered":"2024-03-21T08:29:07.031784+0000","last_unstale":"2024-03-21T08:29:07.032862+0000","last_undegraded":"2024-03-21T08:29:07.032862+0000","last_fullsized":"2024-03-21T08:29:07.032862+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:34.037495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519247+0000","last_change":"2024-03-21T08:28:36.660812+0000","last_active":"2024-03-21T08:29:06.519247+0000","last_peered":"2024-03-21T08:29:06.519247+0000","last_clean":"2024-03-21T08:29:06.519247+0000","last_became_active":"2024-03-21T08:28:36.660044+0000","last_became_peered":"2024-03-21T08:28:36.660044+0000","last_unstale":"2024-03-21T08:29:06.519247+0000","last_undegraded":"2024-03-21T08:29:06.519247+0000","last_fullsized":"2024-03-21T08:29:06.519247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:31.598678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.521779+0000","last_change":"2024-03-21T08:28:36.669441+0000","last_active":"2024-03-21T08:29:06.521779+0000","last_peered":"2024-03-21T08:29:06.521779+0000","last_clean":"2024-03-21T08:29:06.521779+0000","last_became_active":"2024-03-21T08:28:36.668668+0000","last_became_peered":"2024-03-21T08:28:36.668668+0000","last_unstale":"2024-03-21T08:29:06.521779+0000","last_undegraded":"2024-03-21T08:29:06.521779+0000","last_fullsized":"2024-03-21T08:29:06.521779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:04.828297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":128,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":45289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285591,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":510926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547228,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1687552,"data_stored":1106132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710756,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":511162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972395,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":511082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266739,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1658880,"data_stored":1102504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":510434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":514186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":501086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":535226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":510628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":510988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1091546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":29513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:29:13.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.606+0000 7f2d59200640 1 -- 172.21.15.3:0/3836729651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54108db0 msgr2=0x7f2d54109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.606+0000 7f2d59200640 1 --2- 172.21.15.3:0/3836729651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54108db0 0x7f2d54109230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009900 tx=0x7f2d4002bc30 comp rx=0 tx=0).stop 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.607+0000 7f2d59200640 1 -- 172.21.15.3:0/3836729651 shutdown_connections 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.607+0000 7f2d59200640 1 --2- 172.21.15.3:0/3836729651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54109770 0x7f2d54110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.607+0000 7f2d59200640 1 --2- 172.21.15.3:0/3836729651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54108db0 0x7f2d54109230 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.607+0000 7f2d59200640 1 --2- 172.21.15.3:0/3836729651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d54107bc0 0x7f2d54107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.607+0000 7f2d59200640 1 -- 172.21.15.3:0/3836729651 >> 172.21.15.3:0/3836729651 conn(0x7f2d54076b60 msgr2=0x7f2d54078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.607+0000 7f2d59200640 1 -- 172.21.15.3:0/3836729651 shutdown_connections 2024-03-21T08:29:13.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.607+0000 7f2d59200640 1 -- 172.21.15.3:0/3836729651 wait complete. 2024-03-21T08:29:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 Processor -- start 2024-03-21T08:29:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 -- start start 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54107bc0 0x7f2d5419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d54108db0 0x7f2d5419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54109770 0x7f2d541a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54113740 con 0x7f2d54107bc0 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d541135c0 con 0x7f2d54109770 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d59200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d541138c0 con 0x7f2d54108db0 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54107bc0 0x7f2d5419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54107bc0 0x7f2d5419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51098/0 (socket says 172.21.15.3:51098) 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.608+0000 7f2d53400640 1 -- 172.21.15.3:0/3151258970 learned_addr learned my addr 172.21.15.3:0/3151258970 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d53e00640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54109770 0x7f2d541a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d52a00640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d54108db0 0x7f2d5419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d53400640 1 -- 172.21.15.3:0/3151258970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d54108db0 msgr2=0x7f2d5419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d53400640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d54108db0 0x7f2d5419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d53400640 1 -- 172.21.15.3:0/3151258970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54109770 msgr2=0x7f2d541a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d53400640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54109770 0x7f2d541a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d53400640 1 -- 172.21.15.3:0/3151258970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d541a59d0 con 0x7f2d54107bc0 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d53400640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54107bc0 0x7f2d5419ddb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d950 tx=0x7f2d4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.609+0000 7f2d4be00640 1 -- 172.21.15.3:0/3151258970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44017070 con 0x7f2d54107bc0 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.610+0000 7f2d4be00640 1 -- 172.21.15.3:0/3151258970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d44009c30 con 0x7f2d54107bc0 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.610+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d541a5cc0 con 0x7f2d54107bc0 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.610+0000 7f2d4be00640 1 -- 172.21.15.3:0/3151258970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400f620 con 0x7f2d54107bc0 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.610+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d541a6190 con 0x7f2d54107bc0 2024-03-21T08:29:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.611+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5410eee0 con 0x7f2d54107bc0 2024-03-21T08:29:13.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.613+0000 7f2d4be00640 1 -- 172.21.15.3:0/3151258970 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f2d4401a070 con 0x7f2d54107bc0 2024-03-21T08:29:13.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.614+0000 7f2d4be00640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d380a3020 0x7f2d380a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:13.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.614+0000 7f2d4be00640 1 -- 172.21.15.3:0/3151258970 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2d440cf2f0 con 0x7f2d54107bc0 2024-03-21T08:29:13.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.614+0000 7f2d52a00640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d380a3020 0x7f2d380a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:13.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.615+0000 7f2d52a00640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d380a3020 0x7f2d380a54e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d40006010 tx=0x7f2d40000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.618+0000 7f2d4be00640 1 -- 172.21.15.3:0/3151258970 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d44009de0 con 0x7f2d54107bc0 2024-03-21T08:29:13.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:13 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3382867919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.819+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d54061ae0 con 0x7f2d380a3020 2024-03-21T08:29:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.849+0000 7f2d4be00640 1 -- 172.21.15.3:0/3151258970 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340434 (secure 0 0 0) 0x7f2d54061ae0 con 0x7f2d380a3020 2024-03-21T08:29:13.856 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d380a3020 msgr2=0x7f2d380a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d380a3020 0x7f2d380a54e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d40006010 tx=0x7f2d40000bc0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54107bc0 msgr2=0x7f2d5419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54107bc0 0x7f2d5419ddb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d950 tx=0x7f2d4400de10 comp rx=0 tx=0).stop 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 shutdown_connections 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d54109770 0x7f2d541a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d380a3020 0x7f2d380a54e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d54108db0 0x7f2d5419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 --2- 172.21.15.3:0/3151258970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d54107bc0 0x7f2d5419ddb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.856+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 >> 172.21.15.3:0/3151258970 conn(0x7f2d54076b60 msgr2=0x7f2d5410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.857+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 shutdown_connections 2024-03-21T08:29:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:13.857+0000 7f2d59200640 1 -- 172.21.15.3:0/3151258970 wait complete. 2024-03-21T08:29:13.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:13 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3382867919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:13.880 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-03-21T08:29:13.345061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598211,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":155,"num_read_kb":133,"num_write":310,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":416,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371816,"kb_used_data":14328,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570008,"statfs":{"total":1151940427776,"available":1151559688192,"internally_reserved":0,"allocated":14671872,"data_stored":7978394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3203,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":80,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003509"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.525299+0000","last_change":"2024-03-21T08:28:36.667851+0000","last_active":"2024-03-21T08:29:06.525299+0000","last_peered":"2024-03-21T08:29:06.525299+0000","last_clean":"2024-03-21T08:29:06.525299+0000","last_became_active":"2024-03-21T08:28:36.665713+0000","last_became_peered":"2024-03-21T08:28:36.665713+0000","last_unstale":"2024-03-21T08:29:06.525299+0000","last_undegraded":"2024-03-21T08:29:06.525299+0000","last_fullsized":"2024-03-21T08:29:06.525299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:02.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526623+0000","last_change":"2024-03-21T08:28:36.650986+0000","last_active":"2024-03-21T08:29:06.526623+0000","last_peered":"2024-03-21T08:29:06.526623+0000","last_clean":"2024-03-21T08:29:06.526623+0000","last_became_active":"2024-03-21T08:28:36.650633+0000","last_became_peered":"2024-03-21T08:28:36.650633+0000","last_unstale":"2024-03-21T08:29:06.526623+0000","last_undegraded":"2024-03-21T08:29:06.526623+0000","last_fullsized":"2024-03-21T08:29:06.526623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:40:20.801814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.524980+0000","last_change":"2024-03-21T08:28:36.662984+0000","last_active":"2024-03-21T08:29:06.524980+0000","last_peered":"2024-03-21T08:29:06.524980+0000","last_clean":"2024-03-21T08:29:06.524980+0000","last_became_active":"2024-03-21T08:28:36.662657+0000","last_became_peered":"2024-03-21T08:28:36.662657+0000","last_unstale":"2024-03-21T08:29:06.524980+0000","last_undegraded":"2024-03-21T08:29:06.524980+0000","last_fullsized":"2024-03-21T08:29:06.524980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:41.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.782243+0000","last_change":"2024-03-21T08:28:36.665958+0000","last_active":"2024-03-21T08:29:06.782243+0000","last_peered":"2024-03-21T08:29:06.782243+0000","last_clean":"2024-03-21T08:29:06.782243+0000","last_became_active":"2024-03-21T08:28:36.664309+0000","last_became_peered":"2024-03-21T08:28:36.664309+0000","last_unstale":"2024-03-21T08:29:06.782243+0000","last_undegraded":"2024-03-21T08:29:06.782243+0000","last_fullsized":"2024-03-21T08:29:06.782243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:16.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526265+0000","last_change":"2024-03-21T08:28:36.657767+0000","last_active":"2024-03-21T08:29:06.526265+0000","last_peered":"2024-03-21T08:29:06.526265+0000","last_clean":"2024-03-21T08:29:06.526265+0000","last_became_active":"2024-03-21T08:28:36.657350+0000","last_became_peered":"2024-03-21T08:28:36.657350+0000","last_unstale":"2024-03-21T08:29:06.526265+0000","last_undegraded":"2024-03-21T08:29:06.526265+0000","last_fullsized":"2024-03-21T08:29:06.526265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:47.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916971+0000","last_change":"2024-03-21T08:29:06.916971+0000","last_active":"2024-03-21T08:29:06.916971+0000","last_peered":"2024-03-21T08:29:06.916971+0000","last_clean":"2024-03-21T08:29:06.916971+0000","last_became_active":"2024-03-21T08:29:06.915997+0000","last_became_peered":"2024-03-21T08:29:06.915997+0000","last_unstale":"2024-03-21T08:29:06.916971+0000","last_undegraded":"2024-03-21T08:29:06.916971+0000","last_fullsized":"2024-03-21T08:29:06.916971+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:02.525190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.027701+0000","last_change":"2024-03-21T08:28:36.667787+0000","last_active":"2024-03-21T08:29:07.027701+0000","last_peered":"2024-03-21T08:29:07.027701+0000","last_clean":"2024-03-21T08:29:07.027701+0000","last_became_active":"2024-03-21T08:28:36.667451+0000","last_became_peered":"2024-03-21T08:28:36.667451+0000","last_unstale":"2024-03-21T08:29:07.027701+0000","last_undegraded":"2024-03-21T08:29:07.027701+0000","last_fullsized":"2024-03-21T08:29:07.027701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:01.874897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511084+0000","last_change":"2024-03-21T08:28:36.662445+0000","last_active":"2024-03-21T08:29:06.511084+0000","last_peered":"2024-03-21T08:29:06.511084+0000","last_clean":"2024-03-21T08:29:06.511084+0000","last_became_active":"2024-03-21T08:28:36.662113+0000","last_became_peered":"2024-03-21T08:28:36.662113+0000","last_unstale":"2024-03-21T08:29:06.511084+0000","last_undegraded":"2024-03-21T08:29:06.511084+0000","last_fullsized":"2024-03-21T08:29:06.511084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:21.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.520184+0000","last_change":"2024-03-21T08:28:36.666084+0000","last_active":"2024-03-21T08:29:06.520184+0000","last_peered":"2024-03-21T08:29:06.520184+0000","last_clean":"2024-03-21T08:29:06.520184+0000","last_became_active":"2024-03-21T08:28:36.665354+0000","last_became_peered":"2024-03-21T08:28:36.665354+0000","last_unstale":"2024-03-21T08:29:06.520184+0000","last_undegraded":"2024-03-21T08:29:06.520184+0000","last_fullsized":"2024-03-21T08:29:06.520184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:02:50.160765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.919231+0000","last_change":"2024-03-21T08:29:06.919231+0000","last_active":"2024-03-21T08:29:06.919231+0000","last_peered":"2024-03-21T08:29:06.919231+0000","last_clean":"2024-03-21T08:29:06.919231+0000","last_became_active":"2024-03-21T08:29:06.918199+0000","last_became_peered":"2024-03-21T08:29:06.918199+0000","last_unstale":"2024-03-21T08:29:06.919231+0000","last_undegraded":"2024-03-21T08:29:06.919231+0000","last_fullsized":"2024-03-21T08:29:06.919231+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:44.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029760+0000","last_change":"2024-03-21T08:28:36.656385+0000","last_active":"2024-03-21T08:29:07.029760+0000","last_peered":"2024-03-21T08:29:07.029760+0000","last_clean":"2024-03-21T08:29:07.029760+0000","last_became_active":"2024-03-21T08:28:36.656014+0000","last_became_peered":"2024-03-21T08:28:36.656014+0000","last_unstale":"2024-03-21T08:29:07.029760+0000","last_undegraded":"2024-03-21T08:29:07.029760+0000","last_fullsized":"2024-03-21T08:29:07.029760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:46.000495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.781932+0000","last_change":"2024-03-21T08:28:36.666254+0000","last_active":"2024-03-21T08:29:06.781932+0000","last_peered":"2024-03-21T08:29:06.781932+0000","last_clean":"2024-03-21T08:29:06.781932+0000","last_became_active":"2024-03-21T08:28:36.664698+0000","last_became_peered":"2024-03-21T08:28:36.664698+0000","last_unstale":"2024-03-21T08:29:06.781932+0000","last_undegraded":"2024-03-21T08:29:06.781932+0000","last_fullsized":"2024-03-21T08:29:06.781932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:03:08.358159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.919930+0000","last_change":"2024-03-21T08:29:06.919930+0000","last_active":"2024-03-21T08:29:06.919930+0000","last_peered":"2024-03-21T08:29:06.919930+0000","last_clean":"2024-03-21T08:29:06.919930+0000","last_became_active":"2024-03-21T08:29:06.917852+0000","last_became_peered":"2024-03-21T08:29:06.917852+0000","last_unstale":"2024-03-21T08:29:06.919930+0000","last_undegraded":"2024-03-21T08:29:06.919930+0000","last_fullsized":"2024-03-21T08:29:06.919930+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:05:01.185388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.523780+0000","last_change":"2024-03-21T08:28:36.655229+0000","last_active":"2024-03-21T08:29:06.523780+0000","last_peered":"2024-03-21T08:29:06.523780+0000","last_clean":"2024-03-21T08:29:06.523780+0000","last_became_active":"2024-03-21T08:28:36.653949+0000","last_became_peered":"2024-03-21T08:28:36.653949+0000","last_unstale":"2024-03-21T08:29:06.523780+0000","last_undegraded":"2024-03-21T08:29:06.523780+0000","last_fullsized":"2024-03-21T08:29:06.523780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:57:38.370897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.523277+0000","last_change":"2024-03-21T08:28:36.664894+0000","last_active":"2024-03-21T08:29:06.523277+0000","last_peered":"2024-03-21T08:29:06.523277+0000","last_clean":"2024-03-21T08:29:06.523277+0000","last_became_active":"2024-03-21T08:28:36.664523+0000","last_became_peered":"2024-03-21T08:28:36.664523+0000","last_unstale":"2024-03-21T08:29:06.523277+0000","last_undegraded":"2024-03-21T08:29:06.523277+0000","last_fullsized":"2024-03-21T08:29:06.523277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:52:54.147459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.517632+0000","last_change":"2024-03-21T08:28:36.662374+0000","last_active":"2024-03-21T08:29:06.517632+0000","last_peered":"2024-03-21T08:29:06.517632+0000","last_clean":"2024-03-21T08:29:06.517632+0000","last_became_active":"2024-03-21T08:28:36.662066+0000","last_became_peered":"2024-03-21T08:28:36.662066+0000","last_unstale":"2024-03-21T08:29:06.517632+0000","last_undegraded":"2024-03-21T08:29:06.517632+0000","last_fullsized":"2024-03-21T08:29:06.517632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:06.082925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.524487+0000","last_change":"2024-03-21T08:28:36.672172+0000","last_active":"2024-03-21T08:29:06.524487+0000","last_peered":"2024-03-21T08:29:06.524487+0000","last_clean":"2024-03-21T08:29:06.524487+0000","last_became_active":"2024-03-21T08:28:36.671657+0000","last_became_peered":"2024-03-21T08:28:36.671657+0000","last_unstale":"2024-03-21T08:29:06.524487+0000","last_undegraded":"2024-03-21T08:29:06.524487+0000","last_fullsized":"2024-03-21T08:29:06.524487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:56.892818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.918218+0000","last_change":"2024-03-21T08:28:36.666973+0000","last_active":"2024-03-21T08:29:06.918218+0000","last_peered":"2024-03-21T08:29:06.918218+0000","last_clean":"2024-03-21T08:29:06.918218+0000","last_became_active":"2024-03-21T08:28:36.663770+0000","last_became_peered":"2024-03-21T08:28:36.663770+0000","last_unstale":"2024-03-21T08:29:06.918218+0000","last_undegraded":"2024-03-21T08:29:06.918218+0000","last_fullsized":"2024-03-21T08:29:06.918218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:54:26.701009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514090+0000","last_change":"2024-03-21T08:28:36.668793+0000","last_active":"2024-03-21T08:29:06.514090+0000","last_peered":"2024-03-21T08:29:06.514090+0000","last_clean":"2024-03-21T08:29:06.514090+0000","last_became_active":"2024-03-21T08:28:36.668452+0000","last_became_peered":"2024-03-21T08:28:36.668452+0000","last_unstale":"2024-03-21T08:29:06.514090+0000","last_undegraded":"2024-03-21T08:29:06.514090+0000","last_fullsized":"2024-03-21T08:29:06.514090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:45.191849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513391+0000","last_change":"2024-03-21T08:28:40.687618+0000","last_active":"2024-03-21T08:29:06.513391+0000","last_peered":"2024-03-21T08:29:06.513391+0000","last_clean":"2024-03-21T08:29:06.513391+0000","last_became_active":"2024-03-21T08:28:40.687155+0000","last_became_peered":"2024-03-21T08:28:40.687155+0000","last_unstale":"2024-03-21T08:29:06.513391+0000","last_undegraded":"2024-03-21T08:29:06.513391+0000","last_fullsized":"2024-03-21T08:29:06.513391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:08.014790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.336746+0000","last_change":"2024-03-21T08:28:35.679236+0000","last_active":"2024-03-21T08:29:08.336746+0000","last_peered":"2024-03-21T08:29:08.336746+0000","last_clean":"2024-03-21T08:29:08.336746+0000","last_became_active":"2024-03-21T08:28:35.678838+0000","last_became_peered":"2024-03-21T08:28:35.678838+0000","last_unstale":"2024-03-21T08:29:08.336746+0000","last_undegraded":"2024-03-21T08:29:08.336746+0000","last_fullsized":"2024-03-21T08:29:08.336746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:15.310068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512989+0000","last_change":"2024-03-21T08:28:36.670325+0000","last_active":"2024-03-21T08:29:06.512989+0000","last_peered":"2024-03-21T08:29:06.512989+0000","last_clean":"2024-03-21T08:29:06.512989+0000","last_became_active":"2024-03-21T08:28:36.669783+0000","last_became_peered":"2024-03-21T08:28:36.669783+0000","last_unstale":"2024-03-21T08:29:06.512989+0000","last_undegraded":"2024-03-21T08:29:06.512989+0000","last_fullsized":"2024-03-21T08:29:06.512989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:24.166567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.779634+0000","last_change":"2024-03-21T08:28:40.689849+0000","last_active":"2024-03-21T08:29:06.779634+0000","last_peered":"2024-03-21T08:29:06.779634+0000","last_clean":"2024-03-21T08:29:06.779634+0000","last_became_active":"2024-03-21T08:28:40.689307+0000","last_became_peered":"2024-03-21T08:28:40.689307+0000","last_unstale":"2024-03-21T08:29:06.779634+0000","last_undegraded":"2024-03-21T08:29:06.779634+0000","last_fullsized":"2024-03-21T08:29:06.779634+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:34:05.660317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513463+0000","last_change":"2024-03-21T08:28:35.674685+0000","last_active":"2024-03-21T08:29:06.513463+0000","last_peered":"2024-03-21T08:29:06.513463+0000","last_clean":"2024-03-21T08:29:06.513463+0000","last_became_active":"2024-03-21T08:28:35.674312+0000","last_became_peered":"2024-03-21T08:28:35.674312+0000","last_unstale":"2024-03-21T08:29:06.513463+0000","last_undegraded":"2024-03-21T08:29:06.513463+0000","last_fullsized":"2024-03-21T08:29:06.513463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:27.644111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526724+0000","last_change":"2024-03-21T08:28:36.667744+0000","last_active":"2024-03-21T08:29:06.526724+0000","last_peered":"2024-03-21T08:29:06.526724+0000","last_clean":"2024-03-21T08:29:06.526724+0000","last_became_active":"2024-03-21T08:28:36.667279+0000","last_became_peered":"2024-03-21T08:28:36.667279+0000","last_unstale":"2024-03-21T08:29:06.526724+0000","last_undegraded":"2024-03-21T08:29:06.526724+0000","last_fullsized":"2024-03-21T08:29:06.526724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:14:24.974683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.780335+0000","last_change":"2024-03-21T08:28:40.688864+0000","last_active":"2024-03-21T08:29:06.780335+0000","last_peered":"2024-03-21T08:29:06.780335+0000","last_clean":"2024-03-21T08:29:06.780335+0000","last_became_active":"2024-03-21T08:28:40.688029+0000","last_became_peered":"2024-03-21T08:28:40.688029+0000","last_unstale":"2024-03-21T08:29:06.780335+0000","last_undegraded":"2024-03-21T08:29:06.780335+0000","last_fullsized":"2024-03-21T08:29:06.780335+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:39.235807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.333838+0000","last_change":"2024-03-21T08:28:35.687682+0000","last_active":"2024-03-21T08:29:08.333838+0000","last_peered":"2024-03-21T08:29:08.333838+0000","last_clean":"2024-03-21T08:29:08.333838+0000","last_became_active":"2024-03-21T08:28:35.686473+0000","last_became_peered":"2024-03-21T08:28:35.686473+0000","last_unstale":"2024-03-21T08:29:08.333838+0000","last_undegraded":"2024-03-21T08:29:08.333838+0000","last_fullsized":"2024-03-21T08:29:08.333838+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:30.618673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519265+0000","last_change":"2024-03-21T08:28:36.667244+0000","last_active":"2024-03-21T08:29:06.519265+0000","last_peered":"2024-03-21T08:29:06.519265+0000","last_clean":"2024-03-21T08:29:06.519265+0000","last_became_active":"2024-03-21T08:28:36.666829+0000","last_became_peered":"2024-03-21T08:28:36.666829+0000","last_unstale":"2024-03-21T08:29:06.519265+0000","last_undegraded":"2024-03-21T08:29:06.519265+0000","last_fullsized":"2024-03-21T08:29:06.519265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:12:56.842084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.779575+0000","last_change":"2024-03-21T08:28:40.692621+0000","last_active":"2024-03-21T08:29:06.779575+0000","last_peered":"2024-03-21T08:29:06.779575+0000","last_clean":"2024-03-21T08:29:06.779575+0000","last_became_active":"2024-03-21T08:28:40.691984+0000","last_became_peered":"2024-03-21T08:28:40.691984+0000","last_unstale":"2024-03-21T08:29:06.779575+0000","last_undegraded":"2024-03-21T08:29:06.779575+0000","last_fullsized":"2024-03-21T08:29:06.779575+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:29.330702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.529238+0000","last_change":"2024-03-21T08:28:35.676421+0000","last_active":"2024-03-21T08:29:06.529238+0000","last_peered":"2024-03-21T08:29:06.529238+0000","last_clean":"2024-03-21T08:29:06.529238+0000","last_became_active":"2024-03-21T08:28:35.674957+0000","last_became_peered":"2024-03-21T08:28:35.674957+0000","last_unstale":"2024-03-21T08:29:06.529238+0000","last_undegraded":"2024-03-21T08:29:06.529238+0000","last_fullsized":"2024-03-21T08:29:06.529238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:01:38.397757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877668+0000","last_change":"2024-03-21T08:28:36.664662+0000","last_active":"2024-03-21T08:29:06.877668+0000","last_peered":"2024-03-21T08:29:06.877668+0000","last_clean":"2024-03-21T08:29:06.877668+0000","last_became_active":"2024-03-21T08:28:36.664324+0000","last_became_peered":"2024-03-21T08:28:36.664324+0000","last_unstale":"2024-03-21T08:29:06.877668+0000","last_undegraded":"2024-03-21T08:29:06.877668+0000","last_fullsized":"2024-03-21T08:29:06.877668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:57.325869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513597+0000","last_change":"2024-03-21T08:28:40.676297+0000","last_active":"2024-03-21T08:29:06.513597+0000","last_peered":"2024-03-21T08:29:06.513597+0000","last_clean":"2024-03-21T08:29:06.513597+0000","last_became_active":"2024-03-21T08:28:40.675337+0000","last_became_peered":"2024-03-21T08:28:40.675337+0000","last_unstale":"2024-03-21T08:29:06.513597+0000","last_undegraded":"2024-03-21T08:29:06.513597+0000","last_fullsized":"2024-03-21T08:29:06.513597+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:05.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:11.351259+0000","last_change":"2024-03-21T08:28:35.687651+0000","last_active":"2024-03-21T08:29:11.351259+0000","last_peered":"2024-03-21T08:29:11.351259+0000","last_clean":"2024-03-21T08:29:11.351259+0000","last_became_active":"2024-03-21T08:28:35.686284+0000","last_became_peered":"2024-03-21T08:28:35.686284+0000","last_unstale":"2024-03-21T08:29:11.351259+0000","last_undegraded":"2024-03-21T08:29:11.351259+0000","last_fullsized":"2024-03-21T08:29:11.351259+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:56.121798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.033436+0000","last_change":"2024-03-21T08:29:07.033436+0000","last_active":"2024-03-21T08:29:07.033436+0000","last_peered":"2024-03-21T08:29:07.033436+0000","last_clean":"2024-03-21T08:29:07.033436+0000","last_became_active":"2024-03-21T08:29:07.032012+0000","last_became_peered":"2024-03-21T08:29:07.032012+0000","last_unstale":"2024-03-21T08:29:07.033436+0000","last_undegraded":"2024-03-21T08:29:07.033436+0000","last_fullsized":"2024-03-21T08:29:07.033436+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:44:47.009078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.882449+0000","last_change":"2024-03-21T08:29:06.882449+0000","last_active":"2024-03-21T08:29:06.882449+0000","last_peered":"2024-03-21T08:29:06.882449+0000","last_clean":"2024-03-21T08:29:06.882449+0000","last_became_active":"2024-03-21T08:29:06.880682+0000","last_became_peered":"2024-03-21T08:29:06.880682+0000","last_unstale":"2024-03-21T08:29:06.882449+0000","last_undegraded":"2024-03-21T08:29:06.882449+0000","last_fullsized":"2024-03-21T08:29:06.882449+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:09.464388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334899+0000","last_change":"2024-03-21T08:28:35.679252+0000","last_active":"2024-03-21T08:29:08.334899+0000","last_peered":"2024-03-21T08:29:08.334899+0000","last_clean":"2024-03-21T08:29:08.334899+0000","last_became_active":"2024-03-21T08:28:35.678848+0000","last_became_peered":"2024-03-21T08:28:35.678848+0000","last_unstale":"2024-03-21T08:29:08.334899+0000","last_undegraded":"2024-03-21T08:29:08.334899+0000","last_fullsized":"2024-03-21T08:29:08.334899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:19:11.261660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878657+0000","last_change":"2024-03-21T08:28:36.670176+0000","last_active":"2024-03-21T08:29:06.878657+0000","last_peered":"2024-03-21T08:29:06.878657+0000","last_clean":"2024-03-21T08:29:06.878657+0000","last_became_active":"2024-03-21T08:28:36.669477+0000","last_became_peered":"2024-03-21T08:28:36.669477+0000","last_unstale":"2024-03-21T08:29:06.878657+0000","last_undegraded":"2024-03-21T08:29:06.878657+0000","last_fullsized":"2024-03-21T08:29:06.878657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:10:21.289484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.784996+0000","last_change":"2024-03-21T08:29:06.784996+0000","last_active":"2024-03-21T08:29:06.784996+0000","last_peered":"2024-03-21T08:29:06.784996+0000","last_clean":"2024-03-21T08:29:06.784996+0000","last_became_active":"2024-03-21T08:29:06.783403+0000","last_became_peered":"2024-03-21T08:29:06.783403+0000","last_unstale":"2024-03-21T08:29:06.784996+0000","last_undegraded":"2024-03-21T08:29:06.784996+0000","last_fullsized":"2024-03-21T08:29:06.784996+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:18:59.912956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334707+0000","last_change":"2024-03-21T08:28:35.667502+0000","last_active":"2024-03-21T08:29:08.334707+0000","last_peered":"2024-03-21T08:29:08.334707+0000","last_clean":"2024-03-21T08:29:08.334707+0000","last_became_active":"2024-03-21T08:28:35.666667+0000","last_became_peered":"2024-03-21T08:28:35.666667+0000","last_unstale":"2024-03-21T08:29:08.334707+0000","last_undegraded":"2024-03-21T08:29:08.334707+0000","last_fullsized":"2024-03-21T08:29:08.334707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:42.196361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511542+0000","last_change":"2024-03-21T08:28:36.670855+0000","last_active":"2024-03-21T08:29:06.511542+0000","last_peered":"2024-03-21T08:29:06.511542+0000","last_clean":"2024-03-21T08:29:06.511542+0000","last_became_active":"2024-03-21T08:28:36.666012+0000","last_became_peered":"2024-03-21T08:28:36.666012+0000","last_unstale":"2024-03-21T08:29:06.511542+0000","last_undegraded":"2024-03-21T08:29:06.511542+0000","last_fullsized":"2024-03-21T08:29:06.511542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:19.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526807+0000","last_change":"2024-03-21T08:28:40.700037+0000","last_active":"2024-03-21T08:29:06.526807+0000","last_peered":"2024-03-21T08:29:06.526807+0000","last_clean":"2024-03-21T08:29:06.526807+0000","last_became_active":"2024-03-21T08:28:40.698127+0000","last_became_peered":"2024-03-21T08:28:40.698127+0000","last_unstale":"2024-03-21T08:29:06.526807+0000","last_undegraded":"2024-03-21T08:29:06.526807+0000","last_fullsized":"2024-03-21T08:29:06.526807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:48:57.135690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877020+0000","last_change":"2024-03-21T08:28:35.677071+0000","last_active":"2024-03-21T08:29:06.877020+0000","last_peered":"2024-03-21T08:29:06.877020+0000","last_clean":"2024-03-21T08:29:06.877020+0000","last_became_active":"2024-03-21T08:28:35.676652+0000","last_became_peered":"2024-03-21T08:28:35.676652+0000","last_unstale":"2024-03-21T08:29:06.877020+0000","last_undegraded":"2024-03-21T08:29:06.877020+0000","last_fullsized":"2024-03-21T08:29:06.877020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:24:54.097952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.915114+0000","last_change":"2024-03-21T08:28:36.677981+0000","last_active":"2024-03-21T08:29:06.915114+0000","last_peered":"2024-03-21T08:29:06.915114+0000","last_clean":"2024-03-21T08:29:06.915114+0000","last_became_active":"2024-03-21T08:28:36.677512+0000","last_became_peered":"2024-03-21T08:28:36.677512+0000","last_unstale":"2024-03-21T08:29:06.915114+0000","last_undegraded":"2024-03-21T08:29:06.915114+0000","last_fullsized":"2024-03-21T08:29:06.915114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:50.742928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512688+0000","last_change":"2024-03-21T08:28:40.692672+0000","last_active":"2024-03-21T08:29:06.512688+0000","last_peered":"2024-03-21T08:29:06.512688+0000","last_clean":"2024-03-21T08:29:06.512688+0000","last_became_active":"2024-03-21T08:28:40.688940+0000","last_became_peered":"2024-03-21T08:28:40.688940+0000","last_unstale":"2024-03-21T08:29:06.512688+0000","last_undegraded":"2024-03-21T08:29:06.512688+0000","last_fullsized":"2024-03-21T08:29:06.512688+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:39.569709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.325701+0000","last_change":"2024-03-21T08:28:35.681970+0000","last_active":"2024-03-21T08:29:07.325701+0000","last_peered":"2024-03-21T08:29:07.325701+0000","last_clean":"2024-03-21T08:29:07.325701+0000","last_became_active":"2024-03-21T08:28:35.681252+0000","last_became_peered":"2024-03-21T08:28:35.681252+0000","last_unstale":"2024-03-21T08:29:07.325701+0000","last_undegraded":"2024-03-21T08:29:07.325701+0000","last_fullsized":"2024-03-21T08:29:07.325701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:46.345359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519859+0000","last_change":"2024-03-21T08:28:36.675419+0000","last_active":"2024-03-21T08:29:06.519859+0000","last_peered":"2024-03-21T08:29:06.519859+0000","last_clean":"2024-03-21T08:29:06.519859+0000","last_became_active":"2024-03-21T08:28:36.674039+0000","last_became_peered":"2024-03-21T08:28:36.674039+0000","last_unstale":"2024-03-21T08:29:06.519859+0000","last_undegraded":"2024-03-21T08:29:06.519859+0000","last_fullsized":"2024-03-21T08:29:06.519859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:09.667634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.915099+0000","last_change":"2024-03-21T08:28:40.690076+0000","last_active":"2024-03-21T08:29:06.915099+0000","last_peered":"2024-03-21T08:29:06.915099+0000","last_clean":"2024-03-21T08:29:06.915099+0000","last_became_active":"2024-03-21T08:28:40.683817+0000","last_became_peered":"2024-03-21T08:28:40.683817+0000","last_unstale":"2024-03-21T08:29:06.915099+0000","last_undegraded":"2024-03-21T08:29:06.915099+0000","last_fullsized":"2024-03-21T08:29:06.915099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:33:00.948879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.321229+0000","last_change":"2024-03-21T08:28:35.690693+0000","last_active":"2024-03-21T08:29:07.321229+0000","last_peered":"2024-03-21T08:29:07.321229+0000","last_clean":"2024-03-21T08:29:07.321229+0000","last_became_active":"2024-03-21T08:28:35.690250+0000","last_became_peered":"2024-03-21T08:28:35.690250+0000","last_unstale":"2024-03-21T08:29:07.321229+0000","last_undegraded":"2024-03-21T08:29:07.321229+0000","last_fullsized":"2024-03-21T08:29:07.321229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:44:21.307772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526656+0000","last_change":"2024-03-21T08:28:36.661872+0000","last_active":"2024-03-21T08:29:06.526656+0000","last_peered":"2024-03-21T08:29:06.526656+0000","last_clean":"2024-03-21T08:29:06.526656+0000","last_became_active":"2024-03-21T08:28:36.661483+0000","last_became_peered":"2024-03-21T08:28:36.661483+0000","last_unstale":"2024-03-21T08:29:06.526656+0000","last_undegraded":"2024-03-21T08:29:06.526656+0000","last_fullsized":"2024-03-21T08:29:06.526656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:03:35.630141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.780990+0000","last_change":"2024-03-21T08:28:40.682753+0000","last_active":"2024-03-21T08:29:06.780990+0000","last_peered":"2024-03-21T08:29:06.780990+0000","last_clean":"2024-03-21T08:29:06.780990+0000","last_became_active":"2024-03-21T08:28:40.682278+0000","last_became_peered":"2024-03-21T08:28:40.682278+0000","last_unstale":"2024-03-21T08:29:06.780990+0000","last_undegraded":"2024-03-21T08:29:06.780990+0000","last_fullsized":"2024-03-21T08:29:06.780990+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:20.280442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.325789+0000","last_change":"2024-03-21T08:28:35.672524+0000","last_active":"2024-03-21T08:29:07.325789+0000","last_peered":"2024-03-21T08:29:07.325789+0000","last_clean":"2024-03-21T08:29:07.325789+0000","last_became_active":"2024-03-21T08:28:35.672051+0000","last_became_peered":"2024-03-21T08:28:35.672051+0000","last_unstale":"2024-03-21T08:29:07.325789+0000","last_undegraded":"2024-03-21T08:29:07.325789+0000","last_fullsized":"2024-03-21T08:29:07.325789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:53.336320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.532172+0000","last_change":"2024-03-21T08:29:06.532172+0000","last_active":"2024-03-21T08:29:06.532172+0000","last_peered":"2024-03-21T08:29:06.532172+0000","last_clean":"2024-03-21T08:29:06.532172+0000","last_became_active":"2024-03-21T08:29:06.531439+0000","last_became_peered":"2024-03-21T08:29:06.531439+0000","last_unstale":"2024-03-21T08:29:06.532172+0000","last_undegraded":"2024-03-21T08:29:06.532172+0000","last_fullsized":"2024-03-21T08:29:06.532172+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:27.222359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.915105+0000","last_change":"2024-03-21T08:28:40.682294+0000","last_active":"2024-03-21T08:29:06.915105+0000","last_peered":"2024-03-21T08:29:06.915105+0000","last_clean":"2024-03-21T08:29:06.915105+0000","last_became_active":"2024-03-21T08:28:40.681855+0000","last_became_peered":"2024-03-21T08:28:40.681855+0000","last_unstale":"2024-03-21T08:29:06.915105+0000","last_undegraded":"2024-03-21T08:29:06.915105+0000","last_fullsized":"2024-03-21T08:29:06.915105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:49.756322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'12","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.319825+0000","last_change":"2024-03-21T08:28:35.682947+0000","last_active":"2024-03-21T08:29:07.319825+0000","last_peered":"2024-03-21T08:29:07.319825+0000","last_clean":"2024-03-21T08:29:07.319825+0000","last_became_active":"2024-03-21T08:28:35.682395+0000","last_became_peered":"2024-03-21T08:28:35.682395+0000","last_unstale":"2024-03-21T08:29:07.319825+0000","last_undegraded":"2024-03-21T08:29:07.319825+0000","last_fullsized":"2024-03-21T08:29:07.319825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:07:48.072988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.527327+0000","last_change":"2024-03-21T08:28:36.651140+0000","last_active":"2024-03-21T08:29:06.527327+0000","last_peered":"2024-03-21T08:29:06.527327+0000","last_clean":"2024-03-21T08:29:06.527327+0000","last_became_active":"2024-03-21T08:28:36.650551+0000","last_became_peered":"2024-03-21T08:28:36.650551+0000","last_unstale":"2024-03-21T08:29:06.527327+0000","last_undegraded":"2024-03-21T08:29:06.527327+0000","last_fullsized":"2024-03-21T08:29:06.527327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:53.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878806+0000","last_change":"2024-03-21T08:28:40.691814+0000","last_active":"2024-03-21T08:29:06.878806+0000","last_peered":"2024-03-21T08:29:06.878806+0000","last_clean":"2024-03-21T08:29:06.878806+0000","last_became_active":"2024-03-21T08:28:40.691372+0000","last_became_peered":"2024-03-21T08:28:40.691372+0000","last_unstale":"2024-03-21T08:29:06.878806+0000","last_undegraded":"2024-03-21T08:29:06.878806+0000","last_fullsized":"2024-03-21T08:29:06.878806+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:21:42.612037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.332492+0000","last_change":"2024-03-21T08:28:35.687178+0000","last_active":"2024-03-21T08:29:08.332492+0000","last_peered":"2024-03-21T08:29:08.332492+0000","last_clean":"2024-03-21T08:29:08.332492+0000","last_became_active":"2024-03-21T08:28:35.686771+0000","last_became_peered":"2024-03-21T08:28:35.686771+0000","last_unstale":"2024-03-21T08:29:08.332492+0000","last_undegraded":"2024-03-21T08:29:08.332492+0000","last_fullsized":"2024-03-21T08:29:08.332492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:54.665866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518258+0000","last_change":"2024-03-21T08:28:36.662952+0000","last_active":"2024-03-21T08:29:06.518258+0000","last_peered":"2024-03-21T08:29:06.518258+0000","last_clean":"2024-03-21T08:29:06.518258+0000","last_became_active":"2024-03-21T08:28:36.662468+0000","last_became_peered":"2024-03-21T08:28:36.662468+0000","last_unstale":"2024-03-21T08:29:06.518258+0000","last_undegraded":"2024-03-21T08:29:06.518258+0000","last_fullsized":"2024-03-21T08:29:06.518258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:40.486998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.521486+0000","last_change":"2024-03-21T08:28:40.693999+0000","last_active":"2024-03-21T08:29:06.521486+0000","last_peered":"2024-03-21T08:29:06.521486+0000","last_clean":"2024-03-21T08:29:06.521486+0000","last_became_active":"2024-03-21T08:28:40.693507+0000","last_became_peered":"2024-03-21T08:28:40.693507+0000","last_unstale":"2024-03-21T08:29:06.521486+0000","last_undegraded":"2024-03-21T08:29:06.521486+0000","last_fullsized":"2024-03-21T08:29:06.521486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:14.159379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.325489+0000","last_change":"2024-03-21T08:28:35.677536+0000","last_active":"2024-03-21T08:29:07.325489+0000","last_peered":"2024-03-21T08:29:07.325489+0000","last_clean":"2024-03-21T08:29:07.325489+0000","last_became_active":"2024-03-21T08:28:35.677100+0000","last_became_peered":"2024-03-21T08:28:35.677100+0000","last_unstale":"2024-03-21T08:29:07.325489+0000","last_undegraded":"2024-03-21T08:29:07.325489+0000","last_fullsized":"2024-03-21T08:29:07.325489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:52:47.275869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.028502+0000","last_change":"2024-03-21T08:28:36.667190+0000","last_active":"2024-03-21T08:29:07.028502+0000","last_peered":"2024-03-21T08:29:07.028502+0000","last_clean":"2024-03-21T08:29:07.028502+0000","last_became_active":"2024-03-21T08:28:36.665336+0000","last_became_peered":"2024-03-21T08:28:36.665336+0000","last_unstale":"2024-03-21T08:29:07.028502+0000","last_undegraded":"2024-03-21T08:29:07.028502+0000","last_fullsized":"2024-03-21T08:29:07.028502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:21.888426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916441+0000","last_change":"2024-03-21T08:28:40.696582+0000","last_active":"2024-03-21T08:29:06.916441+0000","last_peered":"2024-03-21T08:29:06.916441+0000","last_clean":"2024-03-21T08:29:06.916441+0000","last_became_active":"2024-03-21T08:28:40.689583+0000","last_became_peered":"2024-03-21T08:28:40.689583+0000","last_unstale":"2024-03-21T08:29:06.916441+0000","last_undegraded":"2024-03-21T08:29:06.916441+0000","last_fullsized":"2024-03-21T08:29:06.916441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:57.329533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:09.342354+0000","last_change":"2024-03-21T08:28:35.675560+0000","last_active":"2024-03-21T08:29:09.342354+0000","last_peered":"2024-03-21T08:29:09.342354+0000","last_clean":"2024-03-21T08:29:09.342354+0000","last_became_active":"2024-03-21T08:28:35.673496+0000","last_became_peered":"2024-03-21T08:28:35.673496+0000","last_unstale":"2024-03-21T08:29:09.342354+0000","last_undegraded":"2024-03-21T08:29:09.342354+0000","last_fullsized":"2024-03-21T08:29:09.342354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:41:27.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.027450+0000","last_change":"2024-03-21T08:28:36.660869+0000","last_active":"2024-03-21T08:29:07.027450+0000","last_peered":"2024-03-21T08:29:07.027450+0000","last_clean":"2024-03-21T08:29:07.027450+0000","last_became_active":"2024-03-21T08:28:36.659291+0000","last_became_peered":"2024-03-21T08:28:36.659291+0000","last_unstale":"2024-03-21T08:29:07.027450+0000","last_undegraded":"2024-03-21T08:29:07.027450+0000","last_fullsized":"2024-03-21T08:29:07.027450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:06.534214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519649+0000","last_change":"2024-03-21T08:28:40.701351+0000","last_active":"2024-03-21T08:29:06.519649+0000","last_peered":"2024-03-21T08:29:06.519649+0000","last_clean":"2024-03-21T08:29:06.519649+0000","last_became_active":"2024-03-21T08:28:40.700989+0000","last_became_peered":"2024-03-21T08:28:40.700989+0000","last_unstale":"2024-03-21T08:29:06.519649+0000","last_undegraded":"2024-03-21T08:29:06.519649+0000","last_fullsized":"2024-03-21T08:29:06.519649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:35.697013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.335679+0000","last_change":"2024-03-21T08:28:35.685024+0000","last_active":"2024-03-21T08:29:08.335679+0000","last_peered":"2024-03-21T08:29:08.335679+0000","last_clean":"2024-03-21T08:29:08.335679+0000","last_became_active":"2024-03-21T08:28:35.684597+0000","last_became_peered":"2024-03-21T08:28:35.684597+0000","last_unstale":"2024-03-21T08:29:08.335679+0000","last_undegraded":"2024-03-21T08:29:08.335679+0000","last_fullsized":"2024-03-21T08:29:08.335679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:09:46.262782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.876971+0000","last_change":"2024-03-21T08:28:36.664688+0000","last_active":"2024-03-21T08:29:06.876971+0000","last_peered":"2024-03-21T08:29:06.876971+0000","last_clean":"2024-03-21T08:29:06.876971+0000","last_became_active":"2024-03-21T08:28:36.663990+0000","last_became_peered":"2024-03-21T08:28:36.663990+0000","last_unstale":"2024-03-21T08:29:06.876971+0000","last_undegraded":"2024-03-21T08:29:06.876971+0000","last_fullsized":"2024-03-21T08:29:06.876971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:39.656290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.876815+0000","last_change":"2024-03-21T08:28:40.695269+0000","last_active":"2024-03-21T08:29:06.876815+0000","last_peered":"2024-03-21T08:29:06.876815+0000","last_clean":"2024-03-21T08:29:06.876815+0000","last_became_active":"2024-03-21T08:28:40.694820+0000","last_became_peered":"2024-03-21T08:28:40.694820+0000","last_unstale":"2024-03-21T08:29:06.876815+0000","last_undegraded":"2024-03-21T08:29:06.876815+0000","last_fullsized":"2024-03-21T08:29:06.876815+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:47:25.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526439+0000","last_change":"2024-03-21T08:28:35.677377+0000","last_active":"2024-03-21T08:29:06.526439+0000","last_peered":"2024-03-21T08:29:06.526439+0000","last_clean":"2024-03-21T08:29:06.526439+0000","last_became_active":"2024-03-21T08:28:35.673250+0000","last_became_peered":"2024-03-21T08:28:35.673250+0000","last_unstale":"2024-03-21T08:29:06.526439+0000","last_undegraded":"2024-03-21T08:29:06.526439+0000","last_fullsized":"2024-03-21T08:29:06.526439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:28.616353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.522542+0000","last_change":"2024-03-21T08:28:36.667605+0000","last_active":"2024-03-21T08:29:06.522542+0000","last_peered":"2024-03-21T08:29:06.522542+0000","last_clean":"2024-03-21T08:29:06.522542+0000","last_became_active":"2024-03-21T08:28:36.667300+0000","last_became_peered":"2024-03-21T08:28:36.667300+0000","last_unstale":"2024-03-21T08:29:06.522542+0000","last_undegraded":"2024-03-21T08:29:06.522542+0000","last_fullsized":"2024-03-21T08:29:06.522542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:49.268935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.529247+0000","last_change":"2024-03-21T08:28:40.884964+0000","last_active":"2024-03-21T08:29:06.529247+0000","last_peered":"2024-03-21T08:29:06.529247+0000","last_clean":"2024-03-21T08:29:06.529247+0000","last_became_active":"2024-03-21T08:28:40.883445+0000","last_became_peered":"2024-03-21T08:28:40.883445+0000","last_unstale":"2024-03-21T08:29:06.529247+0000","last_undegraded":"2024-03-21T08:29:06.529247+0000","last_fullsized":"2024-03-21T08:29:06.529247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:36:54.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916126+0000","last_change":"2024-03-21T08:28:35.682581+0000","last_active":"2024-03-21T08:29:06.916126+0000","last_peered":"2024-03-21T08:29:06.916126+0000","last_clean":"2024-03-21T08:29:06.916126+0000","last_became_active":"2024-03-21T08:28:35.682150+0000","last_became_peered":"2024-03-21T08:28:35.682150+0000","last_unstale":"2024-03-21T08:29:06.916126+0000","last_undegraded":"2024-03-21T08:29:06.916126+0000","last_fullsized":"2024-03-21T08:29:06.916126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:08.729710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511564+0000","last_change":"2024-03-21T08:28:36.666632+0000","last_active":"2024-03-21T08:29:06.511564+0000","last_peered":"2024-03-21T08:29:06.511564+0000","last_clean":"2024-03-21T08:29:06.511564+0000","last_became_active":"2024-03-21T08:28:36.666305+0000","last_became_peered":"2024-03-21T08:28:36.666305+0000","last_unstale":"2024-03-21T08:29:06.511564+0000","last_undegraded":"2024-03-21T08:29:06.511564+0000","last_fullsized":"2024-03-21T08:29:06.511564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:15.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519052+0000","last_change":"2024-03-21T08:28:40.700382+0000","last_active":"2024-03-21T08:29:06.519052+0000","last_peered":"2024-03-21T08:29:06.519052+0000","last_clean":"2024-03-21T08:29:06.519052+0000","last_became_active":"2024-03-21T08:28:40.699762+0000","last_became_peered":"2024-03-21T08:28:40.699762+0000","last_unstale":"2024-03-21T08:29:06.519052+0000","last_undegraded":"2024-03-21T08:29:06.519052+0000","last_fullsized":"2024-03-21T08:29:06.519052+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:53.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029010+0000","last_change":"2024-03-21T08:28:35.680310+0000","last_active":"2024-03-21T08:29:07.029010+0000","last_peered":"2024-03-21T08:29:07.029010+0000","last_clean":"2024-03-21T08:29:07.029010+0000","last_became_active":"2024-03-21T08:28:35.678634+0000","last_became_peered":"2024-03-21T08:28:35.678634+0000","last_unstale":"2024-03-21T08:29:07.029010+0000","last_undegraded":"2024-03-21T08:29:07.029010+0000","last_fullsized":"2024-03-21T08:29:07.029010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:56:42.872521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511087+0000","last_change":"2024-03-21T08:28:36.669527+0000","last_active":"2024-03-21T08:29:06.511087+0000","last_peered":"2024-03-21T08:29:06.511087+0000","last_clean":"2024-03-21T08:29:06.511087+0000","last_became_active":"2024-03-21T08:28:36.669196+0000","last_became_peered":"2024-03-21T08:28:36.669196+0000","last_unstale":"2024-03-21T08:29:06.511087+0000","last_undegraded":"2024-03-21T08:29:06.511087+0000","last_fullsized":"2024-03-21T08:29:06.511087+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:07:34.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519996+0000","last_change":"2024-03-21T08:28:40.882949+0000","last_active":"2024-03-21T08:29:06.519996+0000","last_peered":"2024-03-21T08:29:06.519996+0000","last_clean":"2024-03-21T08:29:06.519996+0000","last_became_active":"2024-03-21T08:28:40.882432+0000","last_became_peered":"2024-03-21T08:28:40.882432+0000","last_unstale":"2024-03-21T08:29:06.519996+0000","last_undegraded":"2024-03-21T08:29:06.519996+0000","last_fullsized":"2024-03-21T08:29:06.519996+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:51.171365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877181+0000","last_change":"2024-03-21T08:28:36.665164+0000","last_active":"2024-03-21T08:29:06.877181+0000","last_peered":"2024-03-21T08:29:06.877181+0000","last_clean":"2024-03-21T08:29:06.877181+0000","last_became_active":"2024-03-21T08:28:36.664794+0000","last_became_peered":"2024-03-21T08:28:36.664794+0000","last_unstale":"2024-03-21T08:29:06.877181+0000","last_undegraded":"2024-03-21T08:29:06.877181+0000","last_fullsized":"2024-03-21T08:29:06.877181+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:25:39.355633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334636+0000","last_change":"2024-03-21T08:28:35.671754+0000","last_active":"2024-03-21T08:29:08.334636+0000","last_peered":"2024-03-21T08:29:08.334636+0000","last_clean":"2024-03-21T08:29:08.334636+0000","last_became_active":"2024-03-21T08:28:35.671363+0000","last_became_peered":"2024-03-21T08:28:35.671363+0000","last_unstale":"2024-03-21T08:29:08.334636+0000","last_undegraded":"2024-03-21T08:29:08.334636+0000","last_fullsized":"2024-03-21T08:29:08.334636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:22:11.147325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029587+0000","last_change":"2024-03-21T08:28:40.689038+0000","last_active":"2024-03-21T08:29:07.029587+0000","last_peered":"2024-03-21T08:29:07.029587+0000","last_clean":"2024-03-21T08:29:07.029587+0000","last_became_active":"2024-03-21T08:28:40.688538+0000","last_became_peered":"2024-03-21T08:28:40.688538+0000","last_unstale":"2024-03-21T08:29:07.029587+0000","last_undegraded":"2024-03-21T08:29:07.029587+0000","last_fullsized":"2024-03-21T08:29:07.029587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:14.833594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511367+0000","last_change":"2024-03-21T08:28:36.667725+0000","last_active":"2024-03-21T08:29:06.511367+0000","last_peered":"2024-03-21T08:29:06.511367+0000","last_clean":"2024-03-21T08:29:06.511367+0000","last_became_active":"2024-03-21T08:28:36.667303+0000","last_became_peered":"2024-03-21T08:28:36.667303+0000","last_unstale":"2024-03-21T08:29:06.511367+0000","last_undegraded":"2024-03-21T08:29:06.511367+0000","last_fullsized":"2024-03-21T08:29:06.511367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:32:18.990943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:09.339335+0000","last_change":"2024-03-21T08:28:35.673497+0000","last_active":"2024-03-21T08:29:09.339335+0000","last_peered":"2024-03-21T08:29:09.339335+0000","last_clean":"2024-03-21T08:29:09.339335+0000","last_became_active":"2024-03-21T08:28:35.673045+0000","last_became_peered":"2024-03-21T08:28:35.673045+0000","last_unstale":"2024-03-21T08:29:09.339335+0000","last_undegraded":"2024-03-21T08:29:09.339335+0000","last_fullsized":"2024-03-21T08:29:09.339335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:23.972672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.027114+0000","last_change":"2024-03-21T08:26:58.139094+0000","last_active":"2024-03-21T08:29:07.027114+0000","last_peered":"2024-03-21T08:29:07.027114+0000","last_clean":"2024-03-21T08:29:07.027114+0000","last_became_active":"2024-03-21T08:26:58.137648+0000","last_became_peered":"2024-03-21T08:26:58.137648+0000","last_unstale":"2024-03-21T08:29:07.027114+0000","last_undegraded":"2024-03-21T08:29:07.027114+0000","last_fullsized":"2024-03-21T08:29:07.027114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_clean_scrub_stamp":"2024-03-21T08:22:17.977314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:57.950111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878595+0000","last_change":"2024-03-21T08:28:40.887865+0000","last_active":"2024-03-21T08:29:06.878595+0000","last_peered":"2024-03-21T08:29:06.878595+0000","last_clean":"2024-03-21T08:29:06.878595+0000","last_became_active":"2024-03-21T08:28:40.885538+0000","last_became_peered":"2024-03-21T08:28:40.885538+0000","last_unstale":"2024-03-21T08:29:06.878595+0000","last_undegraded":"2024-03-21T08:29:06.878595+0000","last_fullsized":"2024-03-21T08:29:06.878595+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:26.532005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518547+0000","last_change":"2024-03-21T08:28:36.668600+0000","last_active":"2024-03-21T08:29:06.518547+0000","last_peered":"2024-03-21T08:29:06.518547+0000","last_clean":"2024-03-21T08:29:06.518547+0000","last_became_active":"2024-03-21T08:28:36.668188+0000","last_became_peered":"2024-03-21T08:28:36.668188+0000","last_unstale":"2024-03-21T08:29:06.518547+0000","last_undegraded":"2024-03-21T08:29:06.518547+0000","last_fullsized":"2024-03-21T08:29:06.518547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:13.536003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.334248+0000","last_change":"2024-03-21T08:28:35.674594+0000","last_active":"2024-03-21T08:29:08.334248+0000","last_peered":"2024-03-21T08:29:08.334248+0000","last_clean":"2024-03-21T08:29:08.334248+0000","last_became_active":"2024-03-21T08:28:35.674257+0000","last_became_peered":"2024-03-21T08:28:35.674257+0000","last_unstale":"2024-03-21T08:29:08.334248+0000","last_undegraded":"2024-03-21T08:29:08.334248+0000","last_fullsized":"2024-03-21T08:29:08.334248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:14:51.585631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.032622+0000","last_change":"2024-03-21T08:29:07.032622+0000","last_active":"2024-03-21T08:29:07.032622+0000","last_peered":"2024-03-21T08:29:07.032622+0000","last_clean":"2024-03-21T08:29:07.032622+0000","last_became_active":"2024-03-21T08:29:07.030884+0000","last_became_peered":"2024-03-21T08:29:07.030884+0000","last_unstale":"2024-03-21T08:29:07.032622+0000","last_undegraded":"2024-03-21T08:29:07.032622+0000","last_fullsized":"2024-03-21T08:29:07.032622+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:47.615776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.527720+0000","last_change":"2024-03-21T08:28:35.672948+0000","last_active":"2024-03-21T08:29:06.527720+0000","last_peered":"2024-03-21T08:29:06.527720+0000","last_clean":"2024-03-21T08:29:06.527720+0000","last_became_active":"2024-03-21T08:28:35.671576+0000","last_became_peered":"2024-03-21T08:28:35.671576+0000","last_unstale":"2024-03-21T08:29:06.527720+0000","last_undegraded":"2024-03-21T08:29:06.527720+0000","last_fullsized":"2024-03-21T08:29:06.527720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:36:25.620392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518463+0000","last_change":"2024-03-21T08:28:36.671173+0000","last_active":"2024-03-21T08:29:06.518463+0000","last_peered":"2024-03-21T08:29:06.518463+0000","last_clean":"2024-03-21T08:29:06.518463+0000","last_became_active":"2024-03-21T08:28:36.666264+0000","last_became_peered":"2024-03-21T08:28:36.666264+0000","last_unstale":"2024-03-21T08:29:06.518463+0000","last_undegraded":"2024-03-21T08:29:06.518463+0000","last_fullsized":"2024-03-21T08:29:06.518463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:35.081487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511119+0000","last_change":"2024-03-21T08:28:40.883801+0000","last_active":"2024-03-21T08:29:06.511119+0000","last_peered":"2024-03-21T08:29:06.511119+0000","last_clean":"2024-03-21T08:29:06.511119+0000","last_became_active":"2024-03-21T08:28:40.882556+0000","last_became_peered":"2024-03-21T08:28:40.882556+0000","last_unstale":"2024-03-21T08:29:06.511119+0000","last_undegraded":"2024-03-21T08:29:06.511119+0000","last_fullsized":"2024-03-21T08:29:06.511119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:24.211625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.335741+0000","last_change":"2024-03-21T08:28:35.668657+0000","last_active":"2024-03-21T08:29:08.335741+0000","last_peered":"2024-03-21T08:29:08.335741+0000","last_clean":"2024-03-21T08:29:08.335741+0000","last_became_active":"2024-03-21T08:28:35.668284+0000","last_became_peered":"2024-03-21T08:28:35.668284+0000","last_unstale":"2024-03-21T08:29:08.335741+0000","last_undegraded":"2024-03-21T08:29:08.335741+0000","last_fullsized":"2024-03-21T08:29:08.335741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:26:07.791388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512373+0000","last_change":"2024-03-21T08:28:36.652383+0000","last_active":"2024-03-21T08:29:06.512373+0000","last_peered":"2024-03-21T08:29:06.512373+0000","last_clean":"2024-03-21T08:29:06.512373+0000","last_became_active":"2024-03-21T08:28:36.652046+0000","last_became_peered":"2024-03-21T08:28:36.652046+0000","last_unstale":"2024-03-21T08:29:06.512373+0000","last_undegraded":"2024-03-21T08:29:06.512373+0000","last_fullsized":"2024-03-21T08:29:06.512373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:11:08.699391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512994+0000","last_change":"2024-03-21T08:28:40.686122+0000","last_active":"2024-03-21T08:29:06.512994+0000","last_peered":"2024-03-21T08:29:06.512994+0000","last_clean":"2024-03-21T08:29:06.512994+0000","last_became_active":"2024-03-21T08:28:40.685736+0000","last_became_peered":"2024-03-21T08:28:40.685736+0000","last_unstale":"2024-03-21T08:29:06.512994+0000","last_undegraded":"2024-03-21T08:29:06.512994+0000","last_fullsized":"2024-03-21T08:29:06.512994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:52.941896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.317246+0000","last_change":"2024-03-21T08:28:35.675197+0000","last_active":"2024-03-21T08:29:07.317246+0000","last_peered":"2024-03-21T08:29:07.317246+0000","last_clean":"2024-03-21T08:29:07.317246+0000","last_became_active":"2024-03-21T08:28:35.674688+0000","last_became_peered":"2024-03-21T08:28:35.674688+0000","last_unstale":"2024-03-21T08:29:07.317246+0000","last_undegraded":"2024-03-21T08:29:07.317246+0000","last_fullsized":"2024-03-21T08:29:07.317246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:01:11.489294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526862+0000","last_change":"2024-03-21T08:28:36.668160+0000","last_active":"2024-03-21T08:29:06.526862+0000","last_peered":"2024-03-21T08:29:06.526862+0000","last_clean":"2024-03-21T08:29:06.526862+0000","last_became_active":"2024-03-21T08:28:36.665152+0000","last_became_peered":"2024-03-21T08:28:36.665152+0000","last_unstale":"2024-03-21T08:29:06.526862+0000","last_undegraded":"2024-03-21T08:29:06.526862+0000","last_fullsized":"2024-03-21T08:29:06.526862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:43:41.803822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.781105+0000","last_change":"2024-03-21T08:28:40.884365+0000","last_active":"2024-03-21T08:29:06.781105+0000","last_peered":"2024-03-21T08:29:06.781105+0000","last_clean":"2024-03-21T08:29:06.781105+0000","last_became_active":"2024-03-21T08:28:40.883654+0000","last_became_peered":"2024-03-21T08:28:40.883654+0000","last_unstale":"2024-03-21T08:29:06.781105+0000","last_undegraded":"2024-03-21T08:29:06.781105+0000","last_fullsized":"2024-03-21T08:29:06.781105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:46:43.888121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.322708+0000","last_change":"2024-03-21T08:28:35.682548+0000","last_active":"2024-03-21T08:29:07.322708+0000","last_peered":"2024-03-21T08:29:07.322708+0000","last_clean":"2024-03-21T08:29:07.322708+0000","last_became_active":"2024-03-21T08:28:35.682133+0000","last_became_peered":"2024-03-21T08:28:35.682133+0000","last_unstale":"2024-03-21T08:29:07.322708+0000","last_undegraded":"2024-03-21T08:29:07.322708+0000","last_fullsized":"2024-03-21T08:29:07.322708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:59:13.935159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526167+0000","last_change":"2024-03-21T08:28:36.653797+0000","last_active":"2024-03-21T08:29:06.526167+0000","last_peered":"2024-03-21T08:29:06.526167+0000","last_clean":"2024-03-21T08:29:06.526167+0000","last_became_active":"2024-03-21T08:28:36.651872+0000","last_became_peered":"2024-03-21T08:28:36.651872+0000","last_unstale":"2024-03-21T08:29:06.526167+0000","last_undegraded":"2024-03-21T08:29:06.526167+0000","last_fullsized":"2024-03-21T08:29:06.526167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:09.467573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.916574+0000","last_change":"2024-03-21T08:28:40.692996+0000","last_active":"2024-03-21T08:29:06.916574+0000","last_peered":"2024-03-21T08:29:06.916574+0000","last_clean":"2024-03-21T08:29:06.916574+0000","last_became_active":"2024-03-21T08:28:40.687598+0000","last_became_peered":"2024-03-21T08:28:40.687598+0000","last_unstale":"2024-03-21T08:29:06.916574+0000","last_undegraded":"2024-03-21T08:29:06.916574+0000","last_fullsized":"2024-03-21T08:29:06.916574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:09.557272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.323477+0000","last_change":"2024-03-21T08:28:35.672567+0000","last_active":"2024-03-21T08:29:07.323477+0000","last_peered":"2024-03-21T08:29:07.323477+0000","last_clean":"2024-03-21T08:29:07.323477+0000","last_became_active":"2024-03-21T08:28:35.671849+0000","last_became_peered":"2024-03-21T08:28:35.671849+0000","last_unstale":"2024-03-21T08:29:07.323477+0000","last_undegraded":"2024-03-21T08:29:07.323477+0000","last_fullsized":"2024-03-21T08:29:07.323477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:13.050227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.520556+0000","last_change":"2024-03-21T08:28:36.664110+0000","last_active":"2024-03-21T08:29:06.520556+0000","last_peered":"2024-03-21T08:29:06.520556+0000","last_clean":"2024-03-21T08:29:06.520556+0000","last_became_active":"2024-03-21T08:28:36.663297+0000","last_became_peered":"2024-03-21T08:28:36.663297+0000","last_unstale":"2024-03-21T08:29:06.520556+0000","last_undegraded":"2024-03-21T08:29:06.520556+0000","last_fullsized":"2024-03-21T08:29:06.520556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:55.450399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.779846+0000","last_change":"2024-03-21T08:28:40.682371+0000","last_active":"2024-03-21T08:29:06.779846+0000","last_peered":"2024-03-21T08:29:06.779846+0000","last_clean":"2024-03-21T08:29:06.779846+0000","last_became_active":"2024-03-21T08:28:40.681786+0000","last_became_peered":"2024-03-21T08:28:40.681786+0000","last_unstale":"2024-03-21T08:29:06.779846+0000","last_undegraded":"2024-03-21T08:29:06.779846+0000","last_fullsized":"2024-03-21T08:29:06.779846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:15:38.130807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514277+0000","last_change":"2024-03-21T08:28:35.685312+0000","last_active":"2024-03-21T08:29:06.514277+0000","last_peered":"2024-03-21T08:29:06.514277+0000","last_clean":"2024-03-21T08:29:06.514277+0000","last_became_active":"2024-03-21T08:28:35.684327+0000","last_became_peered":"2024-03-21T08:28:35.684327+0000","last_unstale":"2024-03-21T08:29:06.514277+0000","last_undegraded":"2024-03-21T08:29:06.514277+0000","last_fullsized":"2024-03-21T08:29:06.514277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:12.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513310+0000","last_change":"2024-03-21T08:28:36.671186+0000","last_active":"2024-03-21T08:29:06.513310+0000","last_peered":"2024-03-21T08:29:06.513310+0000","last_clean":"2024-03-21T08:29:06.513310+0000","last_became_active":"2024-03-21T08:28:36.662726+0000","last_became_peered":"2024-03-21T08:28:36.662726+0000","last_unstale":"2024-03-21T08:29:06.513310+0000","last_undegraded":"2024-03-21T08:29:06.513310+0000","last_fullsized":"2024-03-21T08:29:06.513310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:29.129014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.780982+0000","last_change":"2024-03-21T08:28:40.885203+0000","last_active":"2024-03-21T08:29:06.780982+0000","last_peered":"2024-03-21T08:29:06.780982+0000","last_clean":"2024-03-21T08:29:06.780982+0000","last_became_active":"2024-03-21T08:28:40.883810+0000","last_became_peered":"2024-03-21T08:28:40.883810+0000","last_unstale":"2024-03-21T08:29:06.780982+0000","last_undegraded":"2024-03-21T08:29:06.780982+0000","last_fullsized":"2024-03-21T08:29:06.780982+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:12:58.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.324911+0000","last_change":"2024-03-21T08:28:35.690494+0000","last_active":"2024-03-21T08:29:07.324911+0000","last_peered":"2024-03-21T08:29:07.324911+0000","last_clean":"2024-03-21T08:29:07.324911+0000","last_became_active":"2024-03-21T08:28:35.690071+0000","last_became_peered":"2024-03-21T08:28:35.690071+0000","last_unstale":"2024-03-21T08:29:07.324911+0000","last_undegraded":"2024-03-21T08:29:07.324911+0000","last_fullsized":"2024-03-21T08:29:07.324911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:04:46.713006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.028651+0000","last_change":"2024-03-21T08:28:36.665135+0000","last_active":"2024-03-21T08:29:07.028651+0000","last_peered":"2024-03-21T08:29:07.028651+0000","last_clean":"2024-03-21T08:29:07.028651+0000","last_became_active":"2024-03-21T08:28:36.663700+0000","last_became_peered":"2024-03-21T08:28:36.663700+0000","last_unstale":"2024-03-21T08:29:07.028651+0000","last_undegraded":"2024-03-21T08:29:07.028651+0000","last_fullsized":"2024-03-21T08:29:07.028651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:50:00.604958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.517790+0000","last_change":"2024-03-21T08:28:40.696076+0000","last_active":"2024-03-21T08:29:06.517790+0000","last_peered":"2024-03-21T08:29:06.517790+0000","last_clean":"2024-03-21T08:29:06.517790+0000","last_became_active":"2024-03-21T08:28:40.695556+0000","last_became_peered":"2024-03-21T08:28:40.695556+0000","last_unstale":"2024-03-21T08:29:06.517790+0000","last_undegraded":"2024-03-21T08:29:06.517790+0000","last_fullsized":"2024-03-21T08:29:06.517790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:42.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:08.335341+0000","last_change":"2024-03-21T08:28:35.685545+0000","last_active":"2024-03-21T08:29:08.335341+0000","last_peered":"2024-03-21T08:29:08.335341+0000","last_clean":"2024-03-21T08:29:08.335341+0000","last_became_active":"2024-03-21T08:28:35.684795+0000","last_became_peered":"2024-03-21T08:28:35.684795+0000","last_unstale":"2024-03-21T08:29:08.335341+0000","last_undegraded":"2024-03-21T08:29:08.335341+0000","last_fullsized":"2024-03-21T08:29:08.335341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:40.308014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029794+0000","last_change":"2024-03-21T08:28:36.656276+0000","last_active":"2024-03-21T08:29:07.029794+0000","last_peered":"2024-03-21T08:29:07.029794+0000","last_clean":"2024-03-21T08:29:07.029794+0000","last_became_active":"2024-03-21T08:28:36.655871+0000","last_became_peered":"2024-03-21T08:28:36.655871+0000","last_unstale":"2024-03-21T08:29:07.029794+0000","last_undegraded":"2024-03-21T08:29:07.029794+0000","last_fullsized":"2024-03-21T08:29:07.029794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:22.362165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.783155+0000","last_change":"2024-03-21T08:29:06.783155+0000","last_active":"2024-03-21T08:29:06.783155+0000","last_peered":"2024-03-21T08:29:06.783155+0000","last_clean":"2024-03-21T08:29:06.783155+0000","last_became_active":"2024-03-21T08:29:06.782546+0000","last_became_peered":"2024-03-21T08:29:06.782546+0000","last_unstale":"2024-03-21T08:29:06.783155+0000","last_undegraded":"2024-03-21T08:29:06.783155+0000","last_fullsized":"2024-03-21T08:29:06.783155+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:16.489888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.878141+0000","last_change":"2024-03-21T08:28:35.672783+0000","last_active":"2024-03-21T08:29:06.878141+0000","last_peered":"2024-03-21T08:29:06.878141+0000","last_clean":"2024-03-21T08:29:06.878141+0000","last_became_active":"2024-03-21T08:28:35.672161+0000","last_became_peered":"2024-03-21T08:28:35.672161+0000","last_unstale":"2024-03-21T08:29:06.878141+0000","last_undegraded":"2024-03-21T08:29:06.878141+0000","last_fullsized":"2024-03-21T08:29:06.878141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:37.180625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.028739+0000","last_change":"2024-03-21T08:28:36.659447+0000","last_active":"2024-03-21T08:29:07.028739+0000","last_peered":"2024-03-21T08:29:07.028739+0000","last_clean":"2024-03-21T08:29:07.028739+0000","last_became_active":"2024-03-21T08:28:36.657773+0000","last_became_peered":"2024-03-21T08:28:36.657773+0000","last_unstale":"2024-03-21T08:29:07.028739+0000","last_undegraded":"2024-03-21T08:29:07.028739+0000","last_fullsized":"2024-03-21T08:29:07.028739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:05.324498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.876555+0000","last_change":"2024-03-21T08:28:40.679806+0000","last_active":"2024-03-21T08:29:06.876555+0000","last_peered":"2024-03-21T08:29:06.876555+0000","last_clean":"2024-03-21T08:29:06.876555+0000","last_became_active":"2024-03-21T08:28:40.679329+0000","last_became_peered":"2024-03-21T08:28:40.679329+0000","last_unstale":"2024-03-21T08:29:06.876555+0000","last_undegraded":"2024-03-21T08:29:06.876555+0000","last_fullsized":"2024-03-21T08:29:06.876555+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:30:58.297581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519294+0000","last_change":"2024-03-21T08:28:35.690117+0000","last_active":"2024-03-21T08:29:06.519294+0000","last_peered":"2024-03-21T08:29:06.519294+0000","last_clean":"2024-03-21T08:29:06.519294+0000","last_became_active":"2024-03-21T08:28:35.689530+0000","last_became_peered":"2024-03-21T08:28:35.689530+0000","last_unstale":"2024-03-21T08:29:06.519294+0000","last_undegraded":"2024-03-21T08:29:06.519294+0000","last_fullsized":"2024-03-21T08:29:06.519294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:24:02.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.029422+0000","last_change":"2024-03-21T08:28:36.649194+0000","last_active":"2024-03-21T08:29:07.029422+0000","last_peered":"2024-03-21T08:29:07.029422+0000","last_clean":"2024-03-21T08:29:07.029422+0000","last_became_active":"2024-03-21T08:28:36.648663+0000","last_became_peered":"2024-03-21T08:28:36.648663+0000","last_unstale":"2024-03-21T08:29:07.029422+0000","last_undegraded":"2024-03-21T08:29:07.029422+0000","last_fullsized":"2024-03-21T08:29:07.029422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:16.296886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.521057+0000","last_change":"2024-03-21T08:28:36.664473+0000","last_active":"2024-03-21T08:29:06.521057+0000","last_peered":"2024-03-21T08:29:06.521057+0000","last_clean":"2024-03-21T08:29:06.521057+0000","last_became_active":"2024-03-21T08:28:36.663785+0000","last_became_peered":"2024-03-21T08:28:36.663785+0000","last_unstale":"2024-03-21T08:29:06.521057+0000","last_undegraded":"2024-03-21T08:29:06.521057+0000","last_fullsized":"2024-03-21T08:29:06.521057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:00.409224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514656+0000","last_change":"2024-03-21T08:28:36.668372+0000","last_active":"2024-03-21T08:29:06.514656+0000","last_peered":"2024-03-21T08:29:06.514656+0000","last_clean":"2024-03-21T08:29:06.514656+0000","last_became_active":"2024-03-21T08:28:36.666227+0000","last_became_peered":"2024-03-21T08:28:36.666227+0000","last_unstale":"2024-03-21T08:29:06.514656+0000","last_undegraded":"2024-03-21T08:29:06.514656+0000","last_fullsized":"2024-03-21T08:29:06.514656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:40:56.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.513642+0000","last_change":"2024-03-21T08:28:36.670770+0000","last_active":"2024-03-21T08:29:06.513642+0000","last_peered":"2024-03-21T08:29:06.513642+0000","last_clean":"2024-03-21T08:29:06.513642+0000","last_became_active":"2024-03-21T08:28:36.670467+0000","last_became_peered":"2024-03-21T08:28:36.670467+0000","last_unstale":"2024-03-21T08:29:06.513642+0000","last_undegraded":"2024-03-21T08:29:06.513642+0000","last_fullsized":"2024-03-21T08:29:06.513642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:48.075189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.526197+0000","last_change":"2024-03-21T08:28:36.669395+0000","last_active":"2024-03-21T08:29:06.526197+0000","last_peered":"2024-03-21T08:29:06.526197+0000","last_clean":"2024-03-21T08:29:06.526197+0000","last_became_active":"2024-03-21T08:28:36.668788+0000","last_became_peered":"2024-03-21T08:28:36.668788+0000","last_unstale":"2024-03-21T08:29:06.526197+0000","last_undegraded":"2024-03-21T08:29:06.526197+0000","last_fullsized":"2024-03-21T08:29:06.526197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:03.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.518162+0000","last_change":"2024-03-21T08:28:36.669305+0000","last_active":"2024-03-21T08:29:06.518162+0000","last_peered":"2024-03-21T08:29:06.518162+0000","last_clean":"2024-03-21T08:29:06.518162+0000","last_became_active":"2024-03-21T08:28:36.668919+0000","last_became_peered":"2024-03-21T08:28:36.668919+0000","last_unstale":"2024-03-21T08:29:06.518162+0000","last_undegraded":"2024-03-21T08:29:06.518162+0000","last_fullsized":"2024-03-21T08:29:06.518162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:29:19.440425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.522611+0000","last_change":"2024-03-21T08:28:36.666836+0000","last_active":"2024-03-21T08:29:06.522611+0000","last_peered":"2024-03-21T08:29:06.522611+0000","last_clean":"2024-03-21T08:29:06.522611+0000","last_became_active":"2024-03-21T08:28:36.666538+0000","last_became_peered":"2024-03-21T08:28:36.666538+0000","last_unstale":"2024-03-21T08:29:06.522611+0000","last_undegraded":"2024-03-21T08:29:06.522611+0000","last_fullsized":"2024-03-21T08:29:06.522611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:43.353269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.514660+0000","last_change":"2024-03-21T08:28:36.667823+0000","last_active":"2024-03-21T08:29:06.514660+0000","last_peered":"2024-03-21T08:29:06.514660+0000","last_clean":"2024-03-21T08:29:06.514660+0000","last_became_active":"2024-03-21T08:28:36.666096+0000","last_became_peered":"2024-03-21T08:28:36.666096+0000","last_unstale":"2024-03-21T08:29:06.514660+0000","last_undegraded":"2024-03-21T08:29:06.514660+0000","last_fullsized":"2024-03-21T08:29:06.514660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:58.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.512865+0000","last_change":"2024-03-21T08:28:36.672164+0000","last_active":"2024-03-21T08:29:06.512865+0000","last_peered":"2024-03-21T08:29:06.512865+0000","last_clean":"2024-03-21T08:29:06.512865+0000","last_became_active":"2024-03-21T08:28:36.671812+0000","last_became_peered":"2024-03-21T08:28:36.671812+0000","last_unstale":"2024-03-21T08:29:06.512865+0000","last_undegraded":"2024-03-21T08:29:06.512865+0000","last_fullsized":"2024-03-21T08:29:06.512865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:24:19.687108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.877000+0000","last_change":"2024-03-21T08:28:36.675056+0000","last_active":"2024-03-21T08:29:06.877000+0000","last_peered":"2024-03-21T08:29:06.877000+0000","last_clean":"2024-03-21T08:29:06.877000+0000","last_became_active":"2024-03-21T08:28:36.674586+0000","last_became_peered":"2024-03-21T08:28:36.674586+0000","last_unstale":"2024-03-21T08:29:06.877000+0000","last_undegraded":"2024-03-21T08:29:06.877000+0000","last_fullsized":"2024-03-21T08:29:06.877000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:04:51.207066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.511171+0000","last_change":"2024-03-21T08:28:36.666370+0000","last_active":"2024-03-21T08:29:06.511171+0000","last_peered":"2024-03-21T08:29:06.511171+0000","last_clean":"2024-03-21T08:29:06.511171+0000","last_became_active":"2024-03-21T08:28:36.665950+0000","last_became_peered":"2024-03-21T08:28:36.665950+0000","last_unstale":"2024-03-21T08:29:06.511171+0000","last_undegraded":"2024-03-21T08:29:06.511171+0000","last_fullsized":"2024-03-21T08:29:06.511171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:49:31.945890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:07.032862+0000","last_change":"2024-03-21T08:29:07.032862+0000","last_active":"2024-03-21T08:29:07.032862+0000","last_peered":"2024-03-21T08:29:07.032862+0000","last_clean":"2024-03-21T08:29:07.032862+0000","last_became_active":"2024-03-21T08:29:07.031784+0000","last_became_peered":"2024-03-21T08:29:07.031784+0000","last_unstale":"2024-03-21T08:29:07.032862+0000","last_undegraded":"2024-03-21T08:29:07.032862+0000","last_fullsized":"2024-03-21T08:29:07.032862+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:34.037495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.519247+0000","last_change":"2024-03-21T08:28:36.660812+0000","last_active":"2024-03-21T08:29:06.519247+0000","last_peered":"2024-03-21T08:29:06.519247+0000","last_clean":"2024-03-21T08:29:06.519247+0000","last_became_active":"2024-03-21T08:28:36.660044+0000","last_became_peered":"2024-03-21T08:28:36.660044+0000","last_unstale":"2024-03-21T08:29:06.519247+0000","last_undegraded":"2024-03-21T08:29:06.519247+0000","last_fullsized":"2024-03-21T08:29:06.519247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:31.598678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T08:29:06.521779+0000","last_change":"2024-03-21T08:28:36.669441+0000","last_active":"2024-03-21T08:29:06.521779+0000","last_peered":"2024-03-21T08:29:06.521779+0000","last_clean":"2024-03-21T08:29:06.521779+0000","last_became_active":"2024-03-21T08:28:36.668668+0000","last_became_peered":"2024-03-21T08:28:36.668668+0000","last_unstale":"2024-03-21T08:29:06.521779+0000","last_undegraded":"2024-03-21T08:29:06.521779+0000","last_fullsized":"2024-03-21T08:29:06.521779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:04.828297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7843,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":129,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":83697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285591,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":510926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547228,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1687552,"data_stored":1106132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710756,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":511162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972395,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":511082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266739,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1658880,"data_stored":1102504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":510434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":550815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":511822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":535226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":510628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":510988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1106675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":40471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":29513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:29:13.885 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-21T08:29:13.886 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T08:29:13.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T08:29:13.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:13 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3382867919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:14.376 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:14.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:14 smithi003 ceph-mon[31675]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:29:14.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:14 smithi003 ceph-mon[31675]: pgmap v388: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-03-21T08:29:14.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:14 smithi003 ceph-mon[31675]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:29:14.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:14 smithi084 ceph-mon[35477]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:29:14.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:14 smithi084 ceph-mon[35477]: pgmap v388: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-03-21T08:29:14.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:14 smithi084 ceph-mon[35477]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:29:14.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:14 smithi171 ceph-mon[35388]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:29:14.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:14 smithi171 ceph-mon[35388]: pgmap v388: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-03-21T08:29:14.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:14 smithi171 ceph-mon[35388]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 -- 172.21.15.3:0/2792741640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 msgr2=0x7f0f20109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 --2- 172.21.15.3:0/2792741640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 0x7f0f20109e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0f080099c0 tx=0x7f0f08009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 -- 172.21.15.3:0/2792741640 shutdown_connections 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 --2- 172.21.15.3:0/2792741640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f2010a3b0 0x7f0f20110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 --2- 172.21.15.3:0/2792741640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f20108810 0x7f0f20108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 --2- 172.21.15.3:0/2792741640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 0x7f0f20109e70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 -- 172.21.15.3:0/2792741640 >> 172.21.15.3:0/2792741640 conn(0x7f0f200777e0 msgr2=0x7f0f20079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 -- 172.21.15.3:0/2792741640 shutdown_connections 2024-03-21T08:29:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.712+0000 7f0f26a00700 1 -- 172.21.15.3:0/2792741640 wait complete. 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.714+0000 7f0f26a00700 1 Processor -- start 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.714+0000 7f0f26a00700 1 -- start start 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f26a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20108810 0x7f0f2019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f26a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 0x7f0f2019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f26a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f2010a3b0 0x7f0f201a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f26a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201142f0 con 0x7f0f20108810 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f26a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20114170 con 0x7f0f20109a10 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f26a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20114470 con 0x7f0f2010a3b0 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f1fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20108810 0x7f0f2019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f1fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20108810 0x7f0f2019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51112/0 (socket says 172.21.15.3:51112) 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f1fe00700 1 -- 172.21.15.3:0/785132181 learned_addr learned my addr 172.21.15.3:0/785132181 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f17400700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 0x7f0f2019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:16.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f24a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f2010a3b0 0x7f0f201a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:16.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f1fe00700 1 -- 172.21.15.3:0/785132181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f2010a3b0 msgr2=0x7f0f201a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.715+0000 7f0f1fe00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f2010a3b0 0x7f0f201a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f1fe00700 1 -- 172.21.15.3:0/785132181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 msgr2=0x7f0f2019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f1fe00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 0x7f0f2019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f1fe00700 1 -- 172.21.15.3:0/785132181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f201a6530 con 0x7f0f20108810 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f24a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f2010a3b0 0x7f0f201a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f1fe00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20108810 0x7f0f2019e8d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000d9b0 tx=0x7f0f1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f1d600700 1 -- 172.21.15.3:0/785132181 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10017070 con 0x7f0f20108810 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f201a6820 con 0x7f0f20108810 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f1d600700 1 -- 172.21.15.3:0/785132181 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f10009de0 con 0x7f0f20108810 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f201a6d60 con 0x7f0f20108810 2024-03-21T08:29:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.716+0000 7f0f1d600700 1 -- 172.21.15.3:0/785132181 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10020060 con 0x7f0f20108810 2024-03-21T08:29:16.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.718+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f200506a0 con 0x7f0f20108810 2024-03-21T08:29:16.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.720+0000 7f0f1d600700 1 -- 172.21.15.3:0/785132181 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f0f10009930 con 0x7f0f20108810 2024-03-21T08:29:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.721+0000 7f0f1d600700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0f040a30f0 0x7f0f040a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.722+0000 7f0f1d600700 1 -- 172.21.15.3:0/785132181 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f0f100cf550 con 0x7f0f20108810 2024-03-21T08:29:16.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.722+0000 7f0f17400700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0f040a30f0 0x7f0f040a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:16.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.723+0000 7f0f17400700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0f040a30f0 0x7f0f040a55b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f08000c00 tx=0x7f0f08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:16.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:16.726+0000 7f0f1d600700 1 -- 172.21.15.3:0/785132181 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0f1009be60 con 0x7f0f20108810 2024-03-21T08:29:16.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:16 smithi084 ceph-mon[35477]: pgmap v389: 129 pgs: 129 active+clean; 620 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-21T08:29:16.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:16 smithi171 ceph-mon[35388]: pgmap v389: 129 pgs: 129 active+clean; 620 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-21T08:29:16.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:16 smithi003 ceph-mon[31675]: pgmap v389: 129 pgs: 129 active+clean; 620 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-21T08:29:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.039+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0f2002db90 con 0x7f0f20108810 2024-03-21T08:29:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.043+0000 7f0f1d600700 1 -- 172.21.15.3:0/785132181 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0f2002db90 con 0x7f0f20108810 2024-03-21T08:29:17.045 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-03-21T08:29:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQDd7/tl1n11AhAAk9eh7lmpzFOlk7gZK3/FQg== 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0f040a30f0 msgr2=0x7f0f040a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0f040a30f0 0x7f0f040a55b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f08000c00 tx=0x7f0f08005fb0 comp rx=0 tx=0).stop 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20108810 msgr2=0x7f0f2019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20108810 0x7f0f2019e8d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000d9b0 tx=0x7f0f1000dcc0 comp rx=0 tx=0).stop 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 shutdown_connections 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0f040a30f0 0x7f0f040a55b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f20108810 0x7f0f2019e8d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f2010a3b0 0x7f0f201a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 --2- 172.21.15.3:0/785132181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f20109a10 0x7f0f2019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 >> 172.21.15.3:0/785132181 conn(0x7f0f200777e0 msgr2=0x7f0f2010ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 shutdown_connections 2024-03-21T08:29:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:17.046+0000 7f0f26a00700 1 -- 172.21.15.3:0/785132181 wait complete. 2024-03-21T08:29:17.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:17 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/785132181' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:29:17.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:17 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/785132181' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T08:29:17.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:17 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/785132181' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:29:17.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:17 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/785132181' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T08:29:17.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:17 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/785132181' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:29:17.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:17 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/785132181' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T08:29:18.021 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:29:18.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T08:29:18.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T08:29:18.189 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T08:29:18.190 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T08:29:18.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T08:29:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:19.149 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:19 smithi003 ceph-mon[31675]: pgmap v390: 129 pgs: 129 active+clean; 620 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-21T08:29:19.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:19 smithi084 ceph-mon[35477]: pgmap v390: 129 pgs: 129 active+clean; 620 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-21T08:29:19.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:19 smithi171 ceph-mon[35388]: pgmap v390: 129 pgs: 129 active+clean; 620 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-21T08:29:20.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 -- 172.21.15.3:0/3009421694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1096c0 msgr2=0x7f21dc111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 --2- 172.21.15.3:0/3009421694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1096c0 0x7f21dc111290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f21d40099c0 tx=0x7f21d4009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 -- 172.21.15.3:0/3009421694 shutdown_connections 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 --2- 172.21.15.3:0/3009421694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1096c0 0x7f21dc111290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 --2- 172.21.15.3:0/3009421694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21dc1045e0 0x7f21dc1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 --2- 172.21.15.3:0/3009421694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21dc104f90 0x7f21dc108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 -- 172.21.15.3:0/3009421694 >> 172.21.15.3:0/3009421694 conn(0x7f21dc0ffe70 msgr2=0x7f21dc102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 -- 172.21.15.3:0/3009421694 shutdown_connections 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.933+0000 7f21e2600700 1 -- 172.21.15.3:0/3009421694 wait complete. 2024-03-21T08:29:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.935+0000 7f21e2600700 1 Processor -- start 2024-03-21T08:29:20.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.935+0000 7f21e2600700 1 -- start start 2024-03-21T08:29:20.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.935+0000 7f21e2600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1045e0 0x7f21dc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:20.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.935+0000 7f21e2600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21dc104f90 0x7f21dc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:20.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.935+0000 7f21e2600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21dc1096c0 0x7f21dc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21e2600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc1145b0 con 0x7f21dc1045e0 2024-03-21T08:29:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21e2600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc114430 con 0x7f21dc1096c0 2024-03-21T08:29:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21e2600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc114730 con 0x7f21dc104f90 2024-03-21T08:29:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1045e0 0x7f21dc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1045e0 0x7f21dc1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41582/0 (socket says 172.21.15.3:41582) 2024-03-21T08:29:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 -- 172.21.15.3:0/867441984 learned_addr learned my addr 172.21.15.3:0/867441984 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21d2a00700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21dc104f90 0x7f21dc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21dbe00700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21dc1096c0 0x7f21dc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 -- 172.21.15.3:0/867441984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21dc104f90 msgr2=0x7f21dc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21dc104f90 0x7f21dc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 -- 172.21.15.3:0/867441984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21dc1096c0 msgr2=0x7f21dc19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21dc1096c0 0x7f21dc19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.936+0000 7f21db400700 1 -- 172.21.15.3:0/867441984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc19afd0 con 0x7f21dc1045e0 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.937+0000 7f21db400700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1045e0 0x7f21dc1a0670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0060b0 tx=0x7f21cc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.937+0000 7f21d8c00700 1 -- 172.21.15.3:0/867441984 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc01a070 con 0x7f21dc1045e0 2024-03-21T08:29:20.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.937+0000 7f21d8c00700 1 -- 172.21.15.3:0/867441984 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f21cc00de20 con 0x7f21dc1045e0 2024-03-21T08:29:20.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.937+0000 7f21d8c00700 1 -- 172.21.15.3:0/867441984 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc014720 con 0x7f21dc1045e0 2024-03-21T08:29:20.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.937+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc19b2c0 con 0x7f21dc1045e0 2024-03-21T08:29:20.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.937+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc1a74d0 con 0x7f21dc1045e0 2024-03-21T08:29:20.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.938+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc105ac0 con 0x7f21dc1045e0 2024-03-21T08:29:20.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.940+0000 7f21d8c00700 1 -- 172.21.15.3:0/867441984 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f21cc01d070 con 0x7f21dc1045e0 2024-03-21T08:29:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.942+0000 7f21d8c00700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21c00a30c0 0x7f21c00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.942+0000 7f21d8c00700 1 -- 172.21.15.3:0/867441984 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f21cc0cf190 con 0x7f21dc1045e0 2024-03-21T08:29:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.942+0000 7f21d2a00700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21c00a30c0 0x7f21c00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.943+0000 7f21d2a00700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21c00a30c0 0x7f21c00a5580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21dc10abc0 tx=0x7f21c400d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:20.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:20.947+0000 7f21d8c00700 1 -- 172.21.15.3:0/867441984 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f21cc010070 con 0x7f21dc1045e0 2024-03-21T08:29:21.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.237+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f21dc1a77c0 con 0x7f21dc1045e0 2024-03-21T08:29:21.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.244+0000 7f21d8c00700 1 -- 172.21.15.3:0/867441984 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+263540 (secure 0 0 0) 0x7f21cc09baa0 con 0x7f21dc1045e0 2024-03-21T08:29:21.249 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.250+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21c00a30c0 msgr2=0x7f21c00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.250+0000 7f21e2600700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21c00a30c0 0x7f21c00a5580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21dc10abc0 tx=0x7f21c400d4b0 comp rx=0 tx=0).stop 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.250+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1045e0 msgr2=0x7f21dc1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.250+0000 7f21e2600700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1045e0 0x7f21dc1a0670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0060b0 tx=0x7f21cc007f90 comp rx=0 tx=0).stop 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 shutdown_connections 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21c00a30c0 0x7f21c00a5580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21dc1045e0 0x7f21dc1a0670 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21dc1096c0 0x7f21dc19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 --2- 172.21.15.3:0/867441984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21dc104f90 0x7f21dc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 >> 172.21.15.3:0/867441984 conn(0x7f21dc0ffe70 msgr2=0x7f21dc100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 shutdown_connections 2024-03-21T08:29:21.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:21.251+0000 7f21e2600700 1 -- 172.21.15.3:0/867441984 wait complete. 2024-03-21T08:29:21.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:21 smithi084 ceph-mon[35477]: pgmap v391: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T08:29:21.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:21 smithi171 ceph-mon[35388]: pgmap v391: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T08:29:21.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:21 smithi003 ceph-mon[31675]: pgmap v391: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T08:29:22.147 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":1520535032},{"type":"v1","addr":"172.21.15.3:6801","nonce":1520535032}]},"active_addr":"172.21.15.3:6801/1520535032","active_change":"2024-03-21T08:19:05.158075+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.3:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1599875703}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1115678781}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3880262252}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":62217793}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":752131913}]}]} 2024-03-21T08:29:22.153 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T08:29:22.153 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T08:29:22.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T08:29:22.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:22 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/867441984' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T08:29:22.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:22 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/867441984' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T08:29:22.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:22 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/867441984' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T08:29:22.672 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:23.167 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:23 smithi003 ceph-mon[31675]: pgmap v392: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T08:29:23.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:23 smithi084 ceph-mon[35477]: pgmap v392: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T08:29:23.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:23 smithi171 ceph-mon[35388]: pgmap v392: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T08:29:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.114+0000 7f4946000700 1 -- 172.21.15.3:0/3816164980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 msgr2=0x7f49401027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.114+0000 7f4946000700 1 --2- 172.21.15.3:0/3816164980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401027f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49300099c0 tx=0x7f4930009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.115+0000 7f4946000700 1 -- 172.21.15.3:0/3816164980 shutdown_connections 2024-03-21T08:29:25.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.115+0000 7f4946000700 1 --2- 172.21.15.3:0/3816164980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401027f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.115+0000 7f4946000700 1 --2- 172.21.15.3:0/3816164980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49401074f0 0x7f494010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.115+0000 7f4946000700 1 --2- 172.21.15.3:0/3816164980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4940102dc0 0x7f4940106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.115+0000 7f4946000700 1 -- 172.21.15.3:0/3816164980 >> 172.21.15.3:0/3816164980 conn(0x7f49400fdc60 msgr2=0x7f4940100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:25.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.115+0000 7f4946000700 1 -- 172.21.15.3:0/3816164980 shutdown_connections 2024-03-21T08:29:25.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.115+0000 7f4946000700 1 -- 172.21.15.3:0/3816164980 wait complete. 2024-03-21T08:29:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.116+0000 7f4946000700 1 Processor -- start 2024-03-21T08:29:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.116+0000 7f4946000700 1 -- start start 2024-03-21T08:29:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.116+0000 7f4946000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.116+0000 7f4946000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4940102dc0 0x7f49401a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f4946000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49401074f0 0x7f494019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f4946000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49401123b0 con 0x7f4940102410 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f4946000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940112230 con 0x7f4940102dc0 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f4946000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940112530 con 0x7f49401074f0 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41610/0 (socket says 172.21.15.3:41610) 2024-03-21T08:29:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 -- 172.21.15.3:0/2089641782 learned_addr learned my addr 172.21.15.3:0/2089641782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f4937e00700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4940102dc0 0x7f49401a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493fe00700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49401074f0 0x7f494019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 -- 172.21.15.3:0/2089641782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49401074f0 msgr2=0x7f494019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49401074f0 0x7f494019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 -- 172.21.15.3:0/2089641782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4940102dc0 msgr2=0x7f49401a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4940102dc0 0x7f49401a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.117+0000 7f493f400700 1 -- 172.21.15.3:0/2089641782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494019b090 con 0x7f4940102410 2024-03-21T08:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.118+0000 7f493f400700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401a06d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4930006010 tx=0x7f49300056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.118+0000 7f493cc00700 1 -- 172.21.15.3:0/2089641782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930026070 con 0x7f4940102410 2024-03-21T08:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.118+0000 7f493cc00700 1 -- 172.21.15.3:0/2089641782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f493000f460 con 0x7f4940102410 2024-03-21T08:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.118+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494019b320 con 0x7f4940102410 2024-03-21T08:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.118+0000 7f493cc00700 1 -- 172.21.15.3:0/2089641782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930009e60 con 0x7f4940102410 2024-03-21T08:29:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.118+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49401a75a0 con 0x7f4940102410 2024-03-21T08:29:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.119+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4940068e50 con 0x7f4940102410 2024-03-21T08:29:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.122+0000 7f493cc00700 1 -- 172.21.15.3:0/2089641782 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f4930029070 con 0x7f4940102410 2024-03-21T08:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.123+0000 7f493cc00700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49240a30c0 0x7f49240a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.123+0000 7f493cc00700 1 -- 172.21.15.3:0/2089641782 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f49300d0590 con 0x7f4940102410 2024-03-21T08:29:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.123+0000 7f4937e00700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49240a30c0 0x7f49240a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.124+0000 7f4937e00700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49240a30c0 0x7f49240a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f494010f510 tx=0x7f4928009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.126+0000 7f493cc00700 1 -- 172.21.15.3:0/2089641782 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4930030020 con 0x7f4940102410 2024-03-21T08:29:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.355+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49401a7770 con 0x7f4940102410 2024-03-21T08:29:25.358 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:25 smithi003 ceph-mon[31675]: pgmap v393: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-03-21T08:29:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.359+0000 7f493cc00700 1 -- 172.21.15.3:0/2089641782 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24506 (secure 0 0 0) 0x7f4930013070 con 0x7f4940102410 2024-03-21T08:29:25.362 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:25.362 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":94,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:29:06.501461+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:29:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49240a30c0 msgr2=0x7f49240a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49240a30c0 0x7f49240a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f494010f510 tx=0x7f4928009260 comp rx=0 tx=0).stop 2024-03-21T08:29:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 msgr2=0x7f49401a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401a06d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4930006010 tx=0x7f49300056a0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 shutdown_connections 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49240a30c0 0x7f49240a5580 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4940102410 0x7f49401a06d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49401074f0 0x7f494019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 --2- 172.21.15.3:0/2089641782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4940102dc0 0x7f49401a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.362+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 >> 172.21.15.3:0/2089641782 conn(0x7f49400fdc60 msgr2=0x7f49400fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.363+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 shutdown_connections 2024-03-21T08:29:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:25.363+0000 7f4946000700 1 -- 172.21.15.3:0/2089641782 wait complete. 2024-03-21T08:29:25.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:25 smithi084 ceph-mon[35477]: pgmap v393: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-03-21T08:29:25.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:25 smithi171 ceph-mon[35388]: pgmap v393: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-03-21T08:29:26.161 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T08:29:26.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T08:29:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:26 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2089641782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:26.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:26 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2089641782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:26.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:26 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2089641782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:26.663 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:27.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:27 smithi084 ceph-mon[35477]: pgmap v394: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-03-21T08:29:27.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:27 smithi171 ceph-mon[35388]: pgmap v394: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-03-21T08:29:27.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:27 smithi003 ceph-mon[31675]: pgmap v394: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-03-21T08:29:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.915+0000 7f364bc00700 1 -- 172.21.15.3:0/3703443890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3644104f90 msgr2=0x7f3644108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.915+0000 7f364bc00700 1 --2- 172.21.15.3:0/3703443890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3644104f90 0x7f3644108fe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f362c0099c0 tx=0x7f362c009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.916+0000 7f364bc00700 1 -- 172.21.15.3:0/3703443890 shutdown_connections 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.916+0000 7f364bc00700 1 --2- 172.21.15.3:0/3703443890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441096c0 0x7f3644111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.916+0000 7f364bc00700 1 --2- 172.21.15.3:0/3703443890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36441045e0 0x7f36441049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.916+0000 7f364bc00700 1 --2- 172.21.15.3:0/3703443890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3644104f90 0x7f3644108fe0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.916+0000 7f364bc00700 1 -- 172.21.15.3:0/3703443890 >> 172.21.15.3:0/3703443890 conn(0x7f36440ffe70 msgr2=0x7f3644102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.916+0000 7f364bc00700 1 -- 172.21.15.3:0/3703443890 shutdown_connections 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.916+0000 7f364bc00700 1 -- 172.21.15.3:0/3703443890 wait complete. 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.917+0000 7f364bc00700 1 Processor -- start 2024-03-21T08:29:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.917+0000 7f364bc00700 1 -- start start 2024-03-21T08:29:28.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.917+0000 7f364bc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441045e0 0x7f36441a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.917+0000 7f364bc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3644104f90 0x7f36441a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.917+0000 7f364bc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36441096c0 0x7f364419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f364bc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3644114550 con 0x7f36441096c0 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f364bc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441143d0 con 0x7f36441045e0 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f364bc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441146d0 con 0x7f3644104f90 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f3649200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441045e0 0x7f36441a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f3649200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441045e0 0x7f36441a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:42170/0 (socket says 172.21.15.3:42170) 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f3649200700 1 -- 172.21.15.3:0/2482926356 learned_addr learned my addr 172.21.15.3:0/2482926356 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f3649c00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36441096c0 0x7f364419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.918+0000 7f363be00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3644104f90 0x7f36441a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3649200700 1 -- 172.21.15.3:0/2482926356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3644104f90 msgr2=0x7f36441a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3649200700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3644104f90 0x7f36441a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:28.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3649200700 1 -- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36441096c0 msgr2=0x7f364419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:28.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3649200700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36441096c0 0x7f364419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:28.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3649200700 1 -- 172.21.15.3:0/2482926356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364419b030 con 0x7f36441045e0 2024-03-21T08:29:28.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3649200700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441045e0 0x7f36441a06d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f363400d980 tx=0x7f363400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:28.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3642000700 1 -- 172.21.15.3:0/2482926356 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634017070 con 0x7f36441045e0 2024-03-21T08:29:28.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364419b320 con 0x7f36441045e0 2024-03-21T08:29:28.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3642000700 1 -- 172.21.15.3:0/2482926356 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3634005020 con 0x7f36441045e0 2024-03-21T08:29:28.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.919+0000 7f3642000700 1 -- 172.21.15.3:0/2482926356 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400fa90 con 0x7f36441045e0 2024-03-21T08:29:28.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.920+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36441a7530 con 0x7f36441045e0 2024-03-21T08:29:28.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.921+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3644105ac0 con 0x7f36441045e0 2024-03-21T08:29:28.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.923+0000 7f3642000700 1 -- 172.21.15.3:0/2482926356 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f36340051d0 con 0x7f36441045e0 2024-03-21T08:29:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.924+0000 7f3642000700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f36280a30c0 0x7f36280a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.924+0000 7f3642000700 1 -- 172.21.15.3:0/2482926356 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f36340cfb10 con 0x7f36441045e0 2024-03-21T08:29:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.925+0000 7f363be00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f36280a30c0 0x7f36280a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.925+0000 7f363be00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f36280a30c0 0x7f36280a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f362c000c00 tx=0x7f362c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:28.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:28.935+0000 7f3642000700 1 -- 172.21.15.3:0/2482926356 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3634020020 con 0x7f36441045e0 2024-03-21T08:29:29.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.168+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f364402da30 con 0x7f36441045e0 2024-03-21T08:29:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.173+0000 7f3642000700 1 -- 172.21.15.3:0/2482926356 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24506 (secure 0 0 0) 0x7f363409c4a0 con 0x7f36441045e0 2024-03-21T08:29:29.175 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:29:29.175 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":94,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:29:06.501461+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.176+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f36280a30c0 msgr2=0x7f36280a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.176+0000 7f364bc00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f36280a30c0 0x7f36280a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f362c000c00 tx=0x7f362c005fb0 comp rx=0 tx=0).stop 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.176+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441045e0 msgr2=0x7f36441a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.176+0000 7f364bc00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441045e0 0x7f36441a06d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f363400d980 tx=0x7f363400dc90 comp rx=0 tx=0).stop 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 shutdown_connections 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f36280a30c0 0x7f36280a5580 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36441096c0 0x7f364419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36441045e0 0x7f36441a06d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 --2- 172.21.15.3:0/2482926356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3644104f90 0x7f36441a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 >> 172.21.15.3:0/2482926356 conn(0x7f36440ffe70 msgr2=0x7f3644102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 shutdown_connections 2024-03-21T08:29:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:29.177+0000 7f364bc00700 1 -- 172.21.15.3:0/2482926356 wait complete. 2024-03-21T08:29:29.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:29 smithi084 ceph-mon[35477]: pgmap v395: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-21T08:29:29.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:29 smithi171 ceph-mon[35388]: pgmap v395: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-21T08:29:29.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:29 smithi003 ceph-mon[31675]: pgmap v395: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-21T08:29:30.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T08:29:30.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T08:29:30.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T08:29:30.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T08:29:30.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T08:29:30.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T08:29:30.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-21T08:29:30.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-21T08:29:30.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-21T08:29:30.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-21T08:29:30.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-21T08:29:30.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-21T08:29:30.232 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:30 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2482926356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:30.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:30 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2482926356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:30.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:30 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2482926356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:29:31.125 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:31.131 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:31.157 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:31.158 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:31.158 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:31.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:31 smithi171 ceph-mon[35388]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-21T08:29:31.421 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:31 smithi003 ceph-mon[31675]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-21T08:29:31.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:31 smithi084 ceph-mon[35477]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-21T08:29:33.378 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:33 smithi003 ceph-mon[31675]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-21T08:29:33.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:33 smithi084 ceph-mon[35477]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-21T08:29:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:33 smithi171 ceph-mon[35388]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-21T08:29:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:34.212 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:34.220 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:34.222 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:34.226 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:34.227 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:35.415 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:35 smithi003 ceph-mon[31675]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T08:29:35.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:35 smithi084 ceph-mon[35477]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T08:29:35.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:35 smithi171 ceph-mon[35388]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T08:29:37.319 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:37 smithi003 ceph-mon[31675]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-21T08:29:37.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:37 smithi084 ceph-mon[35477]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-21T08:29:37.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:37 smithi171 ceph-mon[35388]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-21T08:29:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 -- 172.21.15.3:0/698248299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 msgr2=0x7fbe74105020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 --2- 172.21.15.3:0/698248299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 0x7fbe74105020 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00b0a0 tx=0x7fbe6c00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 -- 172.21.15.3:0/698248299 shutdown_connections 2024-03-21T08:29:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 --2- 172.21.15.3:0/698248299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 0x7fbe74105020 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 --2- 172.21.15.3:0/698248299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe74107080 0x7fbe74069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 --2- 172.21.15.3:0/698248299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7406a380 0x7fbe7406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 -- 172.21.15.3:0/698248299 >> 172.21.15.3:0/698248299 conn(0x7fbe74077fc0 msgr2=0x7fbe740783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 -- 172.21.15.3:0/698248299 shutdown_connections 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.581+0000 7fbe7a200700 1 -- 172.21.15.3:0/698248299 wait complete. 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.583+0000 7fbe7a200700 1 Processor -- start 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.583+0000 7fbe7a200700 1 -- start start 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe7a200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7406a380 0x7fbe7419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe7a200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 0x7fbe7419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe7a200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe74107080 0x7fbe741a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe7a200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe74118910 con 0x7fbe74104b90 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe7a200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe74118790 con 0x7fbe7406a380 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe7a200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe74118a90 con 0x7fbe74107080 2024-03-21T08:29:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 0x7fbe7419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 0x7fbe7419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53228/0 (socket says 172.21.15.3:53228) 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 -- 172.21.15.3:0/2491085942 learned_addr learned my addr 172.21.15.3:0/2491085942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe73400700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7406a380 0x7fbe7419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe73e00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe74107080 0x7fbe741a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 -- 172.21.15.3:0/2491085942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe74107080 msgr2=0x7fbe741a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe74107080 0x7fbe741a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 -- 172.21.15.3:0/2491085942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7406a380 msgr2=0x7fbe7419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7406a380 0x7fbe7419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe6aa00700 1 -- 172.21.15.3:0/2491085942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe741a3dd0 con 0x7fbe74104b90 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe73e00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe74107080 0x7fbe741a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.584+0000 7fbe73400700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7406a380 0x7fbe7419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.585+0000 7fbe6aa00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 0x7fbe7419f300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00eb30 tx=0x7fbe5c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.585+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c00cd40 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.585+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbe5c0044d0 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.585+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe741a40c0 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.585+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c010430 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.585+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe741ab940 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.586+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe741045d0 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.588+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fbe5c003f60 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.590+0000 7fbe70c00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbe580a3040 0x7fbe580a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.590+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbe5c0cfee0 con 0x7fbe74104b90 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.590+0000 7fbe73400700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbe580a3040 0x7fbe580a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.590+0000 7fbe70c00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] conn(0x7fbe580ae1a0 0x7fbe580b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.590+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 --> [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe5c022e50 con 0x7fbe580ae1a0 2024-03-21T08:29:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.590+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe5c0d0290 con 0x7fbe74104b90 2024-03-21T08:29:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.591+0000 7fbe73e00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] conn(0x7fbe580ae1a0 0x7fbe580b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.591+0000 7fbe73400700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbe580a3040 0x7fbe580a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009b50 tx=0x7fbe64009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.591+0000 7fbe73e00700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] conn(0x7fbe580ae1a0 0x7fbe580b0640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.592+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== osd.0 v2:172.21.15.3:6802/3803233195 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbe5c022e50 con 0x7fbe580ae1a0 2024-03-21T08:29:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.610+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 --> [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe74101670 con 0x7fbe580ae1a0 2024-03-21T08:29:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.611+0000 7fbe70c00700 1 -- 172.21.15.3:0/2491085942 <== osd.0 v2:172.21.15.3:6802/3803233195 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe74101670 con 0x7fbe580ae1a0 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.611+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] conn(0x7fbe580ae1a0 msgr2=0x7fbe580b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.611+0000 7fbe7a200700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] conn(0x7fbe580ae1a0 0x7fbe580b0640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbe580a3040 msgr2=0x7fbe580a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbe580a3040 0x7fbe580a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009b50 tx=0x7fbe64009550 comp rx=0 tx=0).stop 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 msgr2=0x7fbe7419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 --2- 172.21.15.3:0/2491085942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe74104b90 0x7fbe7419f300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00eb30 tx=0x7fbe5c00eef0 comp rx=0 tx=0).stop 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe73e00700 1 -- 172.21.15.3:0/2491085942 reap_dead start 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 shutdown_connections 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 >> 172.21.15.3:0/2491085942 conn(0x7fbe74077fc0 msgr2=0x7fbe74103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 shutdown_connections 2024-03-21T08:29:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:38.612+0000 7fbe7a200700 1 -- 172.21.15.3:0/2491085942 wait complete. 2024-03-21T08:29:39.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:39 smithi003 ceph-mon[31675]: pgmap v400: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.801+0000 7ff8a9600700 1 -- 172.21.15.3:0/116865975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 msgr2=0x7ff8a4109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.801+0000 7ff8a9600700 1 --2- 172.21.15.3:0/116865975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 0x7ff8a4109000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8900099c0 tx=0x7ff890009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.802+0000 7ff8a9600700 1 -- 172.21.15.3:0/116865975 shutdown_connections 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.802+0000 7ff8a9600700 1 --2- 172.21.15.3:0/116865975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8a41096e0 0x7ff8a41112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.802+0000 7ff8a9600700 1 --2- 172.21.15.3:0/116865975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a4104600 0x7ff8a41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.802+0000 7ff8a9600700 1 --2- 172.21.15.3:0/116865975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 0x7ff8a4109000 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.802+0000 7ff8a9600700 1 -- 172.21.15.3:0/116865975 >> 172.21.15.3:0/116865975 conn(0x7ff8a40ffe90 msgr2=0x7ff8a41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.802+0000 7ff8a9600700 1 -- 172.21.15.3:0/116865975 shutdown_connections 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.802+0000 7ff8a9600700 1 -- 172.21.15.3:0/116865975 wait complete. 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 Processor -- start 2024-03-21T08:29:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 -- start start 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a4104600 0x7ff8a41a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 0x7ff8a41a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8a41096e0 0x7ff8a419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a41145a0 con 0x7ff8a4104fb0 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4114420 con 0x7ff8a4104600 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.803+0000 7ff8a9600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4114720 con 0x7ff8a41096e0 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 0x7ff8a41a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 0x7ff8a41a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53218/0 (socket says 172.21.15.3:53218) 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 -- 172.21.15.3:0/4017174207 learned_addr learned my addr 172.21.15.3:0/4017174207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2a00700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a4104600 0x7ff8a41a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a3400700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8a41096e0 0x7ff8a419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 -- 172.21.15.3:0/4017174207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8a41096e0 msgr2=0x7ff8a419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8a41096e0 0x7ff8a419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 -- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a4104600 msgr2=0x7ff8a41a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a4104600 0x7ff8a41a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.804+0000 7ff8a2000700 1 -- 172.21.15.3:0/4017174207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a419b080 con 0x7ff8a4104fb0 2024-03-21T08:29:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.805+0000 7ff8a2000700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 0x7ff8a41a0c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff89000c010 tx=0x7ff8900057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.805+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890026070 con 0x7ff8a4104fb0 2024-03-21T08:29:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.805+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff890003f50 con 0x7ff8a4104fb0 2024-03-21T08:29:39.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.805+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89000fc30 con 0x7ff8a4104fb0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.805+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a419b310 con 0x7ff8a4104fb0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.805+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41a7530 con 0x7ff8a4104fb0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.807+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8a4062920 con 0x7ff8a4104fb0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.809+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7ff890029070 con 0x7ff8a4104fb0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.810+0000 7ff89b400700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8880a3040 0x7ff8880a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.810+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff8900d0b60 con 0x7ff8a4104fb0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.811+0000 7ff8a2a00700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8880a3040 0x7ff8880a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.811+0000 7ff89b400700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] conn(0x7ff8880ae070 0x7ff8880b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.811+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 --> [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8880b0be0 con 0x7ff8880ae070 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.811+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8900d0f10 con 0x7ff8a4104fb0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.811+0000 7ff8a2a00700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8880a3040 0x7ff8880a5500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff894005950 tx=0x7ff89400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:39.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.811+0000 7ff8a3400700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] conn(0x7ff8880ae070 0x7ff8880b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.812+0000 7ff8a3400700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] conn(0x7ff8880ae070 0x7ff8880b0510 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.814+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== osd.4 v2:172.21.15.84:6808/2287981020 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff8880b0be0 con 0x7ff8880ae070 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.834+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 --> [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8a4062da0 con 0x7ff8880ae070 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.834+0000 7ff89b400700 1 -- 172.21.15.3:0/4017174207 <== osd.4 v2:172.21.15.84:6808/2287981020 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff8a4062da0 con 0x7ff8880ae070 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] conn(0x7ff8880ae070 msgr2=0x7ff8880b0510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] conn(0x7ff8880ae070 0x7ff8880b0510 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8880a3040 msgr2=0x7ff8880a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff8880a3040 0x7ff8880a5500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff894005950 tx=0x7ff89400b410 comp rx=0 tx=0).stop 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 msgr2=0x7ff8a41a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 --2- 172.21.15.3:0/4017174207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8a4104fb0 0x7ff8a41a0c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff89000c010 tx=0x7ff8900057c0 comp rx=0 tx=0).stop 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a3400700 1 -- 172.21.15.3:0/4017174207 reap_dead start 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 shutdown_connections 2024-03-21T08:29:39.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 >> 172.21.15.3:0/4017174207 conn(0x7ff8a40ffe90 msgr2=0x7ff8a4100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 shutdown_connections 2024-03-21T08:29:39.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:35.835+0000 7ff8a9600700 1 -- 172.21.15.3:0/4017174207 wait complete. 2024-03-21T08:29:39.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:39 smithi084 ceph-mon[35477]: pgmap v400: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:29:39.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:39 smithi171 ceph-mon[35388]: pgmap v400: 129 pgs: 129 active+clean; 640 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.645+0000 7f7366a00700 1 -- 172.21.15.3:0/227749068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 msgr2=0x7f736010efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 --2- 172.21.15.3:0/227749068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 0x7f736010efe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7358009fc0 tx=0x7f735800b330 comp rx=0 tx=0).stop 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 -- 172.21.15.3:0/227749068 shutdown_connections 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 --2- 172.21.15.3:0/227749068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 0x7f736010efe0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 --2- 172.21.15.3:0/227749068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73601081c0 0x7f73601085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 --2- 172.21.15.3:0/227749068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73601021e0 0x7f7360102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 -- 172.21.15.3:0/227749068 >> 172.21.15.3:0/227749068 conn(0x7f73600792c0 msgr2=0x7f73600796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 -- 172.21.15.3:0/227749068 shutdown_connections 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.646+0000 7f7366a00700 1 -- 172.21.15.3:0/227749068 wait complete. 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.647+0000 7f7366a00700 1 Processor -- start 2024-03-21T08:29:41.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.647+0000 7f7366a00700 1 -- start start 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f7366a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73601021e0 0x7f73601a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f7366a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 0x7f73601a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f7366a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73601081c0 0x7f736019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f7366a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7360112320 con 0x7f7360102ba0 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f7366a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73601121a0 con 0x7f73601081c0 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f7366a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73601124a0 con 0x7f73601021e0 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f735f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 0x7f73601a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f735f400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 0x7f73601a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54726/0 (socket says 172.21.15.3:54726) 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f735f400700 1 -- 172.21.15.3:0/1323969370 learned_addr learned my addr 172.21.15.3:0/1323969370 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f7364a00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73601081c0 0x7f736019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.648+0000 7f735fe00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73601021e0 0x7f73601a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735f400700 1 -- 172.21.15.3:0/1323969370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73601021e0 msgr2=0x7f73601a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735f400700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73601021e0 0x7f73601a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735f400700 1 -- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73601081c0 msgr2=0x7f736019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735f400700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73601081c0 0x7f736019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735f400700 1 -- 172.21.15.3:0/1323969370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736019af70 con 0x7f7360102ba0 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f7364a00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73601081c0 0x7f736019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735f400700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 0x7f73601a0b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f734c00d980 tx=0x7f734c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c017070 con 0x7f7360102ba0 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f734c004830 con 0x7f7360102ba0 2024-03-21T08:29:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c005440 con 0x7f7360102ba0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.649+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736019b260 con 0x7f7360102ba0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.650+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73601a7470 con 0x7f7360102ba0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.650+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7360076080 con 0x7f7360102ba0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.653+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f734c0055e0 con 0x7f7360102ba0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.654+0000 7f735cc00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f73440a30c0 0x7f73440a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.654+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f734c0cfd70 con 0x7f7360102ba0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.655+0000 7f735fe00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f73440a30c0 0x7f73440a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.655+0000 7f735cc00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] conn(0x7f73440ae220 0x7f73440b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.655+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 --> [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73440b0d90 con 0x7f73440ae220 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.655+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f734c0d0120 con 0x7f7360102ba0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.655+0000 7f7364a00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] conn(0x7f73440ae220 0x7f73440b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.655+0000 7f735fe00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f73440a30c0 0x7f73440a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7350005fd0 tx=0x7f7350009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.656+0000 7f7364a00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] conn(0x7f73440ae220 0x7f73440b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.658+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== osd.1 v2:172.21.15.84:6800/2209911029 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f73440b0d90 con 0x7f73440ae220 2024-03-21T08:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.681+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 --> [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7360103870 con 0x7f73440ae220 2024-03-21T08:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.682+0000 7f735cc00700 1 -- 172.21.15.3:0/1323969370 <== osd.1 v2:172.21.15.84:6800/2209911029 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7360103870 con 0x7f73440ae220 2024-03-21T08:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] conn(0x7f73440ae220 msgr2=0x7f73440b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] conn(0x7f73440ae220 0x7f73440b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f73440a30c0 msgr2=0x7f73440a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f73440a30c0 0x7f73440a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7350005fd0 tx=0x7f7350009310 comp rx=0 tx=0).stop 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 msgr2=0x7f73601a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 --2- 172.21.15.3:0/1323969370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7360102ba0 0x7f73601a0b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f734c00d980 tx=0x7f734c00dd40 comp rx=0 tx=0).stop 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7364a00700 1 -- 172.21.15.3:0/1323969370 reap_dead start 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 shutdown_connections 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 >> 172.21.15.3:0/1323969370 conn(0x7f73600792c0 msgr2=0x7f7360074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 shutdown_connections 2024-03-21T08:29:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:40.683+0000 7f7366a00700 1 -- 172.21.15.3:0/1323969370 wait complete. 2024-03-21T08:29:41.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:41 smithi003 ceph-mon[31675]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-21T08:29:41.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:41 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:29:41.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:41 smithi084 ceph-mon[35477]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-21T08:29:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:41 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:29:41.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:41 smithi171 ceph-mon[35388]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-21T08:29:41.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:41 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:29:43.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:42 smithi003 ceph-mon[31675]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:43.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:42 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.041 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:42 smithi084 ceph-mon[35477]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:43.041 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.041 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.041 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.041 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:42 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:42 smithi171 ceph-mon[35388]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:43.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:43.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:42 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.650 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:44 smithi003 ceph-mon[31675]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:44.650 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.650 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.650 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:44 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:44 smithi084 ceph-mon[35477]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:44.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:44 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:44 smithi171 ceph-mon[35388]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:44.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:44.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:44 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:34359738472 2024-03-21T08:29:45.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T08:29:45.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:45 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:45.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:45 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:45.906 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:45 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:29:46.746 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:46 smithi003 ceph-mon[31675]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-21T08:29:46.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:46 smithi084 ceph-mon[35477]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-21T08:29:46.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:46 smithi171 ceph-mon[35388]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-21T08:29:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 -- 172.21.15.3:0/2121455084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce7010e7a0 msgr2=0x7fce7010eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 --2- 172.21.15.3:0/2121455084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce7010e7a0 0x7fce7010eb80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fce64009990 tx=0x7fce64009ca0 comp rx=0 tx=0).stop 2024-03-21T08:29:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 -- 172.21.15.3:0/2121455084 shutdown_connections 2024-03-21T08:29:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 --2- 172.21.15.3:0/2121455084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce7010e7a0 0x7fce7010eb80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 --2- 172.21.15.3:0/2121455084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce70109160 0x7fce70115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 --2- 172.21.15.3:0/2121455084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce701087a0 0x7fce70108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 -- 172.21.15.3:0/2121455084 >> 172.21.15.3:0/2121455084 conn(0x7fce70075960 msgr2=0x7fce70077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 -- 172.21.15.3:0/2121455084 shutdown_connections 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.848+0000 7fce77e00700 1 -- 172.21.15.3:0/2121455084 wait complete. 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.849+0000 7fce77e00700 1 Processor -- start 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.849+0000 7fce77e00700 1 -- start start 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce77e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce701087a0 0x7fce7019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce77e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70109160 0x7fce7019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce77e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce7010e7a0 0x7fce701a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce77e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70118a30 con 0x7fce70109160 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce77e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce701188b0 con 0x7fce701087a0 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce77e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70118bb0 con 0x7fce7010e7a0 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce74a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70109160 0x7fce7019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce74a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70109160 0x7fce7019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54746/0 (socket says 172.21.15.3:54746) 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce74a00700 1 -- 172.21.15.3:0/1625431750 learned_addr learned my addr 172.21.15.3:0/1625431750 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce75400700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce701087a0 0x7fce7019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce75e00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce7010e7a0 0x7fce701a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce74a00700 1 -- 172.21.15.3:0/1625431750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce7010e7a0 msgr2=0x7fce701a37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce74a00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce7010e7a0 0x7fce701a37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce74a00700 1 -- 172.21.15.3:0/1625431750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce701087a0 msgr2=0x7fce7019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.850+0000 7fce74a00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce701087a0 0x7fce7019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.851+0000 7fce74a00700 1 -- 172.21.15.3:0/1625431750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce701a3f10 con 0x7fce70109160 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.851+0000 7fce74a00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70109160 0x7fce7019f440 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fce6000d950 tx=0x7fce6000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.851+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60016070 con 0x7fce70109160 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.851+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fce60010460 con 0x7fce70109160 2024-03-21T08:29:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.851+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce701a4200 con 0x7fce70109160 2024-03-21T08:29:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.851+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6000f620 con 0x7fce70109160 2024-03-21T08:29:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.851+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce701aba80 con 0x7fce70109160 2024-03-21T08:29:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.852+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce700628a0 con 0x7fce70109160 2024-03-21T08:29:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.855+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fce60019070 con 0x7fce70109160 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.856+0000 7fce6e000700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fce580a30c0 0x7fce580a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.857+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fce600cfa20 con 0x7fce70109160 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.857+0000 7fce75400700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fce580a30c0 0x7fce580a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.857+0000 7fce6e000700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] conn(0x7fce580ae220 0x7fce580b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.857+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 --> [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce60005690 con 0x7fce580ae220 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.857+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce600cfdd0 con 0x7fce70109160 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.857+0000 7fce75e00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] conn(0x7fce580ae220 0x7fce580b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.858+0000 7fce75400700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fce580a30c0 0x7fce580a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce6400b5c0 tx=0x7fce6401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.858+0000 7fce75e00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] conn(0x7fce580ae220 0x7fce580b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.860+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== osd.9 v2:172.21.15.3:6826/1803156787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fce60005690 con 0x7fce580ae220 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.882+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 --> [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce70062ca0 con 0x7fce580ae220 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.882+0000 7fce6e000700 1 -- 172.21.15.3:0/1625431750 <== osd.9 v2:172.21.15.3:6826/1803156787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fce70062ca0 con 0x7fce580ae220 2024-03-21T08:29:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] conn(0x7fce580ae220 msgr2=0x7fce580b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] conn(0x7fce580ae220 0x7fce580b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fce580a30c0 msgr2=0x7fce580a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fce580a30c0 0x7fce580a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce6400b5c0 tx=0x7fce6401a040 comp rx=0 tx=0).stop 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70109160 msgr2=0x7fce7019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 --2- 172.21.15.3:0/1625431750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce70109160 0x7fce7019f440 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fce6000d950 tx=0x7fce6000dd10 comp rx=0 tx=0).stop 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce75e00700 1 -- 172.21.15.3:0/1625431750 reap_dead start 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 shutdown_connections 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.883+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 >> 172.21.15.3:0/1625431750 conn(0x7fce70075960 msgr2=0x7fce70076080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.884+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 shutdown_connections 2024-03-21T08:29:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:46.884+0000 7fce77e00700 1 -- 172.21.15.3:0/1625431750 wait complete. 2024-03-21T08:29:48.011 INFO:teuthology.orchestra.run.smithi003.stdout:300647710764 2024-03-21T08:29:48.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-21T08:29:48.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 -- 172.21.15.3:0/228122148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 msgr2=0x7fe89410f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:48.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 --2- 172.21.15.3:0/228122148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 0x7fe89410f030 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b0a0 tx=0x7fe88c00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 -- 172.21.15.3:0/228122148 shutdown_connections 2024-03-21T08:29:48.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 --2- 172.21.15.3:0/228122148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 0x7fe89410f030 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 --2- 172.21.15.3:0/228122148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8941082f0 0x7fe8941086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 --2- 172.21.15.3:0/228122148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe894102210 0x7fe894102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 -- 172.21.15.3:0/228122148 >> 172.21.15.3:0/228122148 conn(0x7fe894079250 msgr2=0x7fe894079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 -- 172.21.15.3:0/228122148 shutdown_connections 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.782+0000 7fe89aa00700 1 -- 172.21.15.3:0/228122148 wait complete. 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.783+0000 7fe89aa00700 1 Processor -- start 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.784+0000 7fe89aa00700 1 -- start start 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.784+0000 7fe89aa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe894102210 0x7fe8941a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.784+0000 7fe89aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 0x7fe8941a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.784+0000 7fe89aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8941082f0 0x7fe89419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.784+0000 7fe89aa00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894112350 con 0x7fe894102bd0 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.784+0000 7fe89aa00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8941121d0 con 0x7fe8941082f0 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.784+0000 7fe89aa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8941124d0 con 0x7fe894102210 2024-03-21T08:29:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 0x7fe8941a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 0x7fe8941a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54766/0 (socket says 172.21.15.3:54766) 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 -- 172.21.15.3:0/3351982637 learned_addr learned my addr 172.21.15.3:0/3351982637 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe898a00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8941082f0 0x7fe89419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893e00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe894102210 0x7fe8941a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 -- 172.21.15.3:0/3351982637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe894102210 msgr2=0x7fe8941a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe894102210 0x7fe8941a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 -- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8941082f0 msgr2=0x7fe89419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8941082f0 0x7fe89419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe893400700 1 -- 172.21.15.3:0/3351982637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89419afa0 con 0x7fe894102bd0 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.785+0000 7fe898a00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8941082f0 0x7fe89419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.786+0000 7fe893400700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 0x7fe8941a0b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e9a0 tx=0x7fe88000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.786+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000cbd0 con 0x7fe894102bd0 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.786+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe88000cd70 con 0x7fe894102bd0 2024-03-21T08:29:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.786+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880018750 con 0x7fe894102bd0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.786+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89419b290 con 0x7fe894102bd0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.786+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8941a74a0 con 0x7fe894102bd0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.787+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe89410f0d0 con 0x7fe894102bd0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.789+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fe880010430 con 0x7fe894102bd0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.791+0000 7fe890c00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8780a3040 0x7fe8780a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.791+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe8800d0070 con 0x7fe894102bd0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.791+0000 7fe893e00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8780a3040 0x7fe8780a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.791+0000 7fe890c00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] conn(0x7fe8780ae1a0 0x7fe8780b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.791+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 --> [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8780b0d10 con 0x7fe8780ae1a0 2024-03-21T08:29:48.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.791+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8800d0420 con 0x7fe894102bd0 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.792+0000 7fe898a00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] conn(0x7fe8780ae1a0 0x7fe8780b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.792+0000 7fe893e00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8780a3040 0x7fe8780a5500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe88400dad0 tx=0x7fe88400d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.792+0000 7fe898a00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] conn(0x7fe8780ae1a0 0x7fe8780b0640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.794+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== osd.7 v2:172.21.15.84:6816/3012309737 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe8780b0d10 con 0x7fe8780ae1a0 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.813+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 --> [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8941038d0 con 0x7fe8780ae1a0 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe890c00700 1 -- 172.21.15.3:0/3351982637 <== osd.7 v2:172.21.15.84:6816/3012309737 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe8941038d0 con 0x7fe8780ae1a0 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] conn(0x7fe8780ae1a0 msgr2=0x7fe8780b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe89aa00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] conn(0x7fe8780ae1a0 0x7fe8780b0640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8780a3040 msgr2=0x7fe8780a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe89aa00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe8780a3040 0x7fe8780a5500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe88400dad0 tx=0x7fe88400d4b0 comp rx=0 tx=0).stop 2024-03-21T08:29:48.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 msgr2=0x7fe8941a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:48.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe89aa00700 1 --2- 172.21.15.3:0/3351982637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe894102bd0 0x7fe8941a0b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e9a0 tx=0x7fe88000ed60 comp rx=0 tx=0).stop 2024-03-21T08:29:48.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.814+0000 7fe898a00700 1 -- 172.21.15.3:0/3351982637 reap_dead start 2024-03-21T08:29:48.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.815+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 shutdown_connections 2024-03-21T08:29:48.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.815+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 >> 172.21.15.3:0/3351982637 conn(0x7fe894079250 msgr2=0x7fe89410d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:48.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.815+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 shutdown_connections 2024-03-21T08:29:48.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:47.815+0000 7fe89aa00700 1 -- 172.21.15.3:0/3351982637 wait complete. 2024-03-21T08:29:48.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:48 smithi084 ceph-mon[35477]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:48.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:48 smithi171 ceph-mon[35388]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:48.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:48 smithi003 ceph-mon[31675]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T08:29:49.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 -- 172.21.15.3:0/2997996861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06601023d0 msgr2=0x7f06601027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 --2- 172.21.15.3:0/2997996861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06601023d0 0x7f06601027b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f06500099c0 tx=0x7f0650009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 -- 172.21.15.3:0/2997996861 shutdown_connections 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 --2- 172.21.15.3:0/2997996861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06601023d0 0x7f06601027b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 --2- 172.21.15.3:0/2997996861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06601074b0 0x7f066010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 --2- 172.21.15.3:0/2997996861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0660102d80 0x7f0660106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 -- 172.21.15.3:0/2997996861 >> 172.21.15.3:0/2997996861 conn(0x7f06600fdc40 msgr2=0x7f0660100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 -- 172.21.15.3:0/2997996861 shutdown_connections 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.559+0000 7f0665400700 1 -- 172.21.15.3:0/2997996861 wait complete. 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.560+0000 7f0665400700 1 Processor -- start 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.560+0000 7f0665400700 1 -- start start 2024-03-21T08:29:49.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.560+0000 7f0665400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06601023d0 0x7f06601090a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.560+0000 7f0665400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0660102d80 0x7f06601095e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.560+0000 7f0665400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06601074b0 0x7f0660109b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.560+0000 7f0665400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06601123b0 con 0x7f0660102d80 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.560+0000 7f0665400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0660112230 con 0x7f06601023d0 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f0665400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0660112530 con 0x7f06601074b0 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0660102d80 0x7f06601095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0660102d80 0x7f06601095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54786/0 (socket says 172.21.15.3:54786) 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 -- 172.21.15.3:0/1731657790 learned_addr learned my addr 172.21.15.3:0/1731657790 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065e800700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06601023d0 0x7f06601090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065f200700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06601074b0 0x7f0660109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 -- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06601074b0 msgr2=0x7f0660109b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06601074b0 0x7f0660109b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 -- 172.21.15.3:0/1731657790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06601023d0 msgr2=0x7f06601090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06601023d0 0x7f06601090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.561+0000 7f065de00700 1 -- 172.21.15.3:0/1731657790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06601ab1d0 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.562+0000 7f065de00700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0660102d80 0x7f06601095e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c970 tx=0x7f064c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.562+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c007b00 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.562+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f064c004ce0 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.562+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06601ab4c0 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.562+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c005590 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.562+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06601ab9b0 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.564+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f066010e740 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.566+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f064c004750 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.567+0000 7f0657400700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f06440a3040 0x7f06440a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.567+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f064c0cf5a0 con 0x7f0660102d80 2024-03-21T08:29:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.567+0000 7f065e800700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f06440a3040 0x7f06440a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.568+0000 7f0657400700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] conn(0x7f06440ae1a0 0x7f06440b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.568+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 --> [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f064c004520 con 0x7f06440ae1a0 2024-03-21T08:29:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.568+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f064c0d6050 con 0x7f0660102d80 2024-03-21T08:29:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.568+0000 7f065e800700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f06440a3040 0x7f06440a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0650005b40 tx=0x7f0650005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.568+0000 7f065f200700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] conn(0x7f06440ae1a0 0x7f06440b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.569+0000 7f065f200700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] conn(0x7f06440ae1a0 0x7f06440b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.572+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== osd.2 v2:172.21.15.171:6800/2357635272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f064c004520 con 0x7f06440ae1a0 2024-03-21T08:29:49.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.600+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 --> [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f06601038b0 con 0x7f06440ae1a0 2024-03-21T08:29:49.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.601+0000 7f0657400700 1 -- 172.21.15.3:0/1731657790 <== osd.2 v2:172.21.15.171:6800/2357635272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f06601038b0 con 0x7f06440ae1a0 2024-03-21T08:29:49.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] conn(0x7f06440ae1a0 msgr2=0x7f06440b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:49.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] conn(0x7f06440ae1a0 0x7f06440b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:49.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f06440a3040 msgr2=0x7f06440a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:49.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f06440a3040 0x7f06440a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0650005b40 tx=0x7f0650005a90 comp rx=0 tx=0).stop 2024-03-21T08:29:49.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0660102d80 msgr2=0x7f06601095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:49.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 --2- 172.21.15.3:0/1731657790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0660102d80 0x7f06601095e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c970 tx=0x7f064c00cd30 comp rx=0 tx=0).stop 2024-03-21T08:29:49.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f065f200700 1 -- 172.21.15.3:0/1731657790 reap_dead start 2024-03-21T08:29:49.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 shutdown_connections 2024-03-21T08:29:49.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.602+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 >> 172.21.15.3:0/1731657790 conn(0x7f06600fdc40 msgr2=0x7f06600fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:49.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.603+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 shutdown_connections 2024-03-21T08:29:49.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:49.603+0000 7f0665400700 1 -- 172.21.15.3:0/1731657790 wait complete. 2024-03-21T08:29:49.784 INFO:teuthology.orchestra.run.smithi003.stdout:55834574946 2024-03-21T08:29:49.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T08:29:50.837 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:50 smithi003 ceph-mon[31675]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-21T08:29:50.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:50 smithi084 ceph-mon[35477]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-21T08:29:50.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:50 smithi171 ceph-mon[35388]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-21T08:29:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 -- 172.21.15.3:0/2739993292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 msgr2=0x7fe84810f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 --2- 172.21.15.3:0/2739993292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 0x7fe84810f020 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b0d0 tx=0x7fe84000b3e0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 -- 172.21.15.3:0/2739993292 shutdown_connections 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 --2- 172.21.15.3:0/2739993292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 0x7fe84810f020 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 --2- 172.21.15.3:0/2739993292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8481081e0 0x7fe8481085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 --2- 172.21.15.3:0/2739993292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe848069e40 0x7fe84806a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 -- 172.21.15.3:0/2739993292 >> 172.21.15.3:0/2739993292 conn(0x7fe848077fc0 msgr2=0x7fe8480783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 -- 172.21.15.3:0/2739993292 shutdown_connections 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.297+0000 7fe84d400700 1 -- 172.21.15.3:0/2739993292 wait complete. 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.299+0000 7fe84d400700 1 Processor -- start 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.299+0000 7fe84d400700 1 -- start start 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.299+0000 7fe84d400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe848069e40 0x7fe8481a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe84d400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 0x7fe8481a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe84d400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8481081e0 0x7fe84819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe84d400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe848112330 con 0x7fe84806a7e0 2024-03-21T08:29:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe84d400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8481121b0 con 0x7fe8481081e0 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe84d400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8481124b0 con 0x7fe848069e40 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 0x7fe8481a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 0x7fe8481a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59052/0 (socket says 172.21.15.3:59052) 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 -- 172.21.15.3:0/4085010432 learned_addr learned my addr 172.21.15.3:0/4085010432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe846800700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe848069e40 0x7fe8481a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 -- 172.21.15.3:0/4085010432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe848069e40 msgr2=0x7fe8481a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe848069e40 0x7fe8481a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 -- 172.21.15.3:0/4085010432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8481081e0 msgr2=0x7fe84819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8481081e0 0x7fe84819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.300+0000 7fe845e00700 1 -- 172.21.15.3:0/4085010432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84819b070 con 0x7fe84806a7e0 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.301+0000 7fe845e00700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 0x7fe8481a0c40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe83400d950 tx=0x7fe83400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.301+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834016070 con 0x7fe84806a7e0 2024-03-21T08:29:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.301+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe834010460 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.301+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84819b360 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.301+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400f620 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.302+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8481a75d0 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.303+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe848100470 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.305+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fe834019070 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.306+0000 7fe83f400700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe82c0a30c0 0x7fe82c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.306+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe8340cf990 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.306+0000 7fe846800700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe82c0a30c0 0x7fe82c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.306+0000 7fe83f400700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] conn(0x7fe82c0ae220 0x7fe82c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.306+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 --> [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe834014e80 con 0x7fe82c0ae220 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.307+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8340cfd40 con 0x7fe84806a7e0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.307+0000 7fe846800700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe82c0a30c0 0x7fe82c0a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe848107d40 tx=0x7fe838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.307+0000 7fe847200700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] conn(0x7fe82c0ae220 0x7fe82c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.308+0000 7fe847200700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] conn(0x7fe82c0ae220 0x7fe82c0b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.311+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== osd.10 v2:172.21.15.84:6824/3984184621 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe834014e80 con 0x7fe82c0ae220 2024-03-21T08:29:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.329+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 --> [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe848103890 con 0x7fe82c0ae220 2024-03-21T08:29:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.330+0000 7fe83f400700 1 -- 172.21.15.3:0/4085010432 <== osd.10 v2:172.21.15.84:6824/3984184621 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe848103890 con 0x7fe82c0ae220 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 >> [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] conn(0x7fe82c0ae220 msgr2=0x7fe82c0b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] conn(0x7fe82c0ae220 0x7fe82c0b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe82c0a30c0 msgr2=0x7fe82c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe82c0a30c0 0x7fe82c0a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe848107d40 tx=0x7fe838009260 comp rx=0 tx=0).stop 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 msgr2=0x7fe8481a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 --2- 172.21.15.3:0/4085010432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe84806a7e0 0x7fe8481a0c40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe83400d950 tx=0x7fe83400dd10 comp rx=0 tx=0).stop 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe847200700 1 -- 172.21.15.3:0/4085010432 reap_dead start 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 shutdown_connections 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 >> 172.21.15.3:0/4085010432 conn(0x7fe848077fc0 msgr2=0x7fe8480ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 shutdown_connections 2024-03-21T08:29:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.331+0000 7fe84d400700 1 -- 172.21.15.3:0/4085010432 wait complete. 2024-03-21T08:29:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 -- 172.21.15.3:0/3523501026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c06a380 msgr2=0x7f228c06a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 --2- 172.21.15.3:0/3523501026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c06a380 0x7f228c06a7e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2274009990 tx=0x7f2274009ca0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 -- 172.21.15.3:0/3523501026 shutdown_connections 2024-03-21T08:29:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 --2- 172.21.15.3:0/3523501026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f228c104b90 0x7f228c105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 --2- 172.21.15.3:0/3523501026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f228c107080 0x7f228c069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 --2- 172.21.15.3:0/3523501026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c06a380 0x7f228c06a7e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 -- 172.21.15.3:0/3523501026 >> 172.21.15.3:0/3523501026 conn(0x7f228c077fc0 msgr2=0x7f228c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 -- 172.21.15.3:0/3523501026 shutdown_connections 2024-03-21T08:29:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.885+0000 7f2291a00700 1 -- 172.21.15.3:0/3523501026 wait complete. 2024-03-21T08:29:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.886+0000 7f2291a00700 1 Processor -- start 2024-03-21T08:29:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.886+0000 7f2291a00700 1 -- start start 2024-03-21T08:29:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2291a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f228c06a380 0x7f228c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2291a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c104b90 0x7f228c19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2291a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f228c107080 0x7f228c1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2291a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c118930 con 0x7f228c104b90 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2291a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c1187b0 con 0x7f228c06a380 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2291a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c118ab0 con 0x7f228c107080 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2282000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c104b90 0x7f228c19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f228aa00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f228c06a380 0x7f228c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2282000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c104b90 0x7f228c19f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59072/0 (socket says 172.21.15.3:59072) 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f2282000700 1 -- 172.21.15.3:0/2961006160 learned_addr learned my addr 172.21.15.3:0/2961006160 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.887+0000 7f228b400700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f228c107080 0x7f228c1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2282000700 1 -- 172.21.15.3:0/2961006160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f228c107080 msgr2=0x7f228c1a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2282000700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f228c107080 0x7f228c1a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2282000700 1 -- 172.21.15.3:0/2961006160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f228c06a380 msgr2=0x7f228c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2282000700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f228c06a380 0x7f228c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2282000700 1 -- 172.21.15.3:0/2961006160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c1a3eb0 con 0x7f228c104b90 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f228aa00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f228c06a380 0x7f228c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2282000700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c104b90 0x7f228c19f3e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2274000c00 tx=0x7f22740057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274027070 con 0x7f228c104b90 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c1a4140 con 0x7f228c104b90 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2274022470 con 0x7f228c104b90 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.888+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274017650 con 0x7f228c104b90 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.889+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c1aba20 con 0x7f228c104b90 2024-03-21T08:29:51.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.890+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f228c1035d0 con 0x7f228c104b90 2024-03-21T08:29:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.892+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f227402a070 con 0x7f228c104b90 2024-03-21T08:29:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.893+0000 7f2283e00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f22700a30c0 0x7f22700a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.893+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f22740d0b40 con 0x7f228c104b90 2024-03-21T08:29:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.894+0000 7f228aa00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f22700a30c0 0x7f22700a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.894+0000 7f2283e00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] conn(0x7f22700ae0f0 0x7f22700b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.894+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 --> [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2274023e50 con 0x7f22700ae0f0 2024-03-21T08:29:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.894+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22740d0ef0 con 0x7f228c104b90 2024-03-21T08:29:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.894+0000 7f228b400700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] conn(0x7f22700ae0f0 0x7f22700b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.894+0000 7f228aa00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f22700a30c0 0x7f22700a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.895+0000 7f228b400700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] conn(0x7f22700ae0f0 0x7f22700b0590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.896+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== osd.3 v2:172.21.15.3:6810/674164001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2274023e50 con 0x7f22700ae0f0 2024-03-21T08:29:51.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.915+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 --> [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f228c101670 con 0x7f22700ae0f0 2024-03-21T08:29:51.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.916+0000 7f2283e00700 1 -- 172.21.15.3:0/2961006160 <== osd.3 v2:172.21.15.3:6810/674164001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f228c101670 con 0x7f22700ae0f0 2024-03-21T08:29:51.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.916+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] conn(0x7f22700ae0f0 msgr2=0x7f22700b0590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.916+0000 7f2291a00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] conn(0x7f22700ae0f0 0x7f22700b0590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f22700a30c0 msgr2=0x7f22700a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f22700a30c0 0x7f22700a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c009260 comp rx=0 tx=0).stop 2024-03-21T08:29:51.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c104b90 msgr2=0x7f228c19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:51.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 --2- 172.21.15.3:0/2961006160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f228c104b90 0x7f228c19f3e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2274000c00 tx=0x7f22740057a0 comp rx=0 tx=0).stop 2024-03-21T08:29:51.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f228b400700 1 -- 172.21.15.3:0/2961006160 reap_dead start 2024-03-21T08:29:51.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 shutdown_connections 2024-03-21T08:29:51.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 >> 172.21.15.3:0/2961006160 conn(0x7f228c077fc0 msgr2=0x7f228c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:51.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 shutdown_connections 2024-03-21T08:29:51.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.917+0000 7f2291a00700 1 -- 172.21.15.3:0/2961006160 wait complete. 2024-03-21T08:29:52.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:52 smithi084 ceph-mon[35477]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-21T08:29:52.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:52 smithi171 ceph-mon[35388]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-21T08:29:52.884 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:52 smithi003 ceph-mon[31675]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-03-21T08:29:52.942 INFO:teuthology.orchestra.run.smithi003.stdout:322122547237 2024-03-21T08:29:52.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-21T08:29:53.684 INFO:teuthology.orchestra.run.smithi003.stdout:146028888140 2024-03-21T08:29:53.684 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T08:29:53.938 INFO:teuthology.orchestra.run.smithi003.stdout:77309411422 2024-03-21T08:29:53.939 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T08:29:54.060 INFO:teuthology.orchestra.run.smithi003.stdout:227633266747 2024-03-21T08:29:54.060 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-21T08:29:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.192+0000 7f7e8a600700 1 -- 172.21.15.3:0/4023040666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 msgr2=0x7f7e841085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.192+0000 7f7e8a600700 1 --2- 172.21.15.3:0/4023040666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 0x7f7e841085c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7e740099c0 tx=0x7f7e74009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.192+0000 7f7e8a600700 1 -- 172.21.15.3:0/4023040666 shutdown_connections 2024-03-21T08:29:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.192+0000 7f7e8a600700 1 --2- 172.21.15.3:0/4023040666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 0x7f7e841085c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.192+0000 7f7e8a600700 1 --2- 172.21.15.3:0/4023040666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8406a7e0 0x7f7e8410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.192+0000 7f7e8a600700 1 --2- 172.21.15.3:0/4023040666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e84069e40 0x7f7e8406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.192+0000 7f7e8a600700 1 -- 172.21.15.3:0/4023040666 >> 172.21.15.3:0/4023040666 conn(0x7f7e84077fc0 msgr2=0x7f7e840783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.193+0000 7f7e8a600700 1 -- 172.21.15.3:0/4023040666 shutdown_connections 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.193+0000 7f7e8a600700 1 -- 172.21.15.3:0/4023040666 wait complete. 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.194+0000 7f7e8a600700 1 Processor -- start 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.194+0000 7f7e8a600700 1 -- start start 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e8a600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e84069e40 0x7f7e84101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e8a600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e8406a7e0 0x7f7e841016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e8a600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 0x7f7e84101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e8a600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e841122f0 con 0x7f7e841081e0 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e8a600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e84112170 con 0x7f7e8406a7e0 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e8a600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e84112470 con 0x7f7e84069e40 2024-03-21T08:29:54.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e83e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 0x7f7e84101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e83e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 0x7f7e84101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59030/0 (socket says 172.21.15.3:59030) 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e83e00700 1 -- 172.21.15.3:0/697916110 learned_addr learned my addr 172.21.15.3:0/697916110 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e7aa00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e8406a7e0 0x7f7e841016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.195+0000 7f7e83400700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e84069e40 0x7f7e84101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e83e00700 1 -- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e84069e40 msgr2=0x7f7e84101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e83e00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e84069e40 0x7f7e84101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e83e00700 1 -- 172.21.15.3:0/697916110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e8406a7e0 msgr2=0x7f7e841016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e83e00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e8406a7e0 0x7f7e841016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e83e00700 1 -- 172.21.15.3:0/697916110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e841a6d50 con 0x7f7e841081e0 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e7aa00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e8406a7e0 0x7f7e841016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:29:54.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e83e00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 0x7f7e84101be0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c007ee0 tx=0x7f7e7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c00dbd0 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e7c00dd70 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e841a7040 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.196+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c005a20 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.197+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e841a7550 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.198+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e840628a0 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.200+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f7e7c01a020 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.201+0000 7f7e80c00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7e680a3040 0x7f7e680a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.201+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7e7c0cf700 con 0x7f7e841081e0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.202+0000 7f7e83400700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7e680a3040 0x7f7e680a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.202+0000 7f7e80c00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] conn(0x7f7e680ae1a0 0x7f7e680b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.202+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 --> [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e7c00ae50 con 0x7f7e680ae1a0 2024-03-21T08:29:54.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.202+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e7c0cfab0 con 0x7f7e841081e0 2024-03-21T08:29:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.202+0000 7f7e83400700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7e680a3040 0x7f7e680a5500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e740059d0 tx=0x7f7e74005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.203+0000 7f7e7aa00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] conn(0x7f7e680ae1a0 0x7f7e680b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.203+0000 7f7e7aa00700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] conn(0x7f7e680ae1a0 0x7f7e680b0640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.206+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== osd.5 v2:172.21.15.171:6808/246158153 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7e7c00ae50 con 0x7f7e680ae1a0 2024-03-21T08:29:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.228+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 --> [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e84062d20 con 0x7f7e680ae1a0 2024-03-21T08:29:54.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.230+0000 7f7e80c00700 1 -- 172.21.15.3:0/697916110 <== osd.5 v2:172.21.15.171:6808/246158153 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7e84062d20 con 0x7f7e680ae1a0 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.230+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] conn(0x7f7e680ae1a0 msgr2=0x7f7e680b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.230+0000 7f7e8a600700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] conn(0x7f7e680ae1a0 0x7f7e680b0640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.231+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7e680a3040 msgr2=0x7f7e680a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.231+0000 7f7e8a600700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7e680a3040 0x7f7e680a5500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e740059d0 tx=0x7f7e74005940 comp rx=0 tx=0).stop 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.231+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 msgr2=0x7f7e84101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.231+0000 7f7e8a600700 1 --2- 172.21.15.3:0/697916110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e841081e0 0x7f7e84101be0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c007ee0 tx=0x7f7e7c00d3b0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.231+0000 7f7e83e00700 1 -- 172.21.15.3:0/697916110 reap_dead start 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.231+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 shutdown_connections 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.231+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 >> 172.21.15.3:0/697916110 conn(0x7f7e84077fc0 msgr2=0x7f7e840ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.232+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 shutdown_connections 2024-03-21T08:29:54.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:51.232+0000 7f7e8a600700 1 -- 172.21.15.3:0/697916110 wait complete. 2024-03-21T08:29:54.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:54 smithi003 ceph-mon[31675]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-21T08:29:54.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:54 smithi084 ceph-mon[35477]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-21T08:29:54.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:54 smithi171 ceph-mon[35388]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-21T08:29:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 -- 172.21.15.3:0/1564214034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081044c0 msgr2=0x7fea081048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 --2- 172.21.15.3:0/1564214034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081044c0 0x7fea081048a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80099c0 tx=0x7fe9f8009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 -- 172.21.15.3:0/1564214034 shutdown_connections 2024-03-21T08:29:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 --2- 172.21.15.3:0/1564214034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081044c0 0x7fea081048a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 --2- 172.21.15.3:0/1564214034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea081095a0 0x7fea08111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 --2- 172.21.15.3:0/1564214034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea08104e70 0x7fea08108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 -- 172.21.15.3:0/1564214034 >> 172.21.15.3:0/1564214034 conn(0x7fea080ffd70 msgr2=0x7fea08102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:54.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 -- 172.21.15.3:0/1564214034 shutdown_connections 2024-03-21T08:29:54.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.923+0000 7fea0d800700 1 -- 172.21.15.3:0/1564214034 wait complete. 2024-03-21T08:29:54.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.924+0000 7fea0d800700 1 Processor -- start 2024-03-21T08:29:54.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.925+0000 7fea0d800700 1 -- start start 2024-03-21T08:29:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.925+0000 7fea0d800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea081044c0 0x7fea081a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.925+0000 7fea0d800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea08104e70 0x7fea081a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.925+0000 7fea0d800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081095a0 0x7fea0819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.925+0000 7fea0d800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea081144b0 con 0x7fea081095a0 2024-03-21T08:29:54.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.925+0000 7fea0d800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea08114330 con 0x7fea08104e70 2024-03-21T08:29:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.925+0000 7fea0d800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea08114630 con 0x7fea081044c0 2024-03-21T08:29:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081095a0 0x7fea0819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea07400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea08104e70 0x7fea081a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081095a0 0x7fea0819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59094/0 (socket says 172.21.15.3:59094) 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 -- 172.21.15.3:0/1307668779 learned_addr learned my addr 172.21.15.3:0/1307668779 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea07400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea08104e70 0x7fea081a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:43930/0 (socket says 172.21.15.3:43930) 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea07e00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea081044c0 0x7fea081a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 -- 172.21.15.3:0/1307668779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea081044c0 msgr2=0x7fea081a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea081044c0 0x7fea081a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 -- 172.21.15.3:0/1307668779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea08104e70 msgr2=0x7fea081a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea08104e70 0x7fea081a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea0ce00700 1 -- 172.21.15.3:0/1307668779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0819b010 con 0x7fea081095a0 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.926+0000 7fea07400700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea08104e70 0x7fea081a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:29:54.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.927+0000 7fea0ce00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081095a0 0x7fea0819a830 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fea00007ee0 tx=0x7fea0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.927+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00004bb0 con 0x7fea081095a0 2024-03-21T08:29:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.927+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea00004d50 con 0x7fea081095a0 2024-03-21T08:29:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.927+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea000058e0 con 0x7fea081095a0 2024-03-21T08:29:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.927+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0819b300 con 0x7fea081095a0 2024-03-21T08:29:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.927+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea081a74c0 con 0x7fea081095a0 2024-03-21T08:29:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.928+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea080628e0 con 0x7fea081095a0 2024-03-21T08:29:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.930+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fea0001a020 con 0x7fea081095a0 2024-03-21T08:29:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.932+0000 7fea04c00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9ec0a30c0 0x7fe9ec0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.932+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fea000cf5a0 con 0x7fea081095a0 2024-03-21T08:29:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.932+0000 7fea07e00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9ec0a30c0 0x7fe9ec0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.932+0000 7fea04c00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] conn(0x7fe9ec0ae220 0x7fe9ec0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.932+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 --> [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea0001ee70 con 0x7fe9ec0ae220 2024-03-21T08:29:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.932+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea000cf950 con 0x7fea081095a0 2024-03-21T08:29:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.933+0000 7fea07400700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] conn(0x7fe9ec0ae220 0x7fe9ec0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.933+0000 7fea07e00700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9ec0a30c0 0x7fe9ec0a5580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8003760 tx=0x7fe9f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.933+0000 7fea07400700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] conn(0x7fe9ec0ae220 0x7fe9ec0b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.934+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== osd.6 v2:172.21.15.3:6818/3180742022 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fea0001ee70 con 0x7fe9ec0ae220 2024-03-21T08:29:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.953+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 --> [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea08062ce0 con 0x7fe9ec0ae220 2024-03-21T08:29:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.953+0000 7fea04c00700 1 -- 172.21.15.3:0/1307668779 <== osd.6 v2:172.21.15.3:6818/3180742022 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fea08062ce0 con 0x7fe9ec0ae220 2024-03-21T08:29:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] conn(0x7fe9ec0ae220 msgr2=0x7fe9ec0b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] conn(0x7fe9ec0ae220 0x7fe9ec0b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9ec0a30c0 msgr2=0x7fe9ec0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe9ec0a30c0 0x7fe9ec0a5580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8003760 tx=0x7fe9f8005fb0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081095a0 msgr2=0x7fea0819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 --2- 172.21.15.3:0/1307668779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea081095a0 0x7fea0819a830 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fea00007ee0 tx=0x7fea0000d3b0 comp rx=0 tx=0).stop 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0ce00700 1 -- 172.21.15.3:0/1307668779 reap_dead start 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 shutdown_connections 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.954+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 >> 172.21.15.3:0/1307668779 conn(0x7fea080ffd70 msgr2=0x7fea08101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.955+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 shutdown_connections 2024-03-21T08:29:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:54.955+0000 7fea0d800700 1 -- 172.21.15.3:0/1307668779 wait complete. 2024-03-21T08:29:55.707 INFO:teuthology.orchestra.run.smithi003.stdout:111669149783 2024-03-21T08:29:55.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T08:29:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.752+0000 7fbad1400700 1 -- 172.21.15.3:0/160971119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102df0 msgr2=0x7fbacc106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.752+0000 7fbad1400700 1 --2- 172.21.15.3:0/160971119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102df0 0x7fbacc106e30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbab80099c0 tx=0x7fbab8009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.753+0000 7fbad1400700 1 -- 172.21.15.3:0/160971119 shutdown_connections 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.753+0000 7fbad1400700 1 --2- 172.21.15.3:0/160971119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbacc107510 0x7fbacc10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.753+0000 7fbad1400700 1 --2- 172.21.15.3:0/160971119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbacc102440 0x7fbacc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.753+0000 7fbad1400700 1 --2- 172.21.15.3:0/160971119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102df0 0x7fbacc106e30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.753+0000 7fbad1400700 1 -- 172.21.15.3:0/160971119 >> 172.21.15.3:0/160971119 conn(0x7fbacc0792f0 msgr2=0x7fbacc079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.753+0000 7fbad1400700 1 -- 172.21.15.3:0/160971119 shutdown_connections 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.753+0000 7fbad1400700 1 -- 172.21.15.3:0/160971119 wait complete. 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 Processor -- start 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 -- start start 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102440 0x7fbacc074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbacc102df0 0x7fbacc0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbacc107510 0x7fbacc074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc112380 con 0x7fbacc102440 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc112200 con 0x7fbacc107510 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.754+0000 7fbad1400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc112500 con 0x7fbacc102df0 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102440 0x7fbacc074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102440 0x7fbacc074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59116/0 (socket says 172.21.15.3:59116) 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 -- 172.21.15.3:0/3986474294 learned_addr learned my addr 172.21.15.3:0/3986474294 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbacb200700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbacc107510 0x7fbacc074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbac9e00700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbacc102df0 0x7fbacc0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 -- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbacc102df0 msgr2=0x7fbacc0749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbacc102df0 0x7fbacc0749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 -- 172.21.15.3:0/3986474294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbacc107510 msgr2=0x7fbacc074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbacc107510 0x7fbacc074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.755+0000 7fbaca800700 1 -- 172.21.15.3:0/3986474294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc1a6ea0 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.756+0000 7fbaca800700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102440 0x7fbacc074490 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00d950 tx=0x7fbabc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.756+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc016070 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.756+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc1a7190 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.756+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbabc009df0 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.756+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00f3c0 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.756+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc1a76a0 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.757+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbacc0628a0 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.761+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fbabc019070 con 0x7fbacc102440 2024-03-21T08:29:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.762+0000 7fbac3400700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbab00a3040 0x7fbab00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.763+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbabc0cfa00 con 0x7fbacc102440 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.763+0000 7fbac9e00700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbab00a3040 0x7fbab00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.763+0000 7fbac3400700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] conn(0x7fbab00ae1a0 0x7fbab00b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.763+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 --> [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbabc024ea0 con 0x7fbab00ae1a0 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.763+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbabc0cfdb0 con 0x7fbacc102440 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.764+0000 7fbac9e00700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbab00a3040 0x7fbab00a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbab8000c00 tx=0x7fbab8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.764+0000 7fbacb200700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] conn(0x7fbab00ae1a0 0x7fbab00b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.765+0000 7fbacb200700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] conn(0x7fbab00ae1a0 0x7fbab00b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.767+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== osd.8 v2:172.21.15.171:6816/1162829385 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbabc024ea0 con 0x7fbab00ae1a0 2024-03-21T08:29:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.786+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 --> [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbacc062ca0 con 0x7fbab00ae1a0 2024-03-21T08:29:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.787+0000 7fbac3400700 1 -- 172.21.15.3:0/3986474294 <== osd.8 v2:172.21.15.171:6816/1162829385 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbacc062ca0 con 0x7fbab00ae1a0 2024-03-21T08:29:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] conn(0x7fbab00ae1a0 msgr2=0x7fbab00b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] conn(0x7fbab00ae1a0 0x7fbab00b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbab00a3040 msgr2=0x7fbab00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbab00a3040 0x7fbab00a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbab8000c00 tx=0x7fbab8005fd0 comp rx=0 tx=0).stop 2024-03-21T08:29:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102440 msgr2=0x7fbacc074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 --2- 172.21.15.3:0/3986474294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbacc102440 0x7fbacc074490 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00d950 tx=0x7fbabc00dc60 comp rx=0 tx=0).stop 2024-03-21T08:29:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbacb200700 1 -- 172.21.15.3:0/3986474294 reap_dead start 2024-03-21T08:29:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 shutdown_connections 2024-03-21T08:29:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 >> 172.21.15.3:0/3986474294 conn(0x7fbacc0792f0 msgr2=0x7fbacc1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 shutdown_connections 2024-03-21T08:29:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:55.788+0000 7fbad1400700 1 -- 172.21.15.3:0/3986474294 wait complete. 2024-03-21T08:29:56.825 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:56 smithi003 ceph-mon[31675]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T08:29:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:56 smithi084 ceph-mon[35477]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T08:29:56.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:56 smithi171 ceph-mon[35388]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T08:29:56.919 INFO:teuthology.orchestra.run.smithi003.stdout:266287972405 2024-03-21T08:29:56.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-21T08:29:57.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 -- 172.21.15.3:0/1336985082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c80743c0 msgr2=0x7f51c80747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:57.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 --2- 172.21.15.3:0/1336985082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c80743c0 0x7f51c80747a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f51b80099c0 tx=0x7f51b8009cd0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 -- 172.21.15.3:0/1336985082 shutdown_connections 2024-03-21T08:29:57.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 --2- 172.21.15.3:0/1336985082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c80743c0 0x7f51c80747a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 --2- 172.21.15.3:0/1336985082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c8113360 0x7f51c8115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 --2- 172.21.15.3:0/1336985082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51c8074ce0 0x7f51c8112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 -- 172.21.15.3:0/1336985082 >> 172.21.15.3:0/1336985082 conn(0x7f51c80792f0 msgr2=0x7f51c8079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:57.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 -- 172.21.15.3:0/1336985082 shutdown_connections 2024-03-21T08:29:57.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.141+0000 7f51cca00700 1 -- 172.21.15.3:0/1336985082 wait complete. 2024-03-21T08:29:57.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.142+0000 7f51cca00700 1 Processor -- start 2024-03-21T08:29:57.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.143+0000 7f51cca00700 1 -- start start 2024-03-21T08:29:57.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.143+0000 7f51cca00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51c80743c0 0x7f51c819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:57.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.143+0000 7f51cca00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c8074ce0 0x7f51c819f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.143+0000 7f51cca00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c8113360 0x7f51c81a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.143+0000 7f51cca00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c81189f0 con 0x7f51c8074ce0 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.143+0000 7f51cca00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8118870 con 0x7f51c8113360 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.143+0000 7f51cca00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8118b70 con 0x7f51c80743c0 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c8074ce0 0x7f51c819f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c8074ce0 0x7f51c819f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59144/0 (socket says 172.21.15.3:59144) 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 -- 172.21.15.3:0/2883865187 learned_addr learned my addr 172.21.15.3:0/2883865187 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5e00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51c80743c0 0x7f51c819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c6800700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c8113360 0x7f51c81a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:57.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 -- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51c80743c0 msgr2=0x7f51c819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:57.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51c80743c0 0x7f51c819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 -- 172.21.15.3:0/2883865187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c8113360 msgr2=0x7f51c81a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:57.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51c8113360 0x7f51c81a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.144+0000 7f51c5400700 1 -- 172.21.15.3:0/2883865187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c81a3e10 con 0x7f51c8074ce0 2024-03-21T08:29:57.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.145+0000 7f51c5400700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c8074ce0 0x7f51c819f340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c990 tx=0x7f51b400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:57.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.145+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4007940 con 0x7f51c8074ce0 2024-03-21T08:29:57.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.145+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51b400f450 con 0x7f51c8074ce0 2024-03-21T08:29:57.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.145+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c81a4100 con 0x7f51c8074ce0 2024-03-21T08:29:57.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.145+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4018660 con 0x7f51c8074ce0 2024-03-21T08:29:57.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.145+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c81ab980 con 0x7f51c8074ce0 2024-03-21T08:29:57.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.146+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f51c8112b00 con 0x7f51c8074ce0 2024-03-21T08:29:57.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.149+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f51b400f600 con 0x7f51c8074ce0 2024-03-21T08:29:57.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.150+0000 7f51bea00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f51ac0a30c0 0x7f51ac0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:57.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.150+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f51b40cfa10 con 0x7f51c8074ce0 2024-03-21T08:29:57.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.150+0000 7f51c5e00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f51ac0a30c0 0x7f51ac0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:57.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.151+0000 7f51bea00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] conn(0x7f51ac0ae1a0 0x7f51ac0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:29:57.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.151+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 --> [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51b4007e50 con 0x7f51ac0ae1a0 2024-03-21T08:29:57.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.151+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51b40cfdc0 con 0x7f51c8074ce0 2024-03-21T08:29:57.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.151+0000 7f51c5e00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f51ac0a30c0 0x7f51ac0a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f51b8000c00 tx=0x7f51b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:57.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.151+0000 7f51c6800700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] conn(0x7f51ac0ae1a0 0x7f51ac0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:29:57.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.152+0000 7f51c6800700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] conn(0x7f51ac0ae1a0 0x7f51ac0b0640 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:29:57.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.155+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== osd.11 v2:172.21.15.171:6824/2511529180 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f51b4007e50 con 0x7f51ac0ae1a0 2024-03-21T08:29:57.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.173+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 --> [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f51c8075af0 con 0x7f51ac0ae1a0 2024-03-21T08:29:57.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.174+0000 7f51bea00700 1 -- 172.21.15.3:0/2883865187 <== osd.11 v2:172.21.15.171:6824/2511529180 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f51c8075af0 con 0x7f51ac0ae1a0 2024-03-21T08:29:57.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.174+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] conn(0x7f51ac0ae1a0 msgr2=0x7f51ac0b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:57.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.174+0000 7f51cca00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] conn(0x7f51ac0ae1a0 0x7f51ac0b0640 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f51ac0a30c0 msgr2=0x7f51ac0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f51ac0a30c0 0x7f51ac0a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f51b8000c00 tx=0x7f51b8005fb0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c8074ce0 msgr2=0x7f51c819f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 --2- 172.21.15.3:0/2883865187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51c8074ce0 0x7f51c819f340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c990 tx=0x7f51b400cca0 comp rx=0 tx=0).stop 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51c6800700 1 -- 172.21.15.3:0/2883865187 reap_dead start 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 shutdown_connections 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 >> 172.21.15.3:0/2883865187 conn(0x7f51c80792f0 msgr2=0x7f51c8112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 shutdown_connections 2024-03-21T08:29:57.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:29:57.175+0000 7f51cca00700 1 -- 172.21.15.3:0/2883865187 wait complete. 2024-03-21T08:29:58.115 INFO:teuthology.orchestra.run.smithi003.stdout:167503724616 2024-03-21T08:29:58.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T08:29:58.606 INFO:teuthology.orchestra.run.smithi003.stdout:201863462980 2024-03-21T08:29:58.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-21T08:29:58.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:29:58 smithi084 ceph-mon[35477]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T08:29:58.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:29:58 smithi171 ceph-mon[35388]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T08:29:58.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:29:58 smithi003 ceph-mon[31675]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T08:29:59.248 INFO:teuthology.orchestra.run.smithi003.stdout:356482285601 2024-03-21T08:29:59.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-21T08:29:59.312 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.367 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.459 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:29:59.559 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:00.055 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:00.865 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:00 smithi003 ceph-mon[31675]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:00.865 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:00 smithi003 ceph-mon[31675]: overall HEALTH_OK 2024-03-21T08:30:00.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:00 smithi084 ceph-mon[35477]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:00.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:00 smithi084 ceph-mon[35477]: overall HEALTH_OK 2024-03-21T08:30:00.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:00 smithi171 ceph-mon[35388]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:00.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:00 smithi171 ceph-mon[35388]: overall HEALTH_OK 2024-03-21T08:30:02.663 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:02 smithi003 ceph-mon[31675]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:02.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:02 smithi084 ceph-mon[35477]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:02.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:02 smithi171 ceph-mon[35388]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:04.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:04 smithi003 ceph-mon[31675]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-21T08:30:04.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:04 smithi084 ceph-mon[35477]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-21T08:30:04.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:04 smithi171 ceph-mon[35388]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-21T08:30:05.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-03-21T08:30:05.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-21T08:30:05.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-21T08:30:05.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 6]}]: dispatch 2024-03-21T08:30:05.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T08:30:05.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-21T08:30:05.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-03-21T08:30:05.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:05 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-21T08:30:06.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-03-21T08:30:06.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-21T08:30:06.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-21T08:30:06.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 6]}]: dispatch 2024-03-21T08:30:06.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T08:30:06.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-21T08:30:06.128 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-03-21T08:30:06.128 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:05 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 6]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-03-21T08:30:06.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:05 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-21T08:30:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-21T08:30:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-03-21T08:30:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-21T08:30:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-21T08:30:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 6]}]': finished 2024-03-21T08:30:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T08:30:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-21T08:30:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-03-21T08:30:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-21T08:30:06.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:06 smithi003 ceph-mon[31675]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T08:30:07.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 6]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-21T08:30:07.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:06 smithi084 ceph-mon[35477]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T08:30:07.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [5, 6]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-21T08:30:07.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:06 smithi171 ceph-mon[35388]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T08:30:07.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:07 smithi003 ceph-mon[31675]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T08:30:07.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:07 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3432658409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T08:30:08.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:07 smithi084 ceph-mon[35477]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T08:30:08.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:07 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3432658409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T08:30:08.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:07 smithi171 ceph-mon[35388]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T08:30:08.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:07 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3432658409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T08:30:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 -- 172.21.15.3:0/2000876709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102dc0 msgr2=0x7f1df8106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 --2- 172.21.15.3:0/2000876709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102dc0 0x7f1df8106e10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1de00099c0 tx=0x7f1de0009cd0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 -- 172.21.15.3:0/2000876709 shutdown_connections 2024-03-21T08:30:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 --2- 172.21.15.3:0/2000876709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 0x7f1df810f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 --2- 172.21.15.3:0/2000876709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1df8102410 0x7f1df81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 --2- 172.21.15.3:0/2000876709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102dc0 0x7f1df8106e10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 -- 172.21.15.3:0/2000876709 >> 172.21.15.3:0/2000876709 conn(0x7f1df80fdc60 msgr2=0x7f1df8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 -- 172.21.15.3:0/2000876709 shutdown_connections 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.519+0000 7f1dfe200700 1 -- 172.21.15.3:0/2000876709 wait complete. 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.520+0000 7f1dfe200700 1 Processor -- start 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.520+0000 7f1dfe200700 1 -- start start 2024-03-21T08:30:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1dfe200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102410 0x7f1df81a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:08.698 INFO:teuthology.orchestra.run.smithi003.stdout:55834574951 2024-03-21T08:30:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1dfe200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1df8102dc0 0x7f1df81a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1dfe200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 0x7f1df819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1dfe200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df81123b0 con 0x7f1df8102410 2024-03-21T08:30:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1dfe200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8112230 con 0x7f1df81074f0 2024-03-21T08:30:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1dfe200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8112530 con 0x7f1df8102dc0 2024-03-21T08:30:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1df7e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 0x7f1df819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1df7400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102410 0x7f1df81a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1defe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1df8102dc0 0x7f1df81a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1df7e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 0x7f1df819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:58632/0 (socket says 172.21.15.3:58632) 2024-03-21T08:30:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.521+0000 7f1df7e00700 1 -- 172.21.15.3:0/3432658409 learned_addr learned my addr 172.21.15.3:0/3432658409 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.522+0000 7f1df7e00700 1 -- 172.21.15.3:0/3432658409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1df8102dc0 msgr2=0x7f1df81a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.522+0000 7f1df7e00700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1df8102dc0 0x7f1df81a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.522+0000 7f1df7e00700 1 -- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102410 msgr2=0x7f1df81a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.522+0000 7f1df7e00700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102410 0x7f1df81a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.522+0000 7f1df7e00700 1 -- 172.21.15.3:0/3432658409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df819b050 con 0x7f1df81074f0 2024-03-21T08:30:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.522+0000 7f1df7400700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102410 0x7f1df81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:30:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.523+0000 7f1df7e00700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 0x7f1df819a8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1df000ef30 tx=0x7f1df000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.523+0000 7f1df4c00700 1 -- 172.21.15.3:0/3432658409 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df00092e0 con 0x7f1df81074f0 2024-03-21T08:30:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.523+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df819b2e0 con 0x7f1df81074f0 2024-03-21T08:30:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.523+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df81a7600 con 0x7f1df81074f0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.523+0000 7f1df4c00700 1 -- 172.21.15.3:0/3432658409 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1df0004bb0 con 0x7f1df81074f0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.523+0000 7f1df4c00700 1 -- 172.21.15.3:0/3432658409 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0004020 con 0x7f1df81074f0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.525+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd4005330 con 0x7f1df81074f0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.527+0000 7f1df4c00700 1 -- 172.21.15.3:0/3432658409 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f1df001b030 con 0x7f1df81074f0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.529+0000 7f1df4c00700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f1ddc0a3040 0x7f1ddc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.529+0000 7f1df4c00700 1 -- 172.21.15.3:0/3432658409 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f1df00cfc20 con 0x7f1df81074f0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.529+0000 7f1df7400700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f1ddc0a3040 0x7f1ddc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.530+0000 7f1df7400700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f1ddc0a3040 0x7f1ddc0a5500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1df819ba30 tx=0x7f1de80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.541+0000 7f1df4c00700 1 -- 172.21.15.3:0/3432658409 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1df0022020 con 0x7f1df81074f0 2024-03-21T08:30:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.806+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1dd40058f0 con 0x7f1df81074f0 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.807+0000 7f1df4c00700 1 -- 172.21.15.3:0/3432658409 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1df0010040 con 0x7f1df81074f0 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.809+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f1ddc0a3040 msgr2=0x7f1ddc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.809+0000 7f1dfe200700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f1ddc0a3040 0x7f1ddc0a5500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1df819ba30 tx=0x7f1de80073d0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.809+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 msgr2=0x7f1df819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.809+0000 7f1dfe200700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 0x7f1df819a8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1df000ef30 tx=0x7f1df000c3a0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 shutdown_connections 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f1ddc0a3040 0x7f1ddc0a5500 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1df8102410 0x7f1df81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df81074f0 0x7f1df819a8f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 --2- 172.21.15.3:0/3432658409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1df8102dc0 0x7f1df81a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 >> 172.21.15.3:0/3432658409 conn(0x7f1df80fdc60 msgr2=0x7f1df80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 shutdown_connections 2024-03-21T08:30:08.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:06.810+0000 7f1dfe200700 1 -- 172.21.15.3:0/3432658409 wait complete. 2024-03-21T08:30:08.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:08 smithi003 ceph-mon[31675]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-21T08:30:09.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:08 smithi084 ceph-mon[35477]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-21T08:30:09.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:08 smithi171 ceph-mon[35388]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 -- 172.21.15.3:0/1747913748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d84102da0 msgr2=0x7f2d84106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 --2- 172.21.15.3:0/1747913748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d84102da0 0x7f2d84106df0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d700099c0 tx=0x7f2d70009cd0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 -- 172.21.15.3:0/1747913748 shutdown_connections 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 --2- 172.21.15.3:0/1747913748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d841074d0 0x7f2d8410f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 --2- 172.21.15.3:0/1747913748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d841023f0 0x7f2d841027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 --2- 172.21.15.3:0/1747913748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d84102da0 0x7f2d84106df0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 -- 172.21.15.3:0/1747913748 >> 172.21.15.3:0/1747913748 conn(0x7f2d840fdc60 msgr2=0x7f2d84100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:10.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 -- 172.21.15.3:0/1747913748 shutdown_connections 2024-03-21T08:30:10.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.530+0000 7f2d8a200700 1 -- 172.21.15.3:0/1747913748 wait complete. 2024-03-21T08:30:10.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.531+0000 7f2d8a200700 1 Processor -- start 2024-03-21T08:30:10.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.531+0000 7f2d8a200700 1 -- start start 2024-03-21T08:30:10.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d8a200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d841023f0 0x7f2d84074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:10.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d8a200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d84102da0 0x7f2d840749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:10.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d8a200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d841074d0 0x7f2d84074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:10.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d8a200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84112340 con 0x7f2d841023f0 2024-03-21T08:30:10.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d8a200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841121c0 con 0x7f2d84102da0 2024-03-21T08:30:10.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d8a200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841124c0 con 0x7f2d841074d0 2024-03-21T08:30:10.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d83400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d841023f0 0x7f2d84074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:10.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d83400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d841023f0 0x7f2d84074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43674/0 (socket says 172.21.15.3:43674) 2024-03-21T08:30:10.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d83400700 1 -- 172.21.15.3:0/462691404 learned_addr learned my addr 172.21.15.3:0/462691404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:10.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d82a00700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d84102da0 0x7f2d840749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:10.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.532+0000 7f2d83e00700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d841074d0 0x7f2d84074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.533+0000 7f2d83400700 1 -- 172.21.15.3:0/462691404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d841074d0 msgr2=0x7f2d84074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.533+0000 7f2d83400700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d841074d0 0x7f2d84074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.533+0000 7f2d83400700 1 -- 172.21.15.3:0/462691404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d84102da0 msgr2=0x7f2d840749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.533+0000 7f2d83400700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d84102da0 0x7f2d840749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.533+0000 7f2d83400700 1 -- 172.21.15.3:0/462691404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d841a6e00 con 0x7f2d841023f0 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.533+0000 7f2d83400700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d841023f0 0x7f2d84074490 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d950 tx=0x7f2d7400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.534+0000 7f2d7be00700 1 -- 172.21.15.3:0/462691404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74016070 con 0x7f2d841023f0 2024-03-21T08:30:10.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.534+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d841a70f0 con 0x7f2d841023f0 2024-03-21T08:30:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.534+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d841a7650 con 0x7f2d841023f0 2024-03-21T08:30:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.534+0000 7f2d7be00700 1 -- 172.21.15.3:0/462691404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d7400fd40 con 0x7f2d841023f0 2024-03-21T08:30:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.534+0000 7f2d7be00700 1 -- 172.21.15.3:0/462691404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400e3b0 con 0x7f2d841023f0 2024-03-21T08:30:10.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.535+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d64005330 con 0x7f2d841023f0 2024-03-21T08:30:10.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.539+0000 7f2d7be00700 1 -- 172.21.15.3:0/462691404 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f2d74019070 con 0x7f2d841023f0 2024-03-21T08:30:10.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.540+0000 7f2d7be00700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d680a30c0 0x7f2d680a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:10.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.540+0000 7f2d7be00700 1 -- 172.21.15.3:0/462691404 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f2d740cf760 con 0x7f2d841023f0 2024-03-21T08:30:10.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.541+0000 7f2d82a00700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d680a30c0 0x7f2d680a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:10.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.541+0000 7f2d82a00700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d680a30c0 0x7f2d680a5580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c00 tx=0x7f2d70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:10.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.543+0000 7f2d7be00700 1 -- 172.21.15.3:0/462691404 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d7409bf90 con 0x7f2d841023f0 2024-03-21T08:30:10.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.783+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d640058f0 con 0x7f2d841023f0 2024-03-21T08:30:10.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.784+0000 7f2d7be00700 1 -- 172.21.15.3:0/462691404 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d7409b890 con 0x7f2d841023f0 2024-03-21T08:30:10.787 INFO:teuthology.orchestra.run.smithi003.stdout:111669149786 2024-03-21T08:30:10.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d680a30c0 msgr2=0x7f2d680a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:10.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d680a30c0 0x7f2d680a5580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c00 tx=0x7f2d70005fb0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d841023f0 msgr2=0x7f2d84074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:10.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d841023f0 0x7f2d84074490 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d950 tx=0x7f2d7400dc60 comp rx=0 tx=0).stop 2024-03-21T08:30:10.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 shutdown_connections 2024-03-21T08:30:10.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2d680a30c0 0x7f2d680a5580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d841023f0 0x7f2d84074490 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d841074d0 0x7f2d84074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 --2- 172.21.15.3:0/462691404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d84102da0 0x7f2d840749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:10.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.787+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 >> 172.21.15.3:0/462691404 conn(0x7f2d840fdc60 msgr2=0x7f2d840fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:10.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.788+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 shutdown_connections 2024-03-21T08:30:10.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:10.788+0000 7f2d8a200700 1 -- 172.21.15.3:0/462691404 wait complete. 2024-03-21T08:30:10.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:10 smithi003 ceph-mon[31675]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T08:30:11.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:10 smithi084 ceph-mon[35477]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T08:30:11.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:10 smithi171 ceph-mon[35388]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T08:30:11.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:11 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/462691404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T08:30:12.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:11 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/462691404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T08:30:12.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:11 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/462691404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T08:30:12.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:12 smithi003 ceph-mon[31675]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 75 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.807+0000 7f623b800700 1 -- 172.21.15.3:0/1526725832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 msgr2=0x7f6234110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.807+0000 7f623b800700 1 --2- 172.21.15.3:0/1526725832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f6234110f80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b0a0 tx=0x7f622c00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:30:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.807+0000 7f623b800700 1 -- 172.21.15.3:0/1526725832 shutdown_connections 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.807+0000 7f623b800700 1 --2- 172.21.15.3:0/1526725832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f6234110f80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.807+0000 7f623b800700 1 --2- 172.21.15.3:0/1526725832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62340743c0 0x7f62340747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.807+0000 7f623b800700 1 --2- 172.21.15.3:0/1526725832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234074ce0 0x7f623410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.808+0000 7f623b800700 1 -- 172.21.15.3:0/1526725832 >> 172.21.15.3:0/1526725832 conn(0x7f62340792f0 msgr2=0x7f6234079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.808+0000 7f623b800700 1 -- 172.21.15.3:0/1526725832 shutdown_connections 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.808+0000 7f623b800700 1 -- 172.21.15.3:0/1526725832 wait complete. 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 Processor -- start 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 -- start start 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62340743c0 0x7f62341a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234074ce0 0x7f62341a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f623419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234114620 con 0x7f623410eb90 2024-03-21T08:30:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62341144a0 con 0x7f62340743c0 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.809+0000 7f623b800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62341147a0 con 0x7f6234074ce0 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f623419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f623419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43694/0 (socket says 172.21.15.3:43694) 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 -- 172.21.15.3:0/3036141501 learned_addr learned my addr 172.21.15.3:0/3036141501 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6238e00700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62340743c0 0x7f62341a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6233e00700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234074ce0 0x7f62341a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 -- 172.21.15.3:0/3036141501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234074ce0 msgr2=0x7f62341a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234074ce0 0x7f62341a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 -- 172.21.15.3:0/3036141501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62340743c0 msgr2=0x7f62341a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62340743c0 0x7f62341a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.810+0000 7f6239800700 1 -- 172.21.15.3:0/3036141501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623419b080 con 0x7f623410eb90 2024-03-21T08:30:12.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.811+0000 7f6239800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f623419a8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b070 tx=0x7f622c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.811+0000 7f6231600700 1 -- 172.21.15.3:0/3036141501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c021070 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.811+0000 7f6231600700 1 -- 172.21.15.3:0/3036141501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f622c007e30 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.811+0000 7f6231600700 1 -- 172.21.15.3:0/3036141501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0046a0 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.811+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623419b310 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.811+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62341a7530 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.812+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234075af0 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.816+0000 7f6231600700 1 -- 172.21.15.3:0/3036141501 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f622c007980 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.817+0000 7f6231600700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f621c0a30c0 0x7f621c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.818+0000 7f6231600700 1 -- 172.21.15.3:0/3036141501 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f622c00f020 con 0x7f623410eb90 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.818+0000 7f6238e00700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f621c0a30c0 0x7f621c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.819+0000 7f6238e00700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f621c0a30c0 0x7f621c0a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6228009c60 tx=0x7f6228009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:12.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:12.823+0000 7f6231600700 1 -- 172.21.15.3:0/3036141501 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f622c02a020 con 0x7f623410eb90 2024-03-21T08:30:13.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.108+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f623419bd80 con 0x7f623410eb90 2024-03-21T08:30:13.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.109+0000 7f6231600700 1 -- 172.21.15.3:0/3036141501 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f622c09d610 con 0x7f623410eb90 2024-03-21T08:30:13.112 INFO:teuthology.orchestra.run.smithi003.stdout:300647710769 2024-03-21T08:30:13.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f621c0a30c0 msgr2=0x7f621c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:13.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f621c0a30c0 0x7f621c0a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6228009c60 tx=0x7f6228009260 comp rx=0 tx=0).stop 2024-03-21T08:30:13.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 msgr2=0x7f623419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:13.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f623419a8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b070 tx=0x7f622c009c90 comp rx=0 tx=0).stop 2024-03-21T08:30:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 shutdown_connections 2024-03-21T08:30:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f621c0a30c0 0x7f621c0a5580 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f623410eb90 0x7f623419a8a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62340743c0 0x7f62341a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:13.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 --2- 172.21.15.3:0/3036141501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234074ce0 0x7f62341a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:13.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 >> 172.21.15.3:0/3036141501 conn(0x7f62340792f0 msgr2=0x7f62340ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:13.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 shutdown_connections 2024-03-21T08:30:13.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:13.112+0000 7f623b800700 1 -- 172.21.15.3:0/3036141501 wait complete. 2024-03-21T08:30:13.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:12 smithi084 ceph-mon[35477]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 75 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:13.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:12 smithi171 ceph-mon[35388]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 75 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:13.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:13 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3036141501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T08:30:14.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:13 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3036141501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T08:30:14.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:13 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3036141501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T08:30:14.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574946 got 55834574951 for osd.1 2024-03-21T08:30:14.143 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:15.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:14 smithi084 ceph-mon[35477]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 75 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:15.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:14 smithi171 ceph-mon[35388]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 75 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:15.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:14 smithi003 ceph-mon[31675]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 75 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 -- 172.21.15.3:0/357096231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384410e7a0 msgr2=0x7f384410eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 --2- 172.21.15.3:0/357096231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384410e7a0 0x7f384410eb80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f382c009990 tx=0x7f382c009ca0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 -- 172.21.15.3:0/357096231 shutdown_connections 2024-03-21T08:30:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 --2- 172.21.15.3:0/357096231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f384410e7a0 0x7f384410eb80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 --2- 172.21.15.3:0/357096231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3844109160 0x7f3844115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 --2- 172.21.15.3:0/357096231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f38441087a0 0x7f3844108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 -- 172.21.15.3:0/357096231 >> 172.21.15.3:0/357096231 conn(0x7f3844075960 msgr2=0x7f3844077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:15.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 -- 172.21.15.3:0/357096231 shutdown_connections 2024-03-21T08:30:15.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.645+0000 7f384a200700 1 -- 172.21.15.3:0/357096231 wait complete. 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.646+0000 7f384a200700 1 Processor -- start 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.646+0000 7f384a200700 1 -- start start 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f384a200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38441087a0 0x7f384419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f384a200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3844109160 0x7f384419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f384a200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f384410e7a0 0x7f38441a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f384a200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38441189d0 con 0x7f3844109160 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f384a200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3844118850 con 0x7f38441087a0 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f384a200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3844118b50 con 0x7f384410e7a0 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f3842a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3844109160 0x7f384419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f3843400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38441087a0 0x7f384419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f3842a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3844109160 0x7f384419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43712/0 (socket says 172.21.15.3:43712) 2024-03-21T08:30:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f3842a00700 1 -- 172.21.15.3:0/1983806088 learned_addr learned my addr 172.21.15.3:0/1983806088 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f3843e00700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f384410e7a0 0x7f38441a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f3843400700 1 -- 172.21.15.3:0/1983806088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f384410e7a0 msgr2=0x7f38441a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.647+0000 7f3843400700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f384410e7a0 0x7f38441a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f3843400700 1 -- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3844109160 msgr2=0x7f384419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f3843400700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3844109160 0x7f384419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f3843400700 1 -- 172.21.15.3:0/1983806088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38441a3dd0 con 0x7f38441087a0 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f3843400700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38441087a0 0x7f384419edc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b2e0 tx=0x7f382c004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f383be00700 1 -- 172.21.15.3:0/1983806088 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00b610 con 0x7f38441087a0 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38441a4060 con 0x7f38441087a0 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f383be00700 1 -- 172.21.15.3:0/1983806088 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f382c004800 con 0x7f38441087a0 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.648+0000 7f383be00700 1 -- 172.21.15.3:0/1983806088 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c005580 con 0x7f38441087a0 2024-03-21T08:30:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.649+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38441ab940 con 0x7f38441087a0 2024-03-21T08:30:15.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.650+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3844109ee0 con 0x7f38441087a0 2024-03-21T08:30:15.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.652+0000 7f383be00700 1 -- 172.21.15.3:0/1983806088 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f382c028060 con 0x7f38441087a0 2024-03-21T08:30:15.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.653+0000 7f383be00700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f38280a3040 0x7f38280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:15.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.654+0000 7f383be00700 1 -- 172.21.15.3:0/1983806088 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f382c0d0720 con 0x7f38441087a0 2024-03-21T08:30:15.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.654+0000 7f3842a00700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f38280a3040 0x7f38280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:15.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.654+0000 7f3842a00700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f38280a3040 0x7f38280a5500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f384410ddc0 tx=0x7f3834005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:15.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.664+0000 7f383be00700 1 -- 172.21.15.3:0/1983806088 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f382c05c740 con 0x7f38441087a0 2024-03-21T08:30:15.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.895+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f384402da30 con 0x7f38441087a0 2024-03-21T08:30:15.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.896+0000 7f383be00700 1 -- 172.21.15.3:0/1983806088 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f382c013070 con 0x7f38441087a0 2024-03-21T08:30:15.899 INFO:teuthology.orchestra.run.smithi003.stdout:146028888148 2024-03-21T08:30:15.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f38280a3040 msgr2=0x7f38280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:15.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f38280a3040 0x7f38280a5500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f384410ddc0 tx=0x7f3834005f90 comp rx=0 tx=0).stop 2024-03-21T08:30:15.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38441087a0 msgr2=0x7f384419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38441087a0 0x7f384419edc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b2e0 tx=0x7f382c004790 comp rx=0 tx=0).stop 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 shutdown_connections 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f38280a3040 0x7f38280a5500 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f384410e7a0 0x7f38441a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38441087a0 0x7f384419edc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 --2- 172.21.15.3:0/1983806088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3844109160 0x7f384419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.899+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 >> 172.21.15.3:0/1983806088 conn(0x7f3844075960 msgr2=0x7f3844075ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.900+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 shutdown_connections 2024-03-21T08:30:15.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:15.900+0000 7f384a200700 1 -- 172.21.15.3:0/1983806088 wait complete. 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.460+0000 7fca13200700 1 -- 172.21.15.3:0/2033607750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 msgr2=0x7fca0c111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.460+0000 7fca13200700 1 --2- 172.21.15.3:0/2033607750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c111290 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b0d0 tx=0x7fca0400b3e0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.461+0000 7fca13200700 1 -- 172.21.15.3:0/2033607750 shutdown_connections 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.461+0000 7fca13200700 1 --2- 172.21.15.3:0/2033607750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c111290 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.461+0000 7fca13200700 1 --2- 172.21.15.3:0/2033607750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca0c1045e0 0x7fca0c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.461+0000 7fca13200700 1 --2- 172.21.15.3:0/2033607750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca0c104f90 0x7fca0c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.461+0000 7fca13200700 1 -- 172.21.15.3:0/2033607750 >> 172.21.15.3:0/2033607750 conn(0x7fca0c0ffe90 msgr2=0x7fca0c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.461+0000 7fca13200700 1 -- 172.21.15.3:0/2033607750 shutdown_connections 2024-03-21T08:30:16.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.461+0000 7fca13200700 1 -- 172.21.15.3:0/2033607750 wait complete. 2024-03-21T08:30:16.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.462+0000 7fca13200700 1 Processor -- start 2024-03-21T08:30:16.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.462+0000 7fca13200700 1 -- start start 2024-03-21T08:30:16.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.462+0000 7fca13200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca0c1045e0 0x7fca0c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:16.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca13200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca0c104f90 0x7fca0c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:16.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca13200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:16.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca13200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1145b0 con 0x7fca0c1096c0 2024-03-21T08:30:16.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca13200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c114430 con 0x7fca0c1045e0 2024-03-21T08:30:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca13200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c114730 con 0x7fca0c104f90 2024-03-21T08:30:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca11200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:16.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca11200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43732/0 (socket says 172.21.15.3:43732) 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca11200700 1 -- 172.21.15.3:0/3967018521 learned_addr learned my addr 172.21.15.3:0/3967018521 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca0be00700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca0c1045e0 0x7fca0c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.463+0000 7fca0b400700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca0c104f90 0x7fca0c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca11200700 1 -- 172.21.15.3:0/3967018521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca0c104f90 msgr2=0x7fca0c1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca11200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca0c104f90 0x7fca0c1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca11200700 1 -- 172.21.15.3:0/3967018521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca0c1045e0 msgr2=0x7fca0c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca11200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca0c1045e0 0x7fca0c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca11200700 1 -- 172.21.15.3:0/3967018521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c19b030 con 0x7fca0c1096c0 2024-03-21T08:30:16.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca0be00700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca0c1045e0 0x7fca0c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:30:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca11200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c19a850 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fca040079a0 tx=0x7fca040079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca08c00700 1 -- 172.21.15.3:0/3967018521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0401f070 con 0x7fca0c1096c0 2024-03-21T08:30:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca08c00700 1 -- 172.21.15.3:0/3967018521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca04003c00 con 0x7fca0c1096c0 2024-03-21T08:30:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca08c00700 1 -- 172.21.15.3:0/3967018521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0401b690 con 0x7fca0c1096c0 2024-03-21T08:30:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.464+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c19b2c0 con 0x7fca0c1096c0 2024-03-21T08:30:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.465+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c1a7530 con 0x7fca0c1096c0 2024-03-21T08:30:16.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.466+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c105ac0 con 0x7fca0c1096c0 2024-03-21T08:30:16.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.468+0000 7fca08c00700 1 -- 172.21.15.3:0/3967018521 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fca04022070 con 0x7fca0c1096c0 2024-03-21T08:30:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.469+0000 7fca08c00700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc9f00a3040 0x7fc9f00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.469+0000 7fca08c00700 1 -- 172.21.15.3:0/3967018521 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fca0400f020 con 0x7fca0c1096c0 2024-03-21T08:30:16.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.470+0000 7fca0be00700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc9f00a3040 0x7fc9f00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:16.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.470+0000 7fca0be00700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc9f00a3040 0x7fc9f00a5500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009cb0 tx=0x7fc9fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:16.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.472+0000 7fca08c00700 1 -- 172.21.15.3:0/3967018521 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca04029020 con 0x7fca0c1096c0 2024-03-21T08:30:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.738+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fca0c19bd80 con 0x7fca0c1096c0 2024-03-21T08:30:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.739+0000 7fca08c00700 1 -- 172.21.15.3:0/3967018521 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca0409d850 con 0x7fca0c1096c0 2024-03-21T08:30:16.741 INFO:teuthology.orchestra.run.smithi003.stdout:34359738479 2024-03-21T08:30:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.742+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc9f00a3040 msgr2=0x7fc9f00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.742+0000 7fca13200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc9f00a3040 0x7fc9f00a5500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009cb0 tx=0x7fc9fc009260 comp rx=0 tx=0).stop 2024-03-21T08:30:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.742+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 msgr2=0x7fca0c19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.742+0000 7fca13200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c19a850 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fca040079a0 tx=0x7fca040079d0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 shutdown_connections 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fc9f00a3040 0x7fc9f00a5500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca0c1096c0 0x7fca0c19a850 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca0c1045e0 0x7fca0c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 --2- 172.21.15.3:0/3967018521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca0c104f90 0x7fca0c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 >> 172.21.15.3:0/3967018521 conn(0x7fca0c0ffe90 msgr2=0x7fca0c100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 shutdown_connections 2024-03-21T08:30:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.743+0000 7fca13200700 1 -- 172.21.15.3:0/3967018521 wait complete. 2024-03-21T08:30:17.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:16 smithi084 ceph-mon[35477]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 62 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:17.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:16 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1983806088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T08:30:17.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:16 smithi171 ceph-mon[35388]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 62 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:17.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:16 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1983806088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T08:30:17.135 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:16 smithi003 ceph-mon[31675]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 62 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T08:30:17.135 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:16 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1983806088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 -- 172.21.15.3:0/2702449456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577806b2e0 msgr2=0x7f5778106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 --2- 172.21.15.3:0/2702449456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577806b2e0 0x7f5778106de0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f57640099c0 tx=0x7f5764009cd0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 -- 172.21.15.3:0/2702449456 shutdown_connections 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 --2- 172.21.15.3:0/2702449456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5778107510 0x7f577810f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 --2- 172.21.15.3:0/2702449456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577806a930 0x7f577806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 --2- 172.21.15.3:0/2702449456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577806b2e0 0x7f5778106de0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 -- 172.21.15.3:0/2702449456 >> 172.21.15.3:0/2702449456 conn(0x7f57780770e0 msgr2=0x7f57780774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 -- 172.21.15.3:0/2702449456 shutdown_connections 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.622+0000 7f577cc00700 1 -- 172.21.15.3:0/2702449456 wait complete. 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 Processor -- start 2024-03-21T08:30:17.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 -- start start 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577806a930 0x7f5778101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f577806b2e0 0x7f57781016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5778107510 0x7f5778101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57781123b0 con 0x7f5778107510 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778112230 con 0x7f577806a930 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.624+0000 7f577cc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778112530 con 0x7f577806b2e0 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5778107510 0x7f5778101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5775e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577806a930 0x7f5778101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5778107510 0x7f5778101c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43768/0 (socket says 172.21.15.3:43768) 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 -- 172.21.15.3:0/1872240023 learned_addr learned my addr 172.21.15.3:0/1872240023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:17.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5775400700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f577806b2e0 0x7f57781016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 -- 172.21.15.3:0/1872240023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f577806b2e0 msgr2=0x7f57781016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f577806b2e0 0x7f57781016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 -- 172.21.15.3:0/1872240023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577806a930 msgr2=0x7f5778101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577806a930 0x7f5778101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.625+0000 7f5776800700 1 -- 172.21.15.3:0/1872240023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57781a6e10 con 0x7f5778107510 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.626+0000 7f5776800700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5778107510 0x7f5778101c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f577000b970 tx=0x7f577000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.626+0000 7f576ea00700 1 -- 172.21.15.3:0/1872240023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770019070 con 0x7f5778107510 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.626+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57781a7100 con 0x7f5778107510 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.626+0000 7f576ea00700 1 -- 172.21.15.3:0/1872240023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f577000cde0 con 0x7f5778107510 2024-03-21T08:30:17.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.626+0000 7f576ea00700 1 -- 172.21.15.3:0/1872240023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577001d3b0 con 0x7f5778107510 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.627+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57781a7610 con 0x7f5778107510 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.628+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577810a6c0 con 0x7f5778107510 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.630+0000 7f576ea00700 1 -- 172.21.15.3:0/1872240023 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f577001c070 con 0x7f5778107510 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.631+0000 7f576ea00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f575c0a3040 0x7f575c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.632+0000 7f576ea00700 1 -- 172.21.15.3:0/1872240023 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f57700cebd0 con 0x7f5778107510 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.632+0000 7f5775e00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f575c0a3040 0x7f575c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.633+0000 7f5775e00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f575c0a3040 0x7f575c0a5500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5768005950 tx=0x7f576800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:17.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.636+0000 7f576ea00700 1 -- 172.21.15.3:0/1872240023 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f577009b570 con 0x7f5778107510 2024-03-21T08:30:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.884+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f57781026c0 con 0x7f5778107510 2024-03-21T08:30:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.885+0000 7f576ea00700 1 -- 172.21.15.3:0/1872240023 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f577009ae70 con 0x7f5778107510 2024-03-21T08:30:17.887 INFO:teuthology.orchestra.run.smithi003.stdout:201863462984 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.888+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f575c0a3040 msgr2=0x7f575c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.888+0000 7f577cc00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f575c0a3040 0x7f575c0a5500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5768005950 tx=0x7f576800b410 comp rx=0 tx=0).stop 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5778107510 msgr2=0x7f5778101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5778107510 0x7f5778101c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f577000b970 tx=0x7f577000bc80 comp rx=0 tx=0).stop 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 shutdown_connections 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f575c0a3040 0x7f575c0a5500 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5778107510 0x7f5778101c10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577806a930 0x7f5778101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 --2- 172.21.15.3:0/1872240023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f577806b2e0 0x7f57781016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 >> 172.21.15.3:0/1872240023 conn(0x7f57780770e0 msgr2=0x7f57780ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.889+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 shutdown_connections 2024-03-21T08:30:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.891+0000 7f577cc00700 1 -- 172.21.15.3:0/1872240023 wait complete. 2024-03-21T08:30:17.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.939+0000 7f8306e00700 1 -- 172.21.15.3:0/774524931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300074ce0 msgr2=0x7f8300112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.939+0000 7f8306e00700 1 --2- 172.21.15.3:0/774524931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300074ce0 0x7f8300112e20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0099c0 tx=0x7f82ec009cd0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.940+0000 7f8306e00700 1 -- 172.21.15.3:0/774524931 shutdown_connections 2024-03-21T08:30:17.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.940+0000 7f8306e00700 1 --2- 172.21.15.3:0/774524931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8300113360 0x7f8300115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.940+0000 7f8306e00700 1 --2- 172.21.15.3:0/774524931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000743c0 0x7f83000747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.940+0000 7f8306e00700 1 --2- 172.21.15.3:0/774524931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300074ce0 0x7f8300112e20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.940+0000 7f8306e00700 1 -- 172.21.15.3:0/774524931 >> 172.21.15.3:0/774524931 conn(0x7f83000792f0 msgr2=0x7f8300079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:17.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.940+0000 7f8306e00700 1 -- 172.21.15.3:0/774524931 shutdown_connections 2024-03-21T08:30:17.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.941+0000 7f8306e00700 1 -- 172.21.15.3:0/774524931 wait complete. 2024-03-21T08:30:17.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 Processor -- start 2024-03-21T08:30:17.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 -- start start 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83000743c0 0x7f830019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8300074ce0 0x7f830019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300113360 0x7f83001a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8300118a50 con 0x7f8300113360 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001188d0 con 0x7f8300074ce0 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.942+0000 7f8306e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8300118bd0 con 0x7f83000743c0 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.943+0000 7f8304e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300113360 0x7f83001a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.943+0000 7f82ff400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8300074ce0 0x7f830019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.943+0000 7f82ff400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8300074ce0 0x7f830019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:36958/0 (socket says 172.21.15.3:36958) 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.943+0000 7f8304e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300113360 0x7f83001a37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43792/0 (socket says 172.21.15.3:43792) 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.943+0000 7f82ff400700 1 -- 172.21.15.3:0/458426966 learned_addr learned my addr 172.21.15.3:0/458426966 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.943+0000 7f82ffe00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83000743c0 0x7f830019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f8304e00700 1 -- 172.21.15.3:0/458426966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83000743c0 msgr2=0x7f830019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f8304e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83000743c0 0x7f830019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f8304e00700 1 -- 172.21.15.3:0/458426966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8300074ce0 msgr2=0x7f830019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f8304e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8300074ce0 0x7f830019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f8304e00700 1 -- 172.21.15.3:0/458426966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83001a3f30 con 0x7f8300113360 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f8304e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300113360 0x7f83001a37f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007f10 tx=0x7f82f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f82fcc00700 1 -- 172.21.15.3:0/458426966 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8004db0 con 0x7f8300113360 2024-03-21T08:30:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f82fcc00700 1 -- 172.21.15.3:0/458426966 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82f8009e20 con 0x7f8300113360 2024-03-21T08:30:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f82fcc00700 1 -- 172.21.15.3:0/458426966 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8005a40 con 0x7f8300113360 2024-03-21T08:30:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.944+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83001a4220 con 0x7f8300113360 2024-03-21T08:30:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.945+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83001abaa0 con 0x7f8300113360 2024-03-21T08:30:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.947+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e0005330 con 0x7f8300113360 2024-03-21T08:30:17.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.949+0000 7f82fcc00700 1 -- 172.21.15.3:0/458426966 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f82f800db20 con 0x7f8300113360 2024-03-21T08:30:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.951+0000 7f82fcc00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82e40a30c0 0x7f82e40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.951+0000 7f82fcc00700 1 -- 172.21.15.3:0/458426966 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f82f80cf140 con 0x7f8300113360 2024-03-21T08:30:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.951+0000 7f82ffe00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82e40a30c0 0x7f82e40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.952+0000 7f82ffe00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82e40a30c0 0x7f82e40a5580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009c60 tx=0x7f82f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:17.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.960+0000 7f82fcc00700 1 -- 172.21.15.3:0/458426966 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f82f8010070 con 0x7f8300113360 2024-03-21T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:17 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3967018521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:17 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/546525043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T08:30:18.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:17 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3967018521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T08:30:18.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:17 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/546525043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T08:30:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 -- 172.21.15.3:0/1070983456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c06a7e0 msgr2=0x7fbf5c10f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 --2- 172.21.15.3:0/1070983456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c06a7e0 0x7fbf5c10f020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009f90 tx=0x7fbf5400b330 comp rx=0 tx=0).stop 2024-03-21T08:30:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 -- 172.21.15.3:0/1070983456 shutdown_connections 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 --2- 172.21.15.3:0/1070983456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5c1081e0 0x7fbf5c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 --2- 172.21.15.3:0/1070983456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c06a7e0 0x7fbf5c10f020 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 --2- 172.21.15.3:0/1070983456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf5c069e40 0x7fbf5c06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 -- 172.21.15.3:0/1070983456 >> 172.21.15.3:0/1070983456 conn(0x7fbf5c077fc0 msgr2=0x7fbf5c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 -- 172.21.15.3:0/1070983456 shutdown_connections 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.512+0000 7fbf61200700 1 -- 172.21.15.3:0/1070983456 wait complete. 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 Processor -- start 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 -- start start 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c069e40 0x7fbf5c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf5c06a7e0 0x7fbf5c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5c1081e0 0x7fbf5c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c112310 con 0x7fbf5c1081e0 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c112190 con 0x7fbf5c069e40 2024-03-21T08:30:18.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.514+0000 7fbf61200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c112490 con 0x7fbf5c06a7e0 2024-03-21T08:30:18.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf5a800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5c1081e0 0x7fbf5c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:18.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf59e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c069e40 0x7fbf5c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:18.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf59e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c069e40 0x7fbf5c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:36942/0 (socket says 172.21.15.3:36942) 2024-03-21T08:30:18.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf59e00700 1 -- 172.21.15.3:0/1675846784 learned_addr learned my addr 172.21.15.3:0/1675846784 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:18.176 INFO:teuthology.orchestra.run.smithi003.stdout:322122547242 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf59400700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf5c06a7e0 0x7fbf5c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf5a800700 1 -- 172.21.15.3:0/1675846784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf5c06a7e0 msgr2=0x7fbf5c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf5a800700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf5c06a7e0 0x7fbf5c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf5a800700 1 -- 172.21.15.3:0/1675846784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c069e40 msgr2=0x7fbf5c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf5a800700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c069e40 0x7fbf5c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf5a800700 1 -- 172.21.15.3:0/1675846784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c19afd0 con 0x7fbf5c1081e0 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.515+0000 7fbf59e00700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c069e40 0x7fbf5c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.516+0000 7fbf5a800700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5c1081e0 0x7fbf5c19a870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009790 tx=0x7fbf5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.516+0000 7fbf52a00700 1 -- 172.21.15.3:0/1675846784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54021070 con 0x7fbf5c1081e0 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.516+0000 7fbf52a00700 1 -- 172.21.15.3:0/1675846784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf5401d450 con 0x7fbf5c1081e0 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.516+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c19b260 con 0x7fbf5c1081e0 2024-03-21T08:30:18.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.516+0000 7fbf52a00700 1 -- 172.21.15.3:0/1675846784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf540052a0 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.516+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c1a7580 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.518+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c068e50 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.519+0000 7fbf52a00700 1 -- 172.21.15.3:0/1675846784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fbf54024070 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.522+0000 7fbf52a00700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf400a3040 0x7fbf400a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.522+0000 7fbf52a00700 1 -- 172.21.15.3:0/1675846784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fbf5400f020 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.522+0000 7fbf59e00700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf400a3040 0x7fbf400a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.523+0000 7fbf59e00700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf400a3040 0x7fbf400a5500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c19b9b0 tx=0x7fbf4c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.527+0000 7fbf52a00700 1 -- 172.21.15.3:0/1675846784 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf5402b020 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.780+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fbf5c1a7750 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.780+0000 7fbf52a00700 1 -- 172.21.15.3:0/1675846784 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fbf5409d6d0 con 0x7fbf5c1081e0 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf400a3040 msgr2=0x7fbf400a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf400a3040 0x7fbf400a5500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c19b9b0 tx=0x7fbf4c009310 comp rx=0 tx=0).stop 2024-03-21T08:30:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5c1081e0 msgr2=0x7fbf5c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5c1081e0 0x7fbf5c19a870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009790 tx=0x7fbf5400ccb0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 shutdown_connections 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf400a3040 0x7fbf400a5500 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf5c1081e0 0x7fbf5c19a870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5c069e40 0x7fbf5c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 --2- 172.21.15.3:0/1675846784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf5c06a7e0 0x7fbf5c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 >> 172.21.15.3:0/1675846784 conn(0x7fbf5c077fc0 msgr2=0x7fbf5c10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 shutdown_connections 2024-03-21T08:30:18.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:17.784+0000 7fbf61200700 1 -- 172.21.15.3:0/1675846784 wait complete. 2024-03-21T08:30:18.180 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:17 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3967018521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T08:30:18.180 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:17 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/546525043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T08:30:18.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.197+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f82e00058f0 con 0x7f8300113360 2024-03-21T08:30:18.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.197+0000 7f82fcc00700 1 -- 172.21.15.3:0/458426966 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82f809b970 con 0x7f8300113360 2024-03-21T08:30:18.199 INFO:teuthology.orchestra.run.smithi003.stdout:167503724621 2024-03-21T08:30:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82e40a30c0 msgr2=0x7f82e40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82e40a30c0 0x7f82e40a5580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009c60 tx=0x7f82f0009260 comp rx=0 tx=0).stop 2024-03-21T08:30:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300113360 msgr2=0x7f83001a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300113360 0x7f83001a37f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007f10 tx=0x7f82f800d3b0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 shutdown_connections 2024-03-21T08:30:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82e40a30c0 0x7f82e40a5580 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8300113360 0x7f83001a37f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83000743c0 0x7f830019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 --2- 172.21.15.3:0/458426966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8300074ce0 0x7f830019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 >> 172.21.15.3:0/458426966 conn(0x7f83000792f0 msgr2=0x7f8300112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 shutdown_connections 2024-03-21T08:30:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:18.200+0000 7f8306e00700 1 -- 172.21.15.3:0/458426966 wait complete. 2024-03-21T08:30:18.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.550+0000 7fb4a7600700 1 -- 172.21.15.3:0/3902986046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 msgr2=0x7fb4a010f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.550+0000 7fb4a7600700 1 --2- 172.21.15.3:0/3902986046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a010f0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b0d0 tx=0x7fb49800b3e0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.551+0000 7fb4a7600700 1 -- 172.21.15.3:0/3902986046 shutdown_connections 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.551+0000 7fb4a7600700 1 --2- 172.21.15.3:0/3902986046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a010f0e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.551+0000 7fb4a7600700 1 --2- 172.21.15.3:0/3902986046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a0102440 0x7fb4a0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.551+0000 7fb4a7600700 1 --2- 172.21.15.3:0/3902986046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a0102df0 0x7fb4a0106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.551+0000 7fb4a7600700 1 -- 172.21.15.3:0/3902986046 >> 172.21.15.3:0/3902986046 conn(0x7fb4a00792f0 msgr2=0x7fb4a0079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.551+0000 7fb4a7600700 1 -- 172.21.15.3:0/3902986046 shutdown_connections 2024-03-21T08:30:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.551+0000 7fb4a7600700 1 -- 172.21.15.3:0/3902986046 wait complete. 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.552+0000 7fb4a7600700 1 Processor -- start 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.553+0000 7fb4a7600700 1 -- start start 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.553+0000 7fb4a7600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a0102440 0x7fb4a0074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.553+0000 7fb4a7600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a0102df0 0x7fb4a00749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.553+0000 7fb4a7600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a0074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.553+0000 7fb4a7600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a0112430 con 0x7fb4a0107510 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.553+0000 7fb4a7600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01122b0 con 0x7fb4a0102440 2024-03-21T08:30:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.553+0000 7fb4a7600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01125b0 con 0x7fb4a0102df0 2024-03-21T08:30:19.000 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:18 smithi003 ceph-mon[31675]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 56 B/s, 0 keys/s, 1 objects/s recovering 2024-03-21T08:30:19.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1675846784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T08:30:19.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1872240023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T08:30:19.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/458426966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T08:30:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a0074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a4c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a0102440 0x7fb4a0074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a0074f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43740/0 (socket says 172.21.15.3:43740) 2024-03-21T08:30:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 -- 172.21.15.3:0/546525043 learned_addr learned my addr 172.21.15.3:0/546525043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb49fe00700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a0102df0 0x7fb4a00749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 -- 172.21.15.3:0/546525043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a0102df0 msgr2=0x7fb4a00749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a0102df0 0x7fb4a00749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 -- 172.21.15.3:0/546525043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a0102440 msgr2=0x7fb4a0074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a0102440 0x7fb4a0074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.554+0000 7fb4a5600700 1 -- 172.21.15.3:0/546525043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a01a6e60 con 0x7fb4a0107510 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.555+0000 7fb4a5600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a0074f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb4980079a0 tx=0x7fb4980079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.555+0000 7fb49d600700 1 -- 172.21.15.3:0/546525043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49801f070 con 0x7fb4a0107510 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.555+0000 7fb49d600700 1 -- 172.21.15.3:0/546525043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb498003c00 con 0x7fb4a0107510 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.555+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a01a70f0 con 0x7fb4a0107510 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.555+0000 7fb49d600700 1 -- 172.21.15.3:0/546525043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49801b430 con 0x7fb4a0107510 2024-03-21T08:30:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.555+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a01a7540 con 0x7fb4a0107510 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.557+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a0068e50 con 0x7fb4a0107510 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.558+0000 7fb49d600700 1 -- 172.21.15.3:0/546525043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fb498022070 con 0x7fb4a0107510 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.560+0000 7fb49d600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb4880a30c0 0x7fb4880a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.560+0000 7fb49d600700 1 -- 172.21.15.3:0/546525043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fb4980d03e0 con 0x7fb4a0107510 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.560+0000 7fb4a4c00700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb4880a30c0 0x7fb4880a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.561+0000 7fb4a4c00700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb4880a30c0 0x7fb4880a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb494009c60 tx=0x7fb494009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.564+0000 7fb49d600700 1 -- 172.21.15.3:0/546525043 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb498003db0 con 0x7fb4a0107510 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.836+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb4a002da30 con 0x7fb4a0107510 2024-03-21T08:30:19.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.837+0000 7fb49d600700 1 -- 172.21.15.3:0/546525043 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb49809cca0 con 0x7fb4a0107510 2024-03-21T08:30:19.005 INFO:teuthology.orchestra.run.smithi003.stdout:227633266753 2024-03-21T08:30:19.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.839+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb4880a30c0 msgr2=0x7fb4880a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.839+0000 7fb4a7600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb4880a30c0 0x7fb4880a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb494009c60 tx=0x7fb494009260 comp rx=0 tx=0).stop 2024-03-21T08:30:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 msgr2=0x7fb4a0074f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a0074f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb4980079a0 tx=0x7fb4980079d0 comp rx=0 tx=0).stop 2024-03-21T08:30:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 shutdown_connections 2024-03-21T08:30:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb4880a30c0 0x7fb4880a5580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a0107510 0x7fb4a0074f10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a0102440 0x7fb4a0074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:19.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 --2- 172.21.15.3:0/546525043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a0102df0 0x7fb4a00749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:19.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 >> 172.21.15.3:0/546525043 conn(0x7fb4a00792f0 msgr2=0x7fb4a0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:19.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 shutdown_connections 2024-03-21T08:30:19.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:16.840+0000 7fb4a7600700 1 -- 172.21.15.3:0/546525043 wait complete. 2024-03-21T08:30:19.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:18 smithi084 ceph-mon[35477]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 56 B/s, 0 keys/s, 1 objects/s recovering 2024-03-21T08:30:19.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:18 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1675846784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T08:30:19.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:18 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1872240023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T08:30:19.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:18 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/458426966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T08:30:19.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:18 smithi171 ceph-mon[35388]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 56 B/s, 0 keys/s, 1 objects/s recovering 2024-03-21T08:30:19.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:18 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1675846784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T08:30:19.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:18 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1872240023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T08:30:19.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:18 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/458426966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T08:30:20.063 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724616 got 167503724621 for osd.5 2024-03-21T08:30:20.063 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:20.476 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710764 got 300647710769 for osd.9 2024-03-21T08:30:20.476 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:20.711 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149783 got 111669149786 for osd.3 2024-03-21T08:30:20.711 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.293+0000 7fa0efe00700 1 -- 172.21.15.3:0/810370807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 msgr2=0x7fa0e80747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.293+0000 7fa0efe00700 1 --2- 172.21.15.3:0/810370807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e80747a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b0d0 tx=0x7fa0dc00b3e0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.293+0000 7fa0efe00700 1 -- 172.21.15.3:0/810370807 shutdown_connections 2024-03-21T08:30:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.293+0000 7fa0efe00700 1 --2- 172.21.15.3:0/810370807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e80747a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.293+0000 7fa0efe00700 1 --2- 172.21.15.3:0/810370807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0e8113360 0x7fa0e8115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.293+0000 7fa0efe00700 1 --2- 172.21.15.3:0/810370807 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0e8074ce0 0x7fa0e8112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.293+0000 7fa0efe00700 1 -- 172.21.15.3:0/810370807 >> 172.21.15.3:0/810370807 conn(0x7fa0e80792f0 msgr2=0x7fa0e8079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.294+0000 7fa0efe00700 1 -- 172.21.15.3:0/810370807 shutdown_connections 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.294+0000 7fa0efe00700 1 -- 172.21.15.3:0/810370807 wait complete. 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 Processor -- start 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 -- start start 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0e8074ce0 0x7fa0e819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0e8113360 0x7fa0e81a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8118a40 con 0x7fa0e80743c0 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e81188c0 con 0x7fa0e8113360 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.295+0000 7fa0efe00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8118bc0 con 0x7fa0e8074ce0 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ede00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0e8113360 0x7fa0e81a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ed400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ede00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0e8113360 0x7fa0e81a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57564/0 (socket says 172.21.15.3:57564) 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ede00700 1 -- 172.21.15.3:0/2501025746 learned_addr learned my addr 172.21.15.3:0/2501025746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ed400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53680/0 (socket says 172.21.15.3:53680) 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0eca00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0e8074ce0 0x7fa0e819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ed400700 1 -- 172.21.15.3:0/2501025746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0e8074ce0 msgr2=0x7fa0e819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ed400700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0e8074ce0 0x7fa0e819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ed400700 1 -- 172.21.15.3:0/2501025746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0e8113360 msgr2=0x7fa0e81a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ed400700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0e8113360 0x7fa0e81a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.296+0000 7fa0ed400700 1 -- 172.21.15.3:0/2501025746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e81a3e80 con 0x7fa0e80743c0 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.297+0000 7fa0ed400700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e819ee70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009fd0 tx=0x7fa0dc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.297+0000 7fa0e6000700 1 -- 172.21.15.3:0/2501025746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc003c00 con 0x7fa0e80743c0 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.297+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e81a4110 con 0x7fa0e80743c0 2024-03-21T08:30:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.297+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e81ab9f0 con 0x7fa0e80743c0 2024-03-21T08:30:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.297+0000 7fa0e6000700 1 -- 172.21.15.3:0/2501025746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0dc022070 con 0x7fa0e80743c0 2024-03-21T08:30:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.297+0000 7fa0e6000700 1 -- 172.21.15.3:0/2501025746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc01b6c0 con 0x7fa0e80743c0 2024-03-21T08:30:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.298+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c8005330 con 0x7fa0e80743c0 2024-03-21T08:30:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.300+0000 7fa0e6000700 1 -- 172.21.15.3:0/2501025746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fa0dc01d470 con 0x7fa0e80743c0 2024-03-21T08:30:20.746 INFO:teuthology.orchestra.run.smithi003.stdout:77309411428 2024-03-21T08:30:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.302+0000 7fa0e6000700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a30c0 0x7fa0d00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.302+0000 7fa0e6000700 1 -- 172.21.15.3:0/2501025746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa0dc00f020 con 0x7fa0e80743c0 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.302+0000 7fa0eca00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a30c0 0x7fa0d00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.303+0000 7fa0eca00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a30c0 0x7fa0d00a5580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8115240 tx=0x7fa0d400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.306+0000 7fa0e6000700 1 -- 172.21.15.3:0/2501025746 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0dc09cf30 con 0x7fa0e80743c0 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.548+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa0c80058f0 con 0x7fa0e80743c0 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.549+0000 7fa0e6000700 1 -- 172.21.15.3:0/2501025746 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0dc09c830 con 0x7fa0e80743c0 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a30c0 msgr2=0x7fa0d00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a30c0 0x7fa0d00a5580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8115240 tx=0x7fa0d400a3b0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 msgr2=0x7fa0e819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e819ee70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009fd0 tx=0x7fa0dc007a00 comp rx=0 tx=0).stop 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 shutdown_connections 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a30c0 0x7fa0d00a5580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e80743c0 0x7fa0e819ee70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0e8113360 0x7fa0e81a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 --2- 172.21.15.3:0/2501025746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0e8074ce0 0x7fa0e819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.552+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 >> 172.21.15.3:0/2501025746 conn(0x7fa0e80792f0 msgr2=0x7fa0e8112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.553+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 shutdown_connections 2024-03-21T08:30:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:20.553+0000 7fa0efe00700 1 -- 172.21.15.3:0/2501025746 wait complete. 2024-03-21T08:30:20.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:20 smithi003 ceph-mon[31675]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T08:30:20.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:20 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2501025746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T08:30:21.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:20 smithi084 ceph-mon[35477]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T08:30:21.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:20 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2501025746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T08:30:21.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:20 smithi171 ceph-mon[35388]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T08:30:21.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:20 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2501025746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T08:30:21.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411422 got 77309411428 for osd.2 2024-03-21T08:30:21.718 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:22.280 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738472 got 34359738479 for osd.0 2024-03-21T08:30:22.281 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:22.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266747 got 227633266753 for osd.7 2024-03-21T08:30:22.392 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:22.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:22 smithi003 ceph-mon[31675]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-03-21T08:30:23.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:22 smithi084 ceph-mon[35477]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-03-21T08:30:23.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:22 smithi171 ceph-mon[35388]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2024-03-21T08:30:23.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547237 got 322122547242 for osd.10 2024-03-21T08:30:23.184 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:23.368 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462980 got 201863462984 for osd.6 2024-03-21T08:30:23.369 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:23.679 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888140 got 146028888148 for osd.4 2024-03-21T08:30:23.679 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:24.097 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:24.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.100+0000 7fa666000700 1 -- 172.21.15.3:0/3170418856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 msgr2=0x7fa660115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.100+0000 7fa666000700 1 --2- 172.21.15.3:0/3170418856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 0x7fa660115430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa65800b0a0 tx=0x7fa65800b3b0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.100+0000 7fa666000700 1 -- 172.21.15.3:0/3170418856 shutdown_connections 2024-03-21T08:30:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.100+0000 7fa666000700 1 --2- 172.21.15.3:0/3170418856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 0x7fa660115430 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.100+0000 7fa666000700 1 --2- 172.21.15.3:0/3170418856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6600766c0 0x7fa660076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.100+0000 7fa666000700 1 --2- 172.21.15.3:0/3170418856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 0x7fa660112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.100+0000 7fa666000700 1 -- 172.21.15.3:0/3170418856 >> 172.21.15.3:0/3170418856 conn(0x7fa6600ffe90 msgr2=0x7fa6601022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:24.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.101+0000 7fa666000700 1 -- 172.21.15.3:0/3170418856 shutdown_connections 2024-03-21T08:30:24.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.101+0000 7fa666000700 1 -- 172.21.15.3:0/3170418856 wait complete. 2024-03-21T08:30:24.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.102+0000 7fa666000700 1 Processor -- start 2024-03-21T08:30:24.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa666000700 1 -- start start 2024-03-21T08:30:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa666000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6600766c0 0x7fa660112600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa666000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 0x7fa66010b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa666000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 0x7fa66010bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa666000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6601189a0 con 0x7fa660113040 2024-03-21T08:30:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa666000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa660118820 con 0x7fa6600766c0 2024-03-21T08:30:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa666000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa660118b20 con 0x7fa660076fe0 2024-03-21T08:30:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.103+0000 7fa65f400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6600766c0 0x7fa660112600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 0x7fa66010b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65f400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6600766c0 0x7fa660112600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57582/0 (socket says 172.21.15.3:57582) 2024-03-21T08:30:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 0x7fa66010b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.3:55114/0 (socket says 172.21.15.3:55114) 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65f400700 1 -- 172.21.15.3:0/4110869301 learned_addr learned my addr 172.21.15.3:0/4110869301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65fe00700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 0x7fa66010bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65ea00700 1 -- 172.21.15.3:0/4110869301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6600766c0 msgr2=0x7fa660112600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65ea00700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6600766c0 0x7fa660112600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65ea00700 1 -- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 msgr2=0x7fa66010bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65ea00700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 0x7fa66010bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.104+0000 7fa65ea00700 1 -- 172.21.15.3:0/4110869301 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66010c260 con 0x7fa660076fe0 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.105+0000 7fa65ea00700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 0x7fa66010b6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00e9f0 tx=0x7fa64c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.105+0000 7fa657e00700 1 -- 172.21.15.3:0/4110869301 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c004220 con 0x7fa660076fe0 2024-03-21T08:30:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.105+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa660110130 con 0x7fa660076fe0 2024-03-21T08:30:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.105+0000 7fa657e00700 1 -- 172.21.15.3:0/4110869301 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa64c0043c0 con 0x7fa660076fe0 2024-03-21T08:30:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.105+0000 7fa657e00700 1 -- 172.21.15.3:0/4110869301 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c005020 con 0x7fa660076fe0 2024-03-21T08:30:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.106+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa660110670 con 0x7fa660076fe0 2024-03-21T08:30:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.107+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66010c4f0 con 0x7fa660076fe0 2024-03-21T08:30:24.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.110+0000 7fa657e00700 1 -- 172.21.15.3:0/4110869301 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7fa64c004750 con 0x7fa660076fe0 2024-03-21T08:30:24.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.112+0000 7fa657e00700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6440a30c0 0x7fa6440a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:24.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.112+0000 7fa657e00700 1 -- 172.21.15.3:0/4110869301 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa64c0cf960 con 0x7fa660076fe0 2024-03-21T08:30:24.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.113+0000 7fa65f400700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6440a30c0 0x7fa6440a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:24.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.113+0000 7fa65f400700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6440a30c0 0x7fa6440a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa650009d20 tx=0x7fa650009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.125+0000 7fa657e00700 1 -- 172.21.15.3:0/4110869301 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa64c09c220 con 0x7fa660076fe0 2024-03-21T08:30:24.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.370+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fa66010cbc0 con 0x7fa660076fe0 2024-03-21T08:30:24.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.371+0000 7fa657e00700 1 -- 172.21.15.3:0/4110869301 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa64c09bb20 con 0x7fa660076fe0 2024-03-21T08:30:24.373 INFO:teuthology.orchestra.run.smithi003.stdout:266287972410 2024-03-21T08:30:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.374+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6440a30c0 msgr2=0x7fa6440a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.374+0000 7fa666000700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6440a30c0 0x7fa6440a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa650009d20 tx=0x7fa650009260 comp rx=0 tx=0).stop 2024-03-21T08:30:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.374+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 msgr2=0x7fa66010b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:24.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.374+0000 7fa666000700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 0x7fa66010b6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00e9f0 tx=0x7fa64c00ed00 comp rx=0 tx=0).stop 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.374+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 shutdown_connections 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.374+0000 7fa666000700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa6440a30c0 0x7fa6440a5580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.375+0000 7fa666000700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660113040 0x7fa66010bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.375+0000 7fa666000700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6600766c0 0x7fa660112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.375+0000 7fa666000700 1 --2- 172.21.15.3:0/4110869301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa660076fe0 0x7fa66010b6e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.375+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 >> 172.21.15.3:0/4110869301 conn(0x7fa6600ffe90 msgr2=0x7fa660102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.375+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 shutdown_connections 2024-03-21T08:30:24.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:24.375+0000 7fa666000700 1 -- 172.21.15.3:0/4110869301 wait complete. 2024-03-21T08:30:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:24 smithi003 ceph-mon[31675]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:24 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:24 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4110869301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T08:30:25.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:24 smithi084 ceph-mon[35477]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:24 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:24 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:25.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:24 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4110869301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T08:30:25.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:24 smithi171 ceph-mon[35388]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:25.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:24 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:25.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:24 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4110869301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T08:30:25.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972405 got 266287972410 for osd.8 2024-03-21T08:30:25.628 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:26.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:30:26.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:30:26.099 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:25 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:30:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:30:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:25 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:26.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:30:26.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:30:26.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:25 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:30:26.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 -- 172.21.15.3:0/2023064939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 msgr2=0x7f8250074d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:26.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 --2- 172.21.15.3:0/2023064939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f8250074d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f823c009990 tx=0x7f823c009ca0 comp rx=0 tx=0).stop 2024-03-21T08:30:26.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 -- 172.21.15.3:0/2023064939 shutdown_connections 2024-03-21T08:30:26.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 --2- 172.21.15.3:0/2023064939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8250112e20 0x7f8250115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:26.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 --2- 172.21.15.3:0/2023064939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82500761c0 0x7f82500743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:26.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 --2- 172.21.15.3:0/2023064939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f8250074d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:26.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 -- 172.21.15.3:0/2023064939 >> 172.21.15.3:0/2023064939 conn(0x7f82500792c0 msgr2=0x7f82500796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:26.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 -- 172.21.15.3:0/2023064939 shutdown_connections 2024-03-21T08:30:26.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.909+0000 7f8256e00700 1 -- 172.21.15.3:0/2023064939 wait complete. 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.910+0000 7f8256e00700 1 Processor -- start 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.910+0000 7f8256e00700 1 -- start start 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f8256e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f825019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f8256e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82500761c0 0x7f825019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f8256e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8250112e20 0x7f82501a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f8256e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250118980 con 0x7f8250074900 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f8256e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250118800 con 0x7f82500761c0 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f8256e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250118b00 con 0x7f8250112e20 2024-03-21T08:30:26.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f824fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f825019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f824fe00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f825019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53708/0 (socket says 172.21.15.3:53708) 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f824fe00700 1 -- 172.21.15.3:0/4238301991 learned_addr learned my addr 172.21.15.3:0/4238301991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.911+0000 7f824f400700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82500761c0 0x7f825019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f8254e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8250112e20 0x7f82501a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f824fe00700 1 -- 172.21.15.3:0/4238301991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8250112e20 msgr2=0x7f82501a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f824fe00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8250112e20 0x7f82501a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f824fe00700 1 -- 172.21.15.3:0/4238301991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82500761c0 msgr2=0x7f825019f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f824fe00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82500761c0 0x7f825019f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:26.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f824fe00700 1 -- 172.21.15.3:0/4238301991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82501a3f40 con 0x7f8250074900 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f8254e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8250112e20 0x7f82501a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.912+0000 7f824fe00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f825019ef30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f824000d980 tx=0x7f824000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.913+0000 7f824cc00700 1 -- 172.21.15.3:0/4238301991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240016070 con 0x7f8250074900 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.913+0000 7f824cc00700 1 -- 172.21.15.3:0/4238301991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8240004830 con 0x7f8250074900 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.913+0000 7f824cc00700 1 -- 172.21.15.3:0/4238301991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82400054b0 con 0x7f8250074900 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.913+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82501a4230 con 0x7f8250074900 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.913+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82501abab0 con 0x7f8250074900 2024-03-21T08:30:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.914+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8250068e50 con 0x7f8250074900 2024-03-21T08:30:26.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.916+0000 7f824cc00700 1 -- 172.21.15.3:0/4238301991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f82400049e0 con 0x7f8250074900 2024-03-21T08:30:26.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.917+0000 7f824cc00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82340a30c0 0x7f82340a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:26.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.918+0000 7f824cc00700 1 -- 172.21.15.3:0/4238301991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f82400cf730 con 0x7f8250074900 2024-03-21T08:30:26.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.918+0000 7f824f400700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82340a30c0 0x7f82340a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:26.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.919+0000 7f824f400700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82340a30c0 0x7f82340a5580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8250079ca0 tx=0x7f823c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:26.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:26.923+0000 7f824cc00700 1 -- 172.21.15.3:0/4238301991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f824001f020 con 0x7f8250074900 2024-03-21T08:30:27.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.152+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f82501abc80 con 0x7f8250074900 2024-03-21T08:30:27.155 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:26 smithi003 ceph-mon[31675]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:27.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.153+0000 7f824cc00700 1 -- 172.21.15.3:0/4238301991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f824009bf60 con 0x7f8250074900 2024-03-21T08:30:27.155 INFO:teuthology.orchestra.run.smithi003.stdout:356482285607 2024-03-21T08:30:27.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.156+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82340a30c0 msgr2=0x7f82340a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:27.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.156+0000 7f8256e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82340a30c0 0x7f82340a5580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8250079ca0 tx=0x7f823c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:30:27.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.156+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 msgr2=0x7f825019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:27.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.156+0000 7f8256e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f825019ef30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f824000d980 tx=0x7f824000dc90 comp rx=0 tx=0).stop 2024-03-21T08:30:27.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 shutdown_connections 2024-03-21T08:30:27.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82340a30c0 0x7f82340a5580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:27.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8250074900 0x7f825019ef30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:27.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8250112e20 0x7f82501a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:27.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 --2- 172.21.15.3:0/4238301991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82500761c0 0x7f825019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:27.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 >> 172.21.15.3:0/4238301991 conn(0x7f82500792c0 msgr2=0x7f82501116a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:27.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 shutdown_connections 2024-03-21T08:30:27.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:27.157+0000 7f8256e00700 1 -- 172.21.15.3:0/4238301991 wait complete. 2024-03-21T08:30:27.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:26 smithi084 ceph-mon[35477]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:27.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:26 smithi171 ceph-mon[35388]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:28.017 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285601 got 356482285607 for osd.11 2024-03-21T08:30:28.017 DEBUG:teuthology.parallel:result is None 2024-03-21T08:30:28.018 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T08:30:28.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T08:30:28.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:27 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4238301991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T08:30:28.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:27 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4238301991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T08:30:28.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:27 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4238301991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T08:30:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:29.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:28 smithi003 ceph-mon[31675]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:29.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:28 smithi084 ceph-mon[35477]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:29.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:28 smithi171 ceph-mon[35388]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.745+0000 7f680e400700 1 -- 172.21.15.3:0/1051494469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102c00 msgr2=0x7f680810eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.745+0000 7f680e400700 1 --2- 172.21.15.3:0/1051494469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102c00 0x7f680810eff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f680000b0a0 tx=0x7f680000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:30:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.746+0000 7f680e400700 1 -- 172.21.15.3:0/1051494469 shutdown_connections 2024-03-21T08:30:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.746+0000 7f680e400700 1 --2- 172.21.15.3:0/1051494469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102c00 0x7f680810eff0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.746+0000 7f680e400700 1 --2- 172.21.15.3:0/1051494469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68081082b0 0x7f6808108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.746+0000 7f680e400700 1 --2- 172.21.15.3:0/1051494469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6808102240 0x7f68081026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:30.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.746+0000 7f680e400700 1 -- 172.21.15.3:0/1051494469 >> 172.21.15.3:0/1051494469 conn(0x7f6808079260 msgr2=0x7f6808079670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:30.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.746+0000 7f680e400700 1 -- 172.21.15.3:0/1051494469 shutdown_connections 2024-03-21T08:30:30.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.746+0000 7f680e400700 1 -- 172.21.15.3:0/1051494469 wait complete. 2024-03-21T08:30:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 Processor -- start 2024-03-21T08:30:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 -- start start 2024-03-21T08:30:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102240 0x7f680819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6808102c00 0x7f680819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68081082b0 0x7f6808198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081123d0 con 0x7f6808102240 2024-03-21T08:30:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6808112250 con 0x7f6808102c00 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.747+0000 7f680e400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6808112550 con 0x7f68081082b0 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102240 0x7f680819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102240 0x7f680819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50692/0 (socket says 172.21.15.3:50692) 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 -- 172.21.15.3:0/551932867 learned_addr learned my addr 172.21.15.3:0/551932867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6806a00700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6808102c00 0x7f680819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807e00700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68081082b0 0x7f6808198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 -- 172.21.15.3:0/551932867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68081082b0 msgr2=0x7f6808198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68081082b0 0x7f6808198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 -- 172.21.15.3:0/551932867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6808102c00 msgr2=0x7f680819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6808102c00 0x7f680819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807400700 1 -- 172.21.15.3:0/551932867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808198f10 con 0x7f6808102240 2024-03-21T08:30:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.748+0000 7f6807e00700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68081082b0 0x7f6808198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:30:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.749+0000 7f6807400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102240 0x7f680819e560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f67f800d950 tx=0x7f67f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.749+0000 7f67ffe00700 1 -- 172.21.15.3:0/551932867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8016070 con 0x7f6808102240 2024-03-21T08:30:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.749+0000 7f67ffe00700 1 -- 172.21.15.3:0/551932867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f67f8009df0 con 0x7f6808102240 2024-03-21T08:30:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.749+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6808199200 con 0x7f6808102240 2024-03-21T08:30:30.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.749+0000 7f67ffe00700 1 -- 172.21.15.3:0/551932867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800f3c0 con 0x7f6808102240 2024-03-21T08:30:30.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.749+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68081a54e0 con 0x7f6808102240 2024-03-21T08:30:30.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.750+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68081038d0 con 0x7f6808102240 2024-03-21T08:30:30.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.752+0000 7f67ffe00700 1 -- 172.21.15.3:0/551932867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f67f8019070 con 0x7f6808102240 2024-03-21T08:30:30.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.754+0000 7f67ffe00700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f67ec0a30c0 0x7f67ec0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:30.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.755+0000 7f67ffe00700 1 -- 172.21.15.3:0/551932867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f67f80cf4b0 con 0x7f6808102240 2024-03-21T08:30:30.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.755+0000 7f6806a00700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f67ec0a30c0 0x7f67ec0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:30.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.755+0000 7f6806a00700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f67ec0a30c0 0x7f67ec0a5580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6808199df0 tx=0x7f67f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:30.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.759+0000 7f67ffe00700 1 -- 172.21.15.3:0/551932867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f67f809bd40 con 0x7f6808102240 2024-03-21T08:30:30.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:30 smithi003 ceph-mon[31675]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.982+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6808199b60 con 0x7f67ec0a30c0 2024-03-21T08:30:31.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:30.999+0000 7f67ffe00700 1 -- 172.21.15.3:0/551932867 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340359 (secure 0 0 0) 0x7f6808199b60 con 0x7f67ec0a30c0 2024-03-21T08:30:31.003 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f67ec0a30c0 msgr2=0x7f67ec0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f67ec0a30c0 0x7f67ec0a5580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6808199df0 tx=0x7f67f4009260 comp rx=0 tx=0).stop 2024-03-21T08:30:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102240 msgr2=0x7f680819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102240 0x7f680819e560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f67f800d950 tx=0x7f67f800dc60 comp rx=0 tx=0).stop 2024-03-21T08:30:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 shutdown_connections 2024-03-21T08:30:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f67ec0a30c0 0x7f67ec0a5580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6808102240 0x7f680819e560 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68081082b0 0x7f6808198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 --2- 172.21.15.3:0/551932867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6808102c00 0x7f680819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 >> 172.21.15.3:0/551932867 conn(0x7f6808079260 msgr2=0x7f6808100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 shutdown_connections 2024-03-21T08:30:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:31.006+0000 7f680e400700 1 -- 172.21.15.3:0/551932867 wait complete. 2024-03-21T08:30:31.011 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-03-21T08:30:31.177 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:30 smithi084 ceph-mon[35477]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:31.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:30 smithi171 ceph-mon[35388]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:31.741 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-03-21T08:30:29.383335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656936,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":639,"num_read_kb":444,"num_write":346,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372408,"kb_used_data":14864,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569416,"statfs":{"total":1151940427776,"available":1151559081984,"internally_reserved":0,"allocated":15220736,"data_stored":8358938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005986"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657944+0000","last_change":"2024-03-21T08:28:36.667851+0000","last_active":"2024-03-21T08:30:06.657944+0000","last_peered":"2024-03-21T08:30:06.657944+0000","last_clean":"2024-03-21T08:30:06.657944+0000","last_became_active":"2024-03-21T08:28:36.665713+0000","last_became_peered":"2024-03-21T08:28:36.665713+0000","last_unstale":"2024-03-21T08:30:06.657944+0000","last_undegraded":"2024-03-21T08:30:06.657944+0000","last_fullsized":"2024-03-21T08:30:06.657944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:02.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653021+0000","last_change":"2024-03-21T08:28:36.650986+0000","last_active":"2024-03-21T08:30:06.653021+0000","last_peered":"2024-03-21T08:30:06.653021+0000","last_clean":"2024-03-21T08:30:06.653021+0000","last_became_active":"2024-03-21T08:28:36.650633+0000","last_became_peered":"2024-03-21T08:28:36.650633+0000","last_unstale":"2024-03-21T08:30:06.653021+0000","last_undegraded":"2024-03-21T08:30:06.653021+0000","last_fullsized":"2024-03-21T08:30:06.653021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:40:20.801814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652575+0000","last_change":"2024-03-21T08:28:36.662984+0000","last_active":"2024-03-21T08:30:06.652575+0000","last_peered":"2024-03-21T08:30:06.652575+0000","last_clean":"2024-03-21T08:30:06.652575+0000","last_became_active":"2024-03-21T08:28:36.662657+0000","last_became_peered":"2024-03-21T08:28:36.662657+0000","last_unstale":"2024-03-21T08:30:06.652575+0000","last_undegraded":"2024-03-21T08:30:06.652575+0000","last_fullsized":"2024-03-21T08:30:06.652575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:41.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664118+0000","last_change":"2024-03-21T08:28:36.665958+0000","last_active":"2024-03-21T08:30:06.664118+0000","last_peered":"2024-03-21T08:30:06.664118+0000","last_clean":"2024-03-21T08:30:06.664118+0000","last_became_active":"2024-03-21T08:28:36.664309+0000","last_became_peered":"2024-03-21T08:28:36.664309+0000","last_unstale":"2024-03-21T08:30:06.664118+0000","last_undegraded":"2024-03-21T08:30:06.664118+0000","last_fullsized":"2024-03-21T08:30:06.664118+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:16.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651421+0000","last_change":"2024-03-21T08:28:36.657767+0000","last_active":"2024-03-21T08:30:06.651421+0000","last_peered":"2024-03-21T08:30:06.651421+0000","last_clean":"2024-03-21T08:30:06.651421+0000","last_became_active":"2024-03-21T08:28:36.657350+0000","last_became_peered":"2024-03-21T08:28:36.657350+0000","last_unstale":"2024-03-21T08:30:06.651421+0000","last_undegraded":"2024-03-21T08:30:06.651421+0000","last_fullsized":"2024-03-21T08:30:06.651421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:47.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645358+0000","last_change":"2024-03-21T08:29:06.916971+0000","last_active":"2024-03-21T08:30:06.645358+0000","last_peered":"2024-03-21T08:30:06.645358+0000","last_clean":"2024-03-21T08:30:06.645358+0000","last_became_active":"2024-03-21T08:29:06.915997+0000","last_became_peered":"2024-03-21T08:29:06.915997+0000","last_unstale":"2024-03-21T08:30:06.645358+0000","last_undegraded":"2024-03-21T08:30:06.645358+0000","last_fullsized":"2024-03-21T08:30:06.645358+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:02.525190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655807+0000","last_change":"2024-03-21T08:28:36.667787+0000","last_active":"2024-03-21T08:30:06.655807+0000","last_peered":"2024-03-21T08:30:06.655807+0000","last_clean":"2024-03-21T08:30:06.655807+0000","last_became_active":"2024-03-21T08:28:36.667451+0000","last_became_peered":"2024-03-21T08:28:36.667451+0000","last_unstale":"2024-03-21T08:30:06.655807+0000","last_undegraded":"2024-03-21T08:30:06.655807+0000","last_fullsized":"2024-03-21T08:30:06.655807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:01.874897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647039+0000","last_change":"2024-03-21T08:28:36.662445+0000","last_active":"2024-03-21T08:30:06.647039+0000","last_peered":"2024-03-21T08:30:06.647039+0000","last_clean":"2024-03-21T08:30:06.647039+0000","last_became_active":"2024-03-21T08:28:36.662113+0000","last_became_peered":"2024-03-21T08:28:36.662113+0000","last_unstale":"2024-03-21T08:30:06.647039+0000","last_undegraded":"2024-03-21T08:30:06.647039+0000","last_fullsized":"2024-03-21T08:30:06.647039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:21.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648804+0000","last_change":"2024-03-21T08:28:36.666084+0000","last_active":"2024-03-21T08:30:06.648804+0000","last_peered":"2024-03-21T08:30:06.648804+0000","last_clean":"2024-03-21T08:30:06.648804+0000","last_became_active":"2024-03-21T08:28:36.665354+0000","last_became_peered":"2024-03-21T08:28:36.665354+0000","last_unstale":"2024-03-21T08:30:06.648804+0000","last_undegraded":"2024-03-21T08:30:06.648804+0000","last_fullsized":"2024-03-21T08:30:06.648804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:02:50.160765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645243+0000","last_change":"2024-03-21T08:29:06.919231+0000","last_active":"2024-03-21T08:30:06.645243+0000","last_peered":"2024-03-21T08:30:06.645243+0000","last_clean":"2024-03-21T08:30:06.645243+0000","last_became_active":"2024-03-21T08:29:06.918199+0000","last_became_peered":"2024-03-21T08:29:06.918199+0000","last_unstale":"2024-03-21T08:30:06.645243+0000","last_undegraded":"2024-03-21T08:30:06.645243+0000","last_fullsized":"2024-03-21T08:30:06.645243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:44.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.658150+0000","last_change":"2024-03-21T08:28:36.656385+0000","last_active":"2024-03-21T08:30:06.658150+0000","last_peered":"2024-03-21T08:30:06.658150+0000","last_clean":"2024-03-21T08:30:06.658150+0000","last_became_active":"2024-03-21T08:28:36.656014+0000","last_became_peered":"2024-03-21T08:28:36.656014+0000","last_unstale":"2024-03-21T08:30:06.658150+0000","last_undegraded":"2024-03-21T08:30:06.658150+0000","last_fullsized":"2024-03-21T08:30:06.658150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:46.000495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663994+0000","last_change":"2024-03-21T08:28:36.666254+0000","last_active":"2024-03-21T08:30:06.663994+0000","last_peered":"2024-03-21T08:30:06.663994+0000","last_clean":"2024-03-21T08:30:06.663994+0000","last_became_active":"2024-03-21T08:28:36.664698+0000","last_became_peered":"2024-03-21T08:28:36.664698+0000","last_unstale":"2024-03-21T08:30:06.663994+0000","last_undegraded":"2024-03-21T08:30:06.663994+0000","last_fullsized":"2024-03-21T08:30:06.663994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:03:08.358159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642152+0000","last_change":"2024-03-21T08:29:06.919930+0000","last_active":"2024-03-21T08:30:06.642152+0000","last_peered":"2024-03-21T08:30:06.642152+0000","last_clean":"2024-03-21T08:30:06.642152+0000","last_became_active":"2024-03-21T08:29:06.917852+0000","last_became_peered":"2024-03-21T08:29:06.917852+0000","last_unstale":"2024-03-21T08:30:06.642152+0000","last_undegraded":"2024-03-21T08:30:06.642152+0000","last_fullsized":"2024-03-21T08:30:06.642152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:05:01.185388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652424+0000","last_change":"2024-03-21T08:28:36.655229+0000","last_active":"2024-03-21T08:30:06.652424+0000","last_peered":"2024-03-21T08:30:06.652424+0000","last_clean":"2024-03-21T08:30:06.652424+0000","last_became_active":"2024-03-21T08:28:36.653949+0000","last_became_peered":"2024-03-21T08:28:36.653949+0000","last_unstale":"2024-03-21T08:30:06.652424+0000","last_undegraded":"2024-03-21T08:30:06.652424+0000","last_fullsized":"2024-03-21T08:30:06.652424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:57:38.370897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652132+0000","last_change":"2024-03-21T08:28:36.664894+0000","last_active":"2024-03-21T08:30:06.652132+0000","last_peered":"2024-03-21T08:30:06.652132+0000","last_clean":"2024-03-21T08:30:06.652132+0000","last_became_active":"2024-03-21T08:28:36.664523+0000","last_became_peered":"2024-03-21T08:28:36.664523+0000","last_unstale":"2024-03-21T08:30:06.652132+0000","last_undegraded":"2024-03-21T08:30:06.652132+0000","last_fullsized":"2024-03-21T08:30:06.652132+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:52:54.147459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648449+0000","last_change":"2024-03-21T08:28:36.662374+0000","last_active":"2024-03-21T08:30:06.648449+0000","last_peered":"2024-03-21T08:30:06.648449+0000","last_clean":"2024-03-21T08:30:06.648449+0000","last_became_active":"2024-03-21T08:28:36.662066+0000","last_became_peered":"2024-03-21T08:28:36.662066+0000","last_unstale":"2024-03-21T08:30:06.648449+0000","last_undegraded":"2024-03-21T08:30:06.648449+0000","last_fullsized":"2024-03-21T08:30:06.648449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:06.082925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654939+0000","last_change":"2024-03-21T08:28:36.672172+0000","last_active":"2024-03-21T08:30:06.654939+0000","last_peered":"2024-03-21T08:30:06.654939+0000","last_clean":"2024-03-21T08:30:06.654939+0000","last_became_active":"2024-03-21T08:28:36.671657+0000","last_became_peered":"2024-03-21T08:28:36.671657+0000","last_unstale":"2024-03-21T08:30:06.654939+0000","last_undegraded":"2024-03-21T08:30:06.654939+0000","last_fullsized":"2024-03-21T08:30:06.654939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:56.892818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649159+0000","last_change":"2024-03-21T08:28:36.666973+0000","last_active":"2024-03-21T08:30:06.649159+0000","last_peered":"2024-03-21T08:30:06.649159+0000","last_clean":"2024-03-21T08:30:06.649159+0000","last_became_active":"2024-03-21T08:28:36.663770+0000","last_became_peered":"2024-03-21T08:28:36.663770+0000","last_unstale":"2024-03-21T08:30:06.649159+0000","last_undegraded":"2024-03-21T08:30:06.649159+0000","last_fullsized":"2024-03-21T08:30:06.649159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:54:26.701009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646127+0000","last_change":"2024-03-21T08:28:36.668793+0000","last_active":"2024-03-21T08:30:06.646127+0000","last_peered":"2024-03-21T08:30:06.646127+0000","last_clean":"2024-03-21T08:30:06.646127+0000","last_became_active":"2024-03-21T08:28:36.668452+0000","last_became_peered":"2024-03-21T08:28:36.668452+0000","last_unstale":"2024-03-21T08:30:06.646127+0000","last_undegraded":"2024-03-21T08:30:06.646127+0000","last_fullsized":"2024-03-21T08:30:06.646127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:45.191849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644857+0000","last_change":"2024-03-21T08:28:40.687618+0000","last_active":"2024-03-21T08:30:06.644857+0000","last_peered":"2024-03-21T08:30:06.644857+0000","last_clean":"2024-03-21T08:30:06.644857+0000","last_became_active":"2024-03-21T08:28:40.687155+0000","last_became_peered":"2024-03-21T08:28:40.687155+0000","last_unstale":"2024-03-21T08:30:06.644857+0000","last_undegraded":"2024-03-21T08:30:06.644857+0000","last_fullsized":"2024-03-21T08:30:06.644857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:08.014790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'9","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672911+0000","last_change":"2024-03-21T08:30:06.672911+0000","last_active":"2024-03-21T08:30:06.672911+0000","last_peered":"2024-03-21T08:30:06.672911+0000","last_clean":"2024-03-21T08:30:06.672911+0000","last_became_active":"2024-03-21T08:30:06.661974+0000","last_became_peered":"2024-03-21T08:30:06.661974+0000","last_unstale":"2024-03-21T08:30:06.672911+0000","last_undegraded":"2024-03-21T08:30:06.672911+0000","last_fullsized":"2024-03-21T08:30:06.672911+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:37:25.848279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647791+0000","last_change":"2024-03-21T08:28:36.670325+0000","last_active":"2024-03-21T08:30:06.647791+0000","last_peered":"2024-03-21T08:30:06.647791+0000","last_clean":"2024-03-21T08:30:06.647791+0000","last_became_active":"2024-03-21T08:28:36.669783+0000","last_became_peered":"2024-03-21T08:28:36.669783+0000","last_unstale":"2024-03-21T08:30:06.647791+0000","last_undegraded":"2024-03-21T08:30:06.647791+0000","last_fullsized":"2024-03-21T08:30:06.647791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:24.166567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663660+0000","last_change":"2024-03-21T08:28:40.689849+0000","last_active":"2024-03-21T08:30:06.663660+0000","last_peered":"2024-03-21T08:30:06.663660+0000","last_clean":"2024-03-21T08:30:06.663660+0000","last_became_active":"2024-03-21T08:28:40.689307+0000","last_became_peered":"2024-03-21T08:28:40.689307+0000","last_unstale":"2024-03-21T08:30:06.663660+0000","last_undegraded":"2024-03-21T08:30:06.663660+0000","last_fullsized":"2024-03-21T08:30:06.663660+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:34:05.660317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647896+0000","last_change":"2024-03-21T08:28:35.674685+0000","last_active":"2024-03-21T08:30:06.647896+0000","last_peered":"2024-03-21T08:30:06.647896+0000","last_clean":"2024-03-21T08:30:06.647896+0000","last_became_active":"2024-03-21T08:28:35.674312+0000","last_became_peered":"2024-03-21T08:28:35.674312+0000","last_unstale":"2024-03-21T08:30:06.647896+0000","last_undegraded":"2024-03-21T08:30:06.647896+0000","last_fullsized":"2024-03-21T08:30:06.647896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:27.644111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652988+0000","last_change":"2024-03-21T08:28:36.667744+0000","last_active":"2024-03-21T08:30:06.652988+0000","last_peered":"2024-03-21T08:30:06.652988+0000","last_clean":"2024-03-21T08:30:06.652988+0000","last_became_active":"2024-03-21T08:28:36.667279+0000","last_became_peered":"2024-03-21T08:28:36.667279+0000","last_unstale":"2024-03-21T08:30:06.652988+0000","last_undegraded":"2024-03-21T08:30:06.652988+0000","last_fullsized":"2024-03-21T08:30:06.652988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:14:24.974683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664150+0000","last_change":"2024-03-21T08:28:40.688864+0000","last_active":"2024-03-21T08:30:06.664150+0000","last_peered":"2024-03-21T08:30:06.664150+0000","last_clean":"2024-03-21T08:30:06.664150+0000","last_became_active":"2024-03-21T08:28:40.688029+0000","last_became_peered":"2024-03-21T08:28:40.688029+0000","last_unstale":"2024-03-21T08:30:06.664150+0000","last_undegraded":"2024-03-21T08:30:06.664150+0000","last_fullsized":"2024-03-21T08:30:06.664150+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:39.235807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.680224+0000","last_change":"2024-03-21T08:30:06.680224+0000","last_active":"2024-03-21T08:30:06.680224+0000","last_peered":"2024-03-21T08:30:06.680224+0000","last_clean":"2024-03-21T08:30:06.680224+0000","last_became_active":"2024-03-21T08:30:06.665898+0000","last_became_peered":"2024-03-21T08:30:06.665898+0000","last_unstale":"2024-03-21T08:30:06.680224+0000","last_undegraded":"2024-03-21T08:30:06.680224+0000","last_fullsized":"2024-03-21T08:30:06.680224+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:21.690576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650614+0000","last_change":"2024-03-21T08:28:36.667244+0000","last_active":"2024-03-21T08:30:06.650614+0000","last_peered":"2024-03-21T08:30:06.650614+0000","last_clean":"2024-03-21T08:30:06.650614+0000","last_became_active":"2024-03-21T08:28:36.666829+0000","last_became_peered":"2024-03-21T08:28:36.666829+0000","last_unstale":"2024-03-21T08:30:06.650614+0000","last_undegraded":"2024-03-21T08:30:06.650614+0000","last_fullsized":"2024-03-21T08:30:06.650614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:12:56.842084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657724+0000","last_change":"2024-03-21T08:28:40.692621+0000","last_active":"2024-03-21T08:30:06.657724+0000","last_peered":"2024-03-21T08:30:06.657724+0000","last_clean":"2024-03-21T08:30:06.657724+0000","last_became_active":"2024-03-21T08:28:40.691984+0000","last_became_peered":"2024-03-21T08:28:40.691984+0000","last_unstale":"2024-03-21T08:30:06.657724+0000","last_undegraded":"2024-03-21T08:30:06.657724+0000","last_fullsized":"2024-03-21T08:30:06.657724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:29.330702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651177+0000","last_change":"2024-03-21T08:28:35.676421+0000","last_active":"2024-03-21T08:30:06.651177+0000","last_peered":"2024-03-21T08:30:06.651177+0000","last_clean":"2024-03-21T08:30:06.651177+0000","last_became_active":"2024-03-21T08:28:35.674957+0000","last_became_peered":"2024-03-21T08:28:35.674957+0000","last_unstale":"2024-03-21T08:30:06.651177+0000","last_undegraded":"2024-03-21T08:30:06.651177+0000","last_fullsized":"2024-03-21T08:30:06.651177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:01:38.397757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666420+0000","last_change":"2024-03-21T08:28:36.664662+0000","last_active":"2024-03-21T08:30:06.666420+0000","last_peered":"2024-03-21T08:30:06.666420+0000","last_clean":"2024-03-21T08:30:06.666420+0000","last_became_active":"2024-03-21T08:28:36.664324+0000","last_became_peered":"2024-03-21T08:28:36.664324+0000","last_unstale":"2024-03-21T08:30:06.666420+0000","last_undegraded":"2024-03-21T08:30:06.666420+0000","last_fullsized":"2024-03-21T08:30:06.666420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:57.325869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645339+0000","last_change":"2024-03-21T08:28:40.676297+0000","last_active":"2024-03-21T08:30:06.645339+0000","last_peered":"2024-03-21T08:30:06.645339+0000","last_clean":"2024-03-21T08:30:06.645339+0000","last_became_active":"2024-03-21T08:28:40.675337+0000","last_became_peered":"2024-03-21T08:28:40.675337+0000","last_unstale":"2024-03-21T08:30:06.645339+0000","last_undegraded":"2024-03-21T08:30:06.645339+0000","last_fullsized":"2024-03-21T08:30:06.645339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:05.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":119,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:26.554429+0000","last_change":"2024-03-21T08:28:35.687651+0000","last_active":"2024-03-21T08:30:26.554429+0000","last_peered":"2024-03-21T08:30:26.554429+0000","last_clean":"2024-03-21T08:30:26.554429+0000","last_became_active":"2024-03-21T08:28:35.686284+0000","last_became_peered":"2024-03-21T08:28:35.686284+0000","last_unstale":"2024-03-21T08:30:26.554429+0000","last_undegraded":"2024-03-21T08:30:26.554429+0000","last_fullsized":"2024-03-21T08:30:26.554429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:56.121798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651915+0000","last_change":"2024-03-21T08:29:07.033436+0000","last_active":"2024-03-21T08:30:06.651915+0000","last_peered":"2024-03-21T08:30:06.651915+0000","last_clean":"2024-03-21T08:30:06.651915+0000","last_became_active":"2024-03-21T08:29:07.032012+0000","last_became_peered":"2024-03-21T08:29:07.032012+0000","last_unstale":"2024-03-21T08:30:06.651915+0000","last_undegraded":"2024-03-21T08:30:06.651915+0000","last_fullsized":"2024-03-21T08:30:06.651915+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:44:47.009078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645632+0000","last_change":"2024-03-21T08:29:06.882449+0000","last_active":"2024-03-21T08:30:06.645632+0000","last_peered":"2024-03-21T08:30:06.645632+0000","last_clean":"2024-03-21T08:30:06.645632+0000","last_became_active":"2024-03-21T08:29:06.880682+0000","last_became_peered":"2024-03-21T08:29:06.880682+0000","last_unstale":"2024-03-21T08:30:06.645632+0000","last_undegraded":"2024-03-21T08:30:06.645632+0000","last_fullsized":"2024-03-21T08:30:06.645632+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:09.464388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.659993+0000","last_change":"2024-03-21T08:30:06.659993+0000","last_active":"2024-03-21T08:30:06.659993+0000","last_peered":"2024-03-21T08:30:06.659993+0000","last_clean":"2024-03-21T08:30:06.659993+0000","last_became_active":"2024-03-21T08:30:06.653383+0000","last_became_peered":"2024-03-21T08:30:06.653383+0000","last_unstale":"2024-03-21T08:30:06.659993+0000","last_undegraded":"2024-03-21T08:30:06.659993+0000","last_fullsized":"2024-03-21T08:30:06.659993+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:43.836881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.667643+0000","last_change":"2024-03-21T08:28:36.670176+0000","last_active":"2024-03-21T08:30:06.667643+0000","last_peered":"2024-03-21T08:30:06.667643+0000","last_clean":"2024-03-21T08:30:06.667643+0000","last_became_active":"2024-03-21T08:28:36.669477+0000","last_became_peered":"2024-03-21T08:28:36.669477+0000","last_unstale":"2024-03-21T08:30:06.667643+0000","last_undegraded":"2024-03-21T08:30:06.667643+0000","last_fullsized":"2024-03-21T08:30:06.667643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:10:21.289484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654633+0000","last_change":"2024-03-21T08:29:06.784996+0000","last_active":"2024-03-21T08:30:06.654633+0000","last_peered":"2024-03-21T08:30:06.654633+0000","last_clean":"2024-03-21T08:30:06.654633+0000","last_became_active":"2024-03-21T08:29:06.783403+0000","last_became_peered":"2024-03-21T08:29:06.783403+0000","last_unstale":"2024-03-21T08:30:06.654633+0000","last_undegraded":"2024-03-21T08:30:06.654633+0000","last_fullsized":"2024-03-21T08:30:06.654633+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:18:59.912956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641054+0000","last_change":"2024-03-21T08:28:35.667502+0000","last_active":"2024-03-21T08:30:06.641054+0000","last_peered":"2024-03-21T08:30:06.641054+0000","last_clean":"2024-03-21T08:30:06.641054+0000","last_became_active":"2024-03-21T08:28:35.666667+0000","last_became_peered":"2024-03-21T08:28:35.666667+0000","last_unstale":"2024-03-21T08:30:06.641054+0000","last_undegraded":"2024-03-21T08:30:06.641054+0000","last_fullsized":"2024-03-21T08:30:06.641054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:42.196361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646415+0000","last_change":"2024-03-21T08:28:36.670855+0000","last_active":"2024-03-21T08:30:06.646415+0000","last_peered":"2024-03-21T08:30:06.646415+0000","last_clean":"2024-03-21T08:30:06.646415+0000","last_became_active":"2024-03-21T08:28:36.666012+0000","last_became_peered":"2024-03-21T08:28:36.666012+0000","last_unstale":"2024-03-21T08:30:06.646415+0000","last_undegraded":"2024-03-21T08:30:06.646415+0000","last_fullsized":"2024-03-21T08:30:06.646415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:19.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652857+0000","last_change":"2024-03-21T08:28:40.700037+0000","last_active":"2024-03-21T08:30:06.652857+0000","last_peered":"2024-03-21T08:30:06.652857+0000","last_clean":"2024-03-21T08:30:06.652857+0000","last_became_active":"2024-03-21T08:28:40.698127+0000","last_became_peered":"2024-03-21T08:28:40.698127+0000","last_unstale":"2024-03-21T08:30:06.652857+0000","last_undegraded":"2024-03-21T08:30:06.652857+0000","last_fullsized":"2024-03-21T08:30:06.652857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:48:57.135690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664454+0000","last_change":"2024-03-21T08:28:35.677071+0000","last_active":"2024-03-21T08:30:06.664454+0000","last_peered":"2024-03-21T08:30:06.664454+0000","last_clean":"2024-03-21T08:30:06.664454+0000","last_became_active":"2024-03-21T08:28:35.676652+0000","last_became_peered":"2024-03-21T08:28:35.676652+0000","last_unstale":"2024-03-21T08:30:06.664454+0000","last_undegraded":"2024-03-21T08:30:06.664454+0000","last_fullsized":"2024-03-21T08:30:06.664454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:24:54.097952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647294+0000","last_change":"2024-03-21T08:28:36.677981+0000","last_active":"2024-03-21T08:30:06.647294+0000","last_peered":"2024-03-21T08:30:06.647294+0000","last_clean":"2024-03-21T08:30:06.647294+0000","last_became_active":"2024-03-21T08:28:36.677512+0000","last_became_peered":"2024-03-21T08:28:36.677512+0000","last_unstale":"2024-03-21T08:30:06.647294+0000","last_undegraded":"2024-03-21T08:30:06.647294+0000","last_fullsized":"2024-03-21T08:30:06.647294+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:50.742928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646682+0000","last_change":"2024-03-21T08:28:40.692672+0000","last_active":"2024-03-21T08:30:06.646682+0000","last_peered":"2024-03-21T08:30:06.646682+0000","last_clean":"2024-03-21T08:30:06.646682+0000","last_became_active":"2024-03-21T08:28:40.688940+0000","last_became_peered":"2024-03-21T08:28:40.688940+0000","last_unstale":"2024-03-21T08:30:06.646682+0000","last_undegraded":"2024-03-21T08:30:06.646682+0000","last_fullsized":"2024-03-21T08:30:06.646682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:39.569709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":114,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:27.555744+0000","last_change":"2024-03-21T08:28:35.681970+0000","last_active":"2024-03-21T08:30:27.555744+0000","last_peered":"2024-03-21T08:30:27.555744+0000","last_clean":"2024-03-21T08:30:27.555744+0000","last_became_active":"2024-03-21T08:28:35.681252+0000","last_became_peered":"2024-03-21T08:28:35.681252+0000","last_unstale":"2024-03-21T08:30:27.555744+0000","last_undegraded":"2024-03-21T08:30:27.555744+0000","last_fullsized":"2024-03-21T08:30:27.555744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:46.345359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648867+0000","last_change":"2024-03-21T08:28:36.675419+0000","last_active":"2024-03-21T08:30:06.648867+0000","last_peered":"2024-03-21T08:30:06.648867+0000","last_clean":"2024-03-21T08:30:06.648867+0000","last_became_active":"2024-03-21T08:28:36.674039+0000","last_became_peered":"2024-03-21T08:28:36.674039+0000","last_unstale":"2024-03-21T08:30:06.648867+0000","last_undegraded":"2024-03-21T08:30:06.648867+0000","last_fullsized":"2024-03-21T08:30:06.648867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:09.667634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648534+0000","last_change":"2024-03-21T08:28:40.690076+0000","last_active":"2024-03-21T08:30:06.648534+0000","last_peered":"2024-03-21T08:30:06.648534+0000","last_clean":"2024-03-21T08:30:06.648534+0000","last_became_active":"2024-03-21T08:28:40.683817+0000","last_became_peered":"2024-03-21T08:28:40.683817+0000","last_unstale":"2024-03-21T08:30:06.648534+0000","last_undegraded":"2024-03-21T08:30:06.648534+0000","last_fullsized":"2024-03-21T08:30:06.648534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:33:00.948879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654037+0000","last_change":"2024-03-21T08:28:35.690693+0000","last_active":"2024-03-21T08:30:06.654037+0000","last_peered":"2024-03-21T08:30:06.654037+0000","last_clean":"2024-03-21T08:30:06.654037+0000","last_became_active":"2024-03-21T08:28:35.690250+0000","last_became_peered":"2024-03-21T08:28:35.690250+0000","last_unstale":"2024-03-21T08:30:06.654037+0000","last_undegraded":"2024-03-21T08:30:06.654037+0000","last_fullsized":"2024-03-21T08:30:06.654037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:44:21.307772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653094+0000","last_change":"2024-03-21T08:28:36.661872+0000","last_active":"2024-03-21T08:30:06.653094+0000","last_peered":"2024-03-21T08:30:06.653094+0000","last_clean":"2024-03-21T08:30:06.653094+0000","last_became_active":"2024-03-21T08:28:36.661483+0000","last_became_peered":"2024-03-21T08:28:36.661483+0000","last_unstale":"2024-03-21T08:30:06.653094+0000","last_undegraded":"2024-03-21T08:30:06.653094+0000","last_fullsized":"2024-03-21T08:30:06.653094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:03:35.630141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664958+0000","last_change":"2024-03-21T08:28:40.682753+0000","last_active":"2024-03-21T08:30:06.664958+0000","last_peered":"2024-03-21T08:30:06.664958+0000","last_clean":"2024-03-21T08:30:06.664958+0000","last_became_active":"2024-03-21T08:28:40.682278+0000","last_became_peered":"2024-03-21T08:28:40.682278+0000","last_unstale":"2024-03-21T08:30:06.664958+0000","last_undegraded":"2024-03-21T08:30:06.664958+0000","last_fullsized":"2024-03-21T08:30:06.664958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:20.280442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666203+0000","last_change":"2024-03-21T08:28:35.672524+0000","last_active":"2024-03-21T08:30:06.666203+0000","last_peered":"2024-03-21T08:30:06.666203+0000","last_clean":"2024-03-21T08:30:06.666203+0000","last_became_active":"2024-03-21T08:28:35.672051+0000","last_became_peered":"2024-03-21T08:28:35.672051+0000","last_unstale":"2024-03-21T08:30:06.666203+0000","last_undegraded":"2024-03-21T08:30:06.666203+0000","last_fullsized":"2024-03-21T08:30:06.666203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:53.336320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651096+0000","last_change":"2024-03-21T08:29:06.532172+0000","last_active":"2024-03-21T08:30:06.651096+0000","last_peered":"2024-03-21T08:30:06.651096+0000","last_clean":"2024-03-21T08:30:06.651096+0000","last_became_active":"2024-03-21T08:29:06.531439+0000","last_became_peered":"2024-03-21T08:29:06.531439+0000","last_unstale":"2024-03-21T08:30:06.651096+0000","last_undegraded":"2024-03-21T08:30:06.651096+0000","last_fullsized":"2024-03-21T08:30:06.651096+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:27.222359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648559+0000","last_change":"2024-03-21T08:28:40.682294+0000","last_active":"2024-03-21T08:30:06.648559+0000","last_peered":"2024-03-21T08:30:06.648559+0000","last_clean":"2024-03-21T08:30:06.648559+0000","last_became_active":"2024-03-21T08:28:40.681855+0000","last_became_peered":"2024-03-21T08:28:40.681855+0000","last_unstale":"2024-03-21T08:30:06.648559+0000","last_undegraded":"2024-03-21T08:30:06.648559+0000","last_fullsized":"2024-03-21T08:30:06.648559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:49.756322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'34","reported_seq":234,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:28.558857+0000","last_change":"2024-03-21T08:28:35.682947+0000","last_active":"2024-03-21T08:30:28.558857+0000","last_peered":"2024-03-21T08:30:28.558857+0000","last_clean":"2024-03-21T08:30:28.558857+0000","last_became_active":"2024-03-21T08:28:35.682395+0000","last_became_peered":"2024-03-21T08:28:35.682395+0000","last_unstale":"2024-03-21T08:30:28.558857+0000","last_undegraded":"2024-03-21T08:30:28.558857+0000","last_fullsized":"2024-03-21T08:30:28.558857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:07:48.072988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62333,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":169,"num_read_kb":146,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653737+0000","last_change":"2024-03-21T08:28:36.651140+0000","last_active":"2024-03-21T08:30:06.653737+0000","last_peered":"2024-03-21T08:30:06.653737+0000","last_clean":"2024-03-21T08:30:06.653737+0000","last_became_active":"2024-03-21T08:28:36.650551+0000","last_became_peered":"2024-03-21T08:28:36.650551+0000","last_unstale":"2024-03-21T08:30:06.653737+0000","last_undegraded":"2024-03-21T08:30:06.653737+0000","last_fullsized":"2024-03-21T08:30:06.653737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:53.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.667005+0000","last_change":"2024-03-21T08:28:40.691814+0000","last_active":"2024-03-21T08:30:06.667005+0000","last_peered":"2024-03-21T08:30:06.667005+0000","last_clean":"2024-03-21T08:30:06.667005+0000","last_became_active":"2024-03-21T08:28:40.691372+0000","last_became_peered":"2024-03-21T08:28:40.691372+0000","last_unstale":"2024-03-21T08:30:06.667005+0000","last_undegraded":"2024-03-21T08:30:06.667005+0000","last_fullsized":"2024-03-21T08:30:06.667005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:21:42.612037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645605+0000","last_change":"2024-03-21T08:28:35.687178+0000","last_active":"2024-03-21T08:30:06.645605+0000","last_peered":"2024-03-21T08:30:06.645605+0000","last_clean":"2024-03-21T08:30:06.645605+0000","last_became_active":"2024-03-21T08:28:35.686771+0000","last_became_peered":"2024-03-21T08:28:35.686771+0000","last_unstale":"2024-03-21T08:30:06.645605+0000","last_undegraded":"2024-03-21T08:30:06.645605+0000","last_fullsized":"2024-03-21T08:30:06.645605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:54.665866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641400+0000","last_change":"2024-03-21T08:28:36.662952+0000","last_active":"2024-03-21T08:30:06.641400+0000","last_peered":"2024-03-21T08:30:06.641400+0000","last_clean":"2024-03-21T08:30:06.641400+0000","last_became_active":"2024-03-21T08:28:36.662468+0000","last_became_peered":"2024-03-21T08:28:36.662468+0000","last_unstale":"2024-03-21T08:30:06.641400+0000","last_undegraded":"2024-03-21T08:30:06.641400+0000","last_fullsized":"2024-03-21T08:30:06.641400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:40.486998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656265+0000","last_change":"2024-03-21T08:28:40.693999+0000","last_active":"2024-03-21T08:30:06.656265+0000","last_peered":"2024-03-21T08:30:06.656265+0000","last_clean":"2024-03-21T08:30:06.656265+0000","last_became_active":"2024-03-21T08:28:40.693507+0000","last_became_peered":"2024-03-21T08:28:40.693507+0000","last_unstale":"2024-03-21T08:30:06.656265+0000","last_undegraded":"2024-03-21T08:30:06.656265+0000","last_fullsized":"2024-03-21T08:30:06.656265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:14.159379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.668638+0000","last_change":"2024-03-21T08:28:35.677536+0000","last_active":"2024-03-21T08:30:06.668638+0000","last_peered":"2024-03-21T08:30:06.668638+0000","last_clean":"2024-03-21T08:30:06.668638+0000","last_became_active":"2024-03-21T08:28:35.677100+0000","last_became_peered":"2024-03-21T08:28:35.677100+0000","last_unstale":"2024-03-21T08:30:06.668638+0000","last_undegraded":"2024-03-21T08:30:06.668638+0000","last_fullsized":"2024-03-21T08:30:06.668638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:52:47.275869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656491+0000","last_change":"2024-03-21T08:28:36.667190+0000","last_active":"2024-03-21T08:30:06.656491+0000","last_peered":"2024-03-21T08:30:06.656491+0000","last_clean":"2024-03-21T08:30:06.656491+0000","last_became_active":"2024-03-21T08:28:36.665336+0000","last_became_peered":"2024-03-21T08:28:36.665336+0000","last_unstale":"2024-03-21T08:30:06.656491+0000","last_undegraded":"2024-03-21T08:30:06.656491+0000","last_fullsized":"2024-03-21T08:30:06.656491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:21.888426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649671+0000","last_change":"2024-03-21T08:28:40.696582+0000","last_active":"2024-03-21T08:30:06.649671+0000","last_peered":"2024-03-21T08:30:06.649671+0000","last_clean":"2024-03-21T08:30:06.649671+0000","last_became_active":"2024-03-21T08:28:40.689583+0000","last_became_peered":"2024-03-21T08:28:40.689583+0000","last_unstale":"2024-03-21T08:30:06.649671+0000","last_undegraded":"2024-03-21T08:30:06.649671+0000","last_fullsized":"2024-03-21T08:30:06.649671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:57.329533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":112,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:23.546322+0000","last_change":"2024-03-21T08:28:35.675560+0000","last_active":"2024-03-21T08:30:23.546322+0000","last_peered":"2024-03-21T08:30:23.546322+0000","last_clean":"2024-03-21T08:30:23.546322+0000","last_became_active":"2024-03-21T08:28:35.673496+0000","last_became_peered":"2024-03-21T08:28:35.673496+0000","last_unstale":"2024-03-21T08:30:23.546322+0000","last_undegraded":"2024-03-21T08:30:23.546322+0000","last_fullsized":"2024-03-21T08:30:23.546322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:41:27.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653646+0000","last_change":"2024-03-21T08:28:36.660869+0000","last_active":"2024-03-21T08:30:06.653646+0000","last_peered":"2024-03-21T08:30:06.653646+0000","last_clean":"2024-03-21T08:30:06.653646+0000","last_became_active":"2024-03-21T08:28:36.659291+0000","last_became_peered":"2024-03-21T08:28:36.659291+0000","last_unstale":"2024-03-21T08:30:06.653646+0000","last_undegraded":"2024-03-21T08:30:06.653646+0000","last_fullsized":"2024-03-21T08:30:06.653646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:06.534214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.643105+0000","last_change":"2024-03-21T08:28:40.701351+0000","last_active":"2024-03-21T08:30:06.643105+0000","last_peered":"2024-03-21T08:30:06.643105+0000","last_clean":"2024-03-21T08:30:06.643105+0000","last_became_active":"2024-03-21T08:28:40.700989+0000","last_became_peered":"2024-03-21T08:28:40.700989+0000","last_unstale":"2024-03-21T08:30:06.643105+0000","last_undegraded":"2024-03-21T08:30:06.643105+0000","last_fullsized":"2024-03-21T08:30:06.643105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:35.697013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650372+0000","last_change":"2024-03-21T08:28:35.685024+0000","last_active":"2024-03-21T08:30:06.650372+0000","last_peered":"2024-03-21T08:30:06.650372+0000","last_clean":"2024-03-21T08:30:06.650372+0000","last_became_active":"2024-03-21T08:28:35.684597+0000","last_became_peered":"2024-03-21T08:28:35.684597+0000","last_unstale":"2024-03-21T08:30:06.650372+0000","last_undegraded":"2024-03-21T08:30:06.650372+0000","last_fullsized":"2024-03-21T08:30:06.650372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:09:46.262782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664546+0000","last_change":"2024-03-21T08:28:36.664688+0000","last_active":"2024-03-21T08:30:06.664546+0000","last_peered":"2024-03-21T08:30:06.664546+0000","last_clean":"2024-03-21T08:30:06.664546+0000","last_became_active":"2024-03-21T08:28:36.663990+0000","last_became_peered":"2024-03-21T08:28:36.663990+0000","last_unstale":"2024-03-21T08:30:06.664546+0000","last_undegraded":"2024-03-21T08:30:06.664546+0000","last_fullsized":"2024-03-21T08:30:06.664546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:39.656290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664326+0000","last_change":"2024-03-21T08:28:40.695269+0000","last_active":"2024-03-21T08:30:06.664326+0000","last_peered":"2024-03-21T08:30:06.664326+0000","last_clean":"2024-03-21T08:30:06.664326+0000","last_became_active":"2024-03-21T08:28:40.694820+0000","last_became_peered":"2024-03-21T08:28:40.694820+0000","last_unstale":"2024-03-21T08:30:06.664326+0000","last_undegraded":"2024-03-21T08:30:06.664326+0000","last_fullsized":"2024-03-21T08:30:06.664326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:47:25.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672662+0000","last_change":"2024-03-21T08:30:06.672662+0000","last_active":"2024-03-21T08:30:06.672662+0000","last_peered":"2024-03-21T08:30:06.672662+0000","last_clean":"2024-03-21T08:30:06.672662+0000","last_became_active":"2024-03-21T08:30:06.660161+0000","last_became_peered":"2024-03-21T08:30:06.660161+0000","last_unstale":"2024-03-21T08:30:06.672662+0000","last_undegraded":"2024-03-21T08:30:06.672662+0000","last_fullsized":"2024-03-21T08:30:06.672662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:41.163709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,4],"acting":[8,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655691+0000","last_change":"2024-03-21T08:28:36.667605+0000","last_active":"2024-03-21T08:30:06.655691+0000","last_peered":"2024-03-21T08:30:06.655691+0000","last_clean":"2024-03-21T08:30:06.655691+0000","last_became_active":"2024-03-21T08:28:36.667300+0000","last_became_peered":"2024-03-21T08:28:36.667300+0000","last_unstale":"2024-03-21T08:30:06.655691+0000","last_undegraded":"2024-03-21T08:30:06.655691+0000","last_fullsized":"2024-03-21T08:30:06.655691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:49.268935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652450+0000","last_change":"2024-03-21T08:28:40.884964+0000","last_active":"2024-03-21T08:30:06.652450+0000","last_peered":"2024-03-21T08:30:06.652450+0000","last_clean":"2024-03-21T08:30:06.652450+0000","last_became_active":"2024-03-21T08:28:40.883445+0000","last_became_peered":"2024-03-21T08:28:40.883445+0000","last_unstale":"2024-03-21T08:30:06.652450+0000","last_undegraded":"2024-03-21T08:30:06.652450+0000","last_fullsized":"2024-03-21T08:30:06.652450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:36:54.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647907+0000","last_change":"2024-03-21T08:28:35.682581+0000","last_active":"2024-03-21T08:30:06.647907+0000","last_peered":"2024-03-21T08:30:06.647907+0000","last_clean":"2024-03-21T08:30:06.647907+0000","last_became_active":"2024-03-21T08:28:35.682150+0000","last_became_peered":"2024-03-21T08:28:35.682150+0000","last_unstale":"2024-03-21T08:30:06.647907+0000","last_undegraded":"2024-03-21T08:30:06.647907+0000","last_fullsized":"2024-03-21T08:30:06.647907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:08.729710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645727+0000","last_change":"2024-03-21T08:28:36.666632+0000","last_active":"2024-03-21T08:30:06.645727+0000","last_peered":"2024-03-21T08:30:06.645727+0000","last_clean":"2024-03-21T08:30:06.645727+0000","last_became_active":"2024-03-21T08:28:36.666305+0000","last_became_peered":"2024-03-21T08:28:36.666305+0000","last_unstale":"2024-03-21T08:30:06.645727+0000","last_undegraded":"2024-03-21T08:30:06.645727+0000","last_fullsized":"2024-03-21T08:30:06.645727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:15.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642839+0000","last_change":"2024-03-21T08:28:40.700382+0000","last_active":"2024-03-21T08:30:06.642839+0000","last_peered":"2024-03-21T08:30:06.642839+0000","last_clean":"2024-03-21T08:30:06.642839+0000","last_became_active":"2024-03-21T08:28:40.699762+0000","last_became_peered":"2024-03-21T08:28:40.699762+0000","last_unstale":"2024-03-21T08:30:06.642839+0000","last_undegraded":"2024-03-21T08:30:06.642839+0000","last_fullsized":"2024-03-21T08:30:06.642839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:53.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657040+0000","last_change":"2024-03-21T08:28:35.680310+0000","last_active":"2024-03-21T08:30:06.657040+0000","last_peered":"2024-03-21T08:30:06.657040+0000","last_clean":"2024-03-21T08:30:06.657040+0000","last_became_active":"2024-03-21T08:28:35.678634+0000","last_became_peered":"2024-03-21T08:28:35.678634+0000","last_unstale":"2024-03-21T08:30:06.657040+0000","last_undegraded":"2024-03-21T08:30:06.657040+0000","last_fullsized":"2024-03-21T08:30:06.657040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:56:42.872521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644184+0000","last_change":"2024-03-21T08:28:36.669527+0000","last_active":"2024-03-21T08:30:06.644184+0000","last_peered":"2024-03-21T08:30:06.644184+0000","last_clean":"2024-03-21T08:30:06.644184+0000","last_became_active":"2024-03-21T08:28:36.669196+0000","last_became_peered":"2024-03-21T08:28:36.669196+0000","last_unstale":"2024-03-21T08:30:06.644184+0000","last_undegraded":"2024-03-21T08:30:06.644184+0000","last_fullsized":"2024-03-21T08:30:06.644184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:07:34.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644005+0000","last_change":"2024-03-21T08:28:40.882949+0000","last_active":"2024-03-21T08:30:06.644005+0000","last_peered":"2024-03-21T08:30:06.644005+0000","last_clean":"2024-03-21T08:30:06.644005+0000","last_became_active":"2024-03-21T08:28:40.882432+0000","last_became_peered":"2024-03-21T08:28:40.882432+0000","last_unstale":"2024-03-21T08:30:06.644005+0000","last_undegraded":"2024-03-21T08:30:06.644005+0000","last_fullsized":"2024-03-21T08:30:06.644005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:51.171365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664621+0000","last_change":"2024-03-21T08:28:36.665164+0000","last_active":"2024-03-21T08:30:06.664621+0000","last_peered":"2024-03-21T08:30:06.664621+0000","last_clean":"2024-03-21T08:30:06.664621+0000","last_became_active":"2024-03-21T08:28:36.664794+0000","last_became_peered":"2024-03-21T08:28:36.664794+0000","last_unstale":"2024-03-21T08:30:06.664621+0000","last_undegraded":"2024-03-21T08:30:06.664621+0000","last_fullsized":"2024-03-21T08:30:06.664621+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:25:39.355633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":115,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:27.555595+0000","last_change":"2024-03-21T08:28:35.671754+0000","last_active":"2024-03-21T08:30:27.555595+0000","last_peered":"2024-03-21T08:30:27.555595+0000","last_clean":"2024-03-21T08:30:27.555595+0000","last_became_active":"2024-03-21T08:28:35.671363+0000","last_became_peered":"2024-03-21T08:28:35.671363+0000","last_unstale":"2024-03-21T08:30:27.555595+0000","last_undegraded":"2024-03-21T08:30:27.555595+0000","last_fullsized":"2024-03-21T08:30:27.555595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:22:11.147325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657802+0000","last_change":"2024-03-21T08:28:40.689038+0000","last_active":"2024-03-21T08:30:06.657802+0000","last_peered":"2024-03-21T08:30:06.657802+0000","last_clean":"2024-03-21T08:30:06.657802+0000","last_became_active":"2024-03-21T08:28:40.688538+0000","last_became_peered":"2024-03-21T08:28:40.688538+0000","last_unstale":"2024-03-21T08:30:06.657802+0000","last_undegraded":"2024-03-21T08:30:06.657802+0000","last_fullsized":"2024-03-21T08:30:06.657802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:14.833594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646707+0000","last_change":"2024-03-21T08:28:36.667725+0000","last_active":"2024-03-21T08:30:06.646707+0000","last_peered":"2024-03-21T08:30:06.646707+0000","last_clean":"2024-03-21T08:30:06.646707+0000","last_became_active":"2024-03-21T08:28:36.667303+0000","last_became_peered":"2024-03-21T08:28:36.667303+0000","last_unstale":"2024-03-21T08:30:06.646707+0000","last_undegraded":"2024-03-21T08:30:06.646707+0000","last_fullsized":"2024-03-21T08:30:06.646707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:32:18.990943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664189+0000","last_change":"2024-03-21T08:28:35.673497+0000","last_active":"2024-03-21T08:30:06.664189+0000","last_peered":"2024-03-21T08:30:06.664189+0000","last_clean":"2024-03-21T08:30:06.664189+0000","last_became_active":"2024-03-21T08:28:35.673045+0000","last_became_peered":"2024-03-21T08:28:35.673045+0000","last_unstale":"2024-03-21T08:30:06.664189+0000","last_undegraded":"2024-03-21T08:30:06.664189+0000","last_fullsized":"2024-03-21T08:30:06.664189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:23.972672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653331+0000","last_change":"2024-03-21T08:26:58.139094+0000","last_active":"2024-03-21T08:30:06.653331+0000","last_peered":"2024-03-21T08:30:06.653331+0000","last_clean":"2024-03-21T08:30:06.653331+0000","last_became_active":"2024-03-21T08:26:58.137648+0000","last_became_peered":"2024-03-21T08:26:58.137648+0000","last_unstale":"2024-03-21T08:30:06.653331+0000","last_undegraded":"2024-03-21T08:30:06.653331+0000","last_fullsized":"2024-03-21T08:30:06.653331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_clean_scrub_stamp":"2024-03-21T08:22:17.977314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:57.950111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665685+0000","last_change":"2024-03-21T08:28:40.887865+0000","last_active":"2024-03-21T08:30:06.665685+0000","last_peered":"2024-03-21T08:30:06.665685+0000","last_clean":"2024-03-21T08:30:06.665685+0000","last_became_active":"2024-03-21T08:28:40.885538+0000","last_became_peered":"2024-03-21T08:28:40.885538+0000","last_unstale":"2024-03-21T08:30:06.665685+0000","last_undegraded":"2024-03-21T08:30:06.665685+0000","last_fullsized":"2024-03-21T08:30:06.665685+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:26.532005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641113+0000","last_change":"2024-03-21T08:28:36.668600+0000","last_active":"2024-03-21T08:30:06.641113+0000","last_peered":"2024-03-21T08:30:06.641113+0000","last_clean":"2024-03-21T08:30:06.641113+0000","last_became_active":"2024-03-21T08:28:36.668188+0000","last_became_peered":"2024-03-21T08:28:36.668188+0000","last_unstale":"2024-03-21T08:30:06.641113+0000","last_undegraded":"2024-03-21T08:30:06.641113+0000","last_fullsized":"2024-03-21T08:30:06.641113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:13.536003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.674753+0000","last_change":"2024-03-21T08:30:06.673425+0000","last_active":"2024-03-21T08:30:06.674753+0000","last_peered":"2024-03-21T08:30:06.674753+0000","last_clean":"2024-03-21T08:30:06.674753+0000","last_became_active":"2024-03-21T08:30:06.666858+0000","last_became_peered":"2024-03-21T08:30:06.666858+0000","last_unstale":"2024-03-21T08:30:06.674753+0000","last_undegraded":"2024-03-21T08:30:06.674753+0000","last_fullsized":"2024-03-21T08:30:06.674753+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:37.159978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655430+0000","last_change":"2024-03-21T08:29:07.032622+0000","last_active":"2024-03-21T08:30:06.655430+0000","last_peered":"2024-03-21T08:30:06.655430+0000","last_clean":"2024-03-21T08:30:06.655430+0000","last_became_active":"2024-03-21T08:29:07.030884+0000","last_became_peered":"2024-03-21T08:29:07.030884+0000","last_unstale":"2024-03-21T08:30:06.655430+0000","last_undegraded":"2024-03-21T08:30:06.655430+0000","last_fullsized":"2024-03-21T08:30:06.655430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:47.615776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653948+0000","last_change":"2024-03-21T08:28:35.672948+0000","last_active":"2024-03-21T08:30:06.653948+0000","last_peered":"2024-03-21T08:30:06.653948+0000","last_clean":"2024-03-21T08:30:06.653948+0000","last_became_active":"2024-03-21T08:28:35.671576+0000","last_became_peered":"2024-03-21T08:28:35.671576+0000","last_unstale":"2024-03-21T08:30:06.653948+0000","last_undegraded":"2024-03-21T08:30:06.653948+0000","last_fullsized":"2024-03-21T08:30:06.653948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:36:25.620392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642652+0000","last_change":"2024-03-21T08:28:36.671173+0000","last_active":"2024-03-21T08:30:06.642652+0000","last_peered":"2024-03-21T08:30:06.642652+0000","last_clean":"2024-03-21T08:30:06.642652+0000","last_became_active":"2024-03-21T08:28:36.666264+0000","last_became_peered":"2024-03-21T08:28:36.666264+0000","last_unstale":"2024-03-21T08:30:06.642652+0000","last_undegraded":"2024-03-21T08:30:06.642652+0000","last_fullsized":"2024-03-21T08:30:06.642652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:35.081487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646298+0000","last_change":"2024-03-21T08:28:40.883801+0000","last_active":"2024-03-21T08:30:06.646298+0000","last_peered":"2024-03-21T08:30:06.646298+0000","last_clean":"2024-03-21T08:30:06.646298+0000","last_became_active":"2024-03-21T08:28:40.882556+0000","last_became_peered":"2024-03-21T08:28:40.882556+0000","last_unstale":"2024-03-21T08:30:06.646298+0000","last_undegraded":"2024-03-21T08:30:06.646298+0000","last_fullsized":"2024-03-21T08:30:06.646298+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:24.211625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'8","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649233+0000","last_change":"2024-03-21T08:28:35.668657+0000","last_active":"2024-03-21T08:30:06.649233+0000","last_peered":"2024-03-21T08:30:06.649233+0000","last_clean":"2024-03-21T08:30:06.649233+0000","last_became_active":"2024-03-21T08:28:35.668284+0000","last_became_peered":"2024-03-21T08:28:35.668284+0000","last_unstale":"2024-03-21T08:30:06.649233+0000","last_undegraded":"2024-03-21T08:30:06.649233+0000","last_fullsized":"2024-03-21T08:30:06.649233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:26:07.791388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648082+0000","last_change":"2024-03-21T08:28:36.652383+0000","last_active":"2024-03-21T08:30:06.648082+0000","last_peered":"2024-03-21T08:30:06.648082+0000","last_clean":"2024-03-21T08:30:06.648082+0000","last_became_active":"2024-03-21T08:28:36.652046+0000","last_became_peered":"2024-03-21T08:28:36.652046+0000","last_unstale":"2024-03-21T08:30:06.648082+0000","last_undegraded":"2024-03-21T08:30:06.648082+0000","last_fullsized":"2024-03-21T08:30:06.648082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:11:08.699391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644328+0000","last_change":"2024-03-21T08:28:40.686122+0000","last_active":"2024-03-21T08:30:06.644328+0000","last_peered":"2024-03-21T08:30:06.644328+0000","last_clean":"2024-03-21T08:30:06.644328+0000","last_became_active":"2024-03-21T08:28:40.685736+0000","last_became_peered":"2024-03-21T08:28:40.685736+0000","last_unstale":"2024-03-21T08:30:06.644328+0000","last_undegraded":"2024-03-21T08:30:06.644328+0000","last_fullsized":"2024-03-21T08:30:06.644328+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:52.941896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665195+0000","last_change":"2024-03-21T08:28:35.675197+0000","last_active":"2024-03-21T08:30:06.665195+0000","last_peered":"2024-03-21T08:30:06.665195+0000","last_clean":"2024-03-21T08:30:06.665195+0000","last_became_active":"2024-03-21T08:28:35.674688+0000","last_became_peered":"2024-03-21T08:28:35.674688+0000","last_unstale":"2024-03-21T08:30:06.665195+0000","last_undegraded":"2024-03-21T08:30:06.665195+0000","last_fullsized":"2024-03-21T08:30:06.665195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:01:11.489294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653149+0000","last_change":"2024-03-21T08:28:36.668160+0000","last_active":"2024-03-21T08:30:06.653149+0000","last_peered":"2024-03-21T08:30:06.653149+0000","last_clean":"2024-03-21T08:30:06.653149+0000","last_became_active":"2024-03-21T08:28:36.665152+0000","last_became_peered":"2024-03-21T08:28:36.665152+0000","last_unstale":"2024-03-21T08:30:06.653149+0000","last_undegraded":"2024-03-21T08:30:06.653149+0000","last_fullsized":"2024-03-21T08:30:06.653149+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:43:41.803822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663541+0000","last_change":"2024-03-21T08:28:40.884365+0000","last_active":"2024-03-21T08:30:06.663541+0000","last_peered":"2024-03-21T08:30:06.663541+0000","last_clean":"2024-03-21T08:30:06.663541+0000","last_became_active":"2024-03-21T08:28:40.883654+0000","last_became_peered":"2024-03-21T08:28:40.883654+0000","last_unstale":"2024-03-21T08:30:06.663541+0000","last_undegraded":"2024-03-21T08:30:06.663541+0000","last_fullsized":"2024-03-21T08:30:06.663541+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:46:43.888121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672111+0000","last_change":"2024-03-21T08:30:06.672111+0000","last_active":"2024-03-21T08:30:06.672111+0000","last_peered":"2024-03-21T08:30:06.672111+0000","last_clean":"2024-03-21T08:30:06.672111+0000","last_became_active":"2024-03-21T08:30:06.665627+0000","last_became_peered":"2024-03-21T08:30:06.665627+0000","last_unstale":"2024-03-21T08:30:06.672111+0000","last_undegraded":"2024-03-21T08:30:06.672111+0000","last_fullsized":"2024-03-21T08:30:06.672111+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:05.512276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650465+0000","last_change":"2024-03-21T08:28:36.653797+0000","last_active":"2024-03-21T08:30:06.650465+0000","last_peered":"2024-03-21T08:30:06.650465+0000","last_clean":"2024-03-21T08:30:06.650465+0000","last_became_active":"2024-03-21T08:28:36.651872+0000","last_became_peered":"2024-03-21T08:28:36.651872+0000","last_unstale":"2024-03-21T08:30:06.650465+0000","last_undegraded":"2024-03-21T08:30:06.650465+0000","last_fullsized":"2024-03-21T08:30:06.650465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:09.467573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649671+0000","last_change":"2024-03-21T08:28:40.692996+0000","last_active":"2024-03-21T08:30:06.649671+0000","last_peered":"2024-03-21T08:30:06.649671+0000","last_clean":"2024-03-21T08:30:06.649671+0000","last_became_active":"2024-03-21T08:28:40.687598+0000","last_became_peered":"2024-03-21T08:28:40.687598+0000","last_unstale":"2024-03-21T08:30:06.649671+0000","last_undegraded":"2024-03-21T08:30:06.649671+0000","last_fullsized":"2024-03-21T08:30:06.649671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:09.557272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645874+0000","last_change":"2024-03-21T08:28:35.672567+0000","last_active":"2024-03-21T08:30:06.645874+0000","last_peered":"2024-03-21T08:30:06.645874+0000","last_clean":"2024-03-21T08:30:06.645874+0000","last_became_active":"2024-03-21T08:28:35.671849+0000","last_became_peered":"2024-03-21T08:28:35.671849+0000","last_unstale":"2024-03-21T08:30:06.645874+0000","last_undegraded":"2024-03-21T08:30:06.645874+0000","last_fullsized":"2024-03-21T08:30:06.645874+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:13.050227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656580+0000","last_change":"2024-03-21T08:28:36.664110+0000","last_active":"2024-03-21T08:30:06.656580+0000","last_peered":"2024-03-21T08:30:06.656580+0000","last_clean":"2024-03-21T08:30:06.656580+0000","last_became_active":"2024-03-21T08:28:36.663297+0000","last_became_peered":"2024-03-21T08:28:36.663297+0000","last_unstale":"2024-03-21T08:30:06.656580+0000","last_undegraded":"2024-03-21T08:30:06.656580+0000","last_fullsized":"2024-03-21T08:30:06.656580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:55.450399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663781+0000","last_change":"2024-03-21T08:28:40.682371+0000","last_active":"2024-03-21T08:30:06.663781+0000","last_peered":"2024-03-21T08:30:06.663781+0000","last_clean":"2024-03-21T08:30:06.663781+0000","last_became_active":"2024-03-21T08:28:40.681786+0000","last_became_peered":"2024-03-21T08:28:40.681786+0000","last_unstale":"2024-03-21T08:30:06.663781+0000","last_undegraded":"2024-03-21T08:30:06.663781+0000","last_fullsized":"2024-03-21T08:30:06.663781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:15:38.130807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644923+0000","last_change":"2024-03-21T08:28:35.685312+0000","last_active":"2024-03-21T08:30:06.644923+0000","last_peered":"2024-03-21T08:30:06.644923+0000","last_clean":"2024-03-21T08:30:06.644923+0000","last_became_active":"2024-03-21T08:28:35.684327+0000","last_became_peered":"2024-03-21T08:28:35.684327+0000","last_unstale":"2024-03-21T08:30:06.644923+0000","last_undegraded":"2024-03-21T08:30:06.644923+0000","last_fullsized":"2024-03-21T08:30:06.644923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:12.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647398+0000","last_change":"2024-03-21T08:28:36.671186+0000","last_active":"2024-03-21T08:30:06.647398+0000","last_peered":"2024-03-21T08:30:06.647398+0000","last_clean":"2024-03-21T08:30:06.647398+0000","last_became_active":"2024-03-21T08:28:36.662726+0000","last_became_peered":"2024-03-21T08:28:36.662726+0000","last_unstale":"2024-03-21T08:30:06.647398+0000","last_undegraded":"2024-03-21T08:30:06.647398+0000","last_fullsized":"2024-03-21T08:30:06.647398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:29.129014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663706+0000","last_change":"2024-03-21T08:28:40.885203+0000","last_active":"2024-03-21T08:30:06.663706+0000","last_peered":"2024-03-21T08:30:06.663706+0000","last_clean":"2024-03-21T08:30:06.663706+0000","last_became_active":"2024-03-21T08:28:40.883810+0000","last_became_peered":"2024-03-21T08:28:40.883810+0000","last_unstale":"2024-03-21T08:30:06.663706+0000","last_undegraded":"2024-03-21T08:30:06.663706+0000","last_fullsized":"2024-03-21T08:30:06.663706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:12:58.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651040+0000","last_change":"2024-03-21T08:28:35.690494+0000","last_active":"2024-03-21T08:30:06.651040+0000","last_peered":"2024-03-21T08:30:06.651040+0000","last_clean":"2024-03-21T08:30:06.651040+0000","last_became_active":"2024-03-21T08:28:35.690071+0000","last_became_peered":"2024-03-21T08:28:35.690071+0000","last_unstale":"2024-03-21T08:30:06.651040+0000","last_undegraded":"2024-03-21T08:30:06.651040+0000","last_fullsized":"2024-03-21T08:30:06.651040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:04:46.713006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656927+0000","last_change":"2024-03-21T08:28:36.665135+0000","last_active":"2024-03-21T08:30:06.656927+0000","last_peered":"2024-03-21T08:30:06.656927+0000","last_clean":"2024-03-21T08:30:06.656927+0000","last_became_active":"2024-03-21T08:28:36.663700+0000","last_became_peered":"2024-03-21T08:28:36.663700+0000","last_unstale":"2024-03-21T08:30:06.656927+0000","last_undegraded":"2024-03-21T08:30:06.656927+0000","last_fullsized":"2024-03-21T08:30:06.656927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:50:00.604958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649936+0000","last_change":"2024-03-21T08:28:40.696076+0000","last_active":"2024-03-21T08:30:06.649936+0000","last_peered":"2024-03-21T08:30:06.649936+0000","last_clean":"2024-03-21T08:30:06.649936+0000","last_became_active":"2024-03-21T08:28:40.695556+0000","last_became_peered":"2024-03-21T08:28:40.695556+0000","last_unstale":"2024-03-21T08:30:06.649936+0000","last_undegraded":"2024-03-21T08:30:06.649936+0000","last_fullsized":"2024-03-21T08:30:06.649936+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:42.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646280+0000","last_change":"2024-03-21T08:28:35.685545+0000","last_active":"2024-03-21T08:30:06.646280+0000","last_peered":"2024-03-21T08:30:06.646280+0000","last_clean":"2024-03-21T08:30:06.646280+0000","last_became_active":"2024-03-21T08:28:35.684795+0000","last_became_peered":"2024-03-21T08:28:35.684795+0000","last_unstale":"2024-03-21T08:30:06.646280+0000","last_undegraded":"2024-03-21T08:30:06.646280+0000","last_fullsized":"2024-03-21T08:30:06.646280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:40.308014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.658263+0000","last_change":"2024-03-21T08:28:36.656276+0000","last_active":"2024-03-21T08:30:06.658263+0000","last_peered":"2024-03-21T08:30:06.658263+0000","last_clean":"2024-03-21T08:30:06.658263+0000","last_became_active":"2024-03-21T08:28:36.655871+0000","last_became_peered":"2024-03-21T08:28:36.655871+0000","last_unstale":"2024-03-21T08:30:06.658263+0000","last_undegraded":"2024-03-21T08:30:06.658263+0000","last_fullsized":"2024-03-21T08:30:06.658263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:22.362165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651916+0000","last_change":"2024-03-21T08:29:06.783155+0000","last_active":"2024-03-21T08:30:06.651916+0000","last_peered":"2024-03-21T08:30:06.651916+0000","last_clean":"2024-03-21T08:30:06.651916+0000","last_became_active":"2024-03-21T08:29:06.782546+0000","last_became_peered":"2024-03-21T08:29:06.782546+0000","last_unstale":"2024-03-21T08:30:06.651916+0000","last_undegraded":"2024-03-21T08:30:06.651916+0000","last_fullsized":"2024-03-21T08:30:06.651916+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:16.489888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666587+0000","last_change":"2024-03-21T08:28:35.672783+0000","last_active":"2024-03-21T08:30:06.666587+0000","last_peered":"2024-03-21T08:30:06.666587+0000","last_clean":"2024-03-21T08:30:06.666587+0000","last_became_active":"2024-03-21T08:28:35.672161+0000","last_became_peered":"2024-03-21T08:28:35.672161+0000","last_unstale":"2024-03-21T08:30:06.666587+0000","last_undegraded":"2024-03-21T08:30:06.666587+0000","last_fullsized":"2024-03-21T08:30:06.666587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:37.180625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657422+0000","last_change":"2024-03-21T08:28:36.659447+0000","last_active":"2024-03-21T08:30:06.657422+0000","last_peered":"2024-03-21T08:30:06.657422+0000","last_clean":"2024-03-21T08:30:06.657422+0000","last_became_active":"2024-03-21T08:28:36.657773+0000","last_became_peered":"2024-03-21T08:28:36.657773+0000","last_unstale":"2024-03-21T08:30:06.657422+0000","last_undegraded":"2024-03-21T08:30:06.657422+0000","last_fullsized":"2024-03-21T08:30:06.657422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:05.324498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664177+0000","last_change":"2024-03-21T08:28:40.679806+0000","last_active":"2024-03-21T08:30:06.664177+0000","last_peered":"2024-03-21T08:30:06.664177+0000","last_clean":"2024-03-21T08:30:06.664177+0000","last_became_active":"2024-03-21T08:28:40.679329+0000","last_became_peered":"2024-03-21T08:28:40.679329+0000","last_unstale":"2024-03-21T08:30:06.664177+0000","last_undegraded":"2024-03-21T08:30:06.664177+0000","last_fullsized":"2024-03-21T08:30:06.664177+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:30:58.297581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.675069+0000","last_change":"2024-03-21T08:30:06.675069+0000","last_active":"2024-03-21T08:30:06.675069+0000","last_peered":"2024-03-21T08:30:06.675069+0000","last_clean":"2024-03-21T08:30:06.675069+0000","last_became_active":"2024-03-21T08:30:06.665699+0000","last_became_peered":"2024-03-21T08:30:06.665699+0000","last_unstale":"2024-03-21T08:30:06.675069+0000","last_undegraded":"2024-03-21T08:30:06.675069+0000","last_fullsized":"2024-03-21T08:30:06.675069+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:30:07.336423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655166+0000","last_change":"2024-03-21T08:28:36.649194+0000","last_active":"2024-03-21T08:30:06.655166+0000","last_peered":"2024-03-21T08:30:06.655166+0000","last_clean":"2024-03-21T08:30:06.655166+0000","last_became_active":"2024-03-21T08:28:36.648663+0000","last_became_peered":"2024-03-21T08:28:36.648663+0000","last_unstale":"2024-03-21T08:30:06.655166+0000","last_undegraded":"2024-03-21T08:30:06.655166+0000","last_fullsized":"2024-03-21T08:30:06.655166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:16.296886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656934+0000","last_change":"2024-03-21T08:28:36.664473+0000","last_active":"2024-03-21T08:30:06.656934+0000","last_peered":"2024-03-21T08:30:06.656934+0000","last_clean":"2024-03-21T08:30:06.656934+0000","last_became_active":"2024-03-21T08:28:36.663785+0000","last_became_peered":"2024-03-21T08:28:36.663785+0000","last_unstale":"2024-03-21T08:30:06.656934+0000","last_undegraded":"2024-03-21T08:30:06.656934+0000","last_fullsized":"2024-03-21T08:30:06.656934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:00.409224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645357+0000","last_change":"2024-03-21T08:28:36.668372+0000","last_active":"2024-03-21T08:30:06.645357+0000","last_peered":"2024-03-21T08:30:06.645357+0000","last_clean":"2024-03-21T08:30:06.645357+0000","last_became_active":"2024-03-21T08:28:36.666227+0000","last_became_peered":"2024-03-21T08:28:36.666227+0000","last_unstale":"2024-03-21T08:30:06.645357+0000","last_undegraded":"2024-03-21T08:30:06.645357+0000","last_fullsized":"2024-03-21T08:30:06.645357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:40:56.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644522+0000","last_change":"2024-03-21T08:28:36.670770+0000","last_active":"2024-03-21T08:30:06.644522+0000","last_peered":"2024-03-21T08:30:06.644522+0000","last_clean":"2024-03-21T08:30:06.644522+0000","last_became_active":"2024-03-21T08:28:36.670467+0000","last_became_peered":"2024-03-21T08:28:36.670467+0000","last_unstale":"2024-03-21T08:30:06.644522+0000","last_undegraded":"2024-03-21T08:30:06.644522+0000","last_fullsized":"2024-03-21T08:30:06.644522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:48.075189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652220+0000","last_change":"2024-03-21T08:28:36.669395+0000","last_active":"2024-03-21T08:30:06.652220+0000","last_peered":"2024-03-21T08:30:06.652220+0000","last_clean":"2024-03-21T08:30:06.652220+0000","last_became_active":"2024-03-21T08:28:36.668788+0000","last_became_peered":"2024-03-21T08:28:36.668788+0000","last_unstale":"2024-03-21T08:30:06.652220+0000","last_undegraded":"2024-03-21T08:30:06.652220+0000","last_fullsized":"2024-03-21T08:30:06.652220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:03.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655160+0000","last_change":"2024-03-21T08:30:06.655160+0000","last_active":"2024-03-21T08:30:06.655160+0000","last_peered":"2024-03-21T08:30:06.655160+0000","last_clean":"2024-03-21T08:30:06.655160+0000","last_became_active":"2024-03-21T08:30:06.654744+0000","last_became_peered":"2024-03-21T08:30:06.654744+0000","last_unstale":"2024-03-21T08:30:06.655160+0000","last_undegraded":"2024-03-21T08:30:06.655160+0000","last_fullsized":"2024-03-21T08:30:06.655160+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:19:54.690562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655090+0000","last_change":"2024-03-21T08:28:36.666836+0000","last_active":"2024-03-21T08:30:06.655090+0000","last_peered":"2024-03-21T08:30:06.655090+0000","last_clean":"2024-03-21T08:30:06.655090+0000","last_became_active":"2024-03-21T08:28:36.666538+0000","last_became_peered":"2024-03-21T08:28:36.666538+0000","last_unstale":"2024-03-21T08:30:06.655090+0000","last_undegraded":"2024-03-21T08:30:06.655090+0000","last_fullsized":"2024-03-21T08:30:06.655090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:43.353269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646323+0000","last_change":"2024-03-21T08:28:36.667823+0000","last_active":"2024-03-21T08:30:06.646323+0000","last_peered":"2024-03-21T08:30:06.646323+0000","last_clean":"2024-03-21T08:30:06.646323+0000","last_became_active":"2024-03-21T08:28:36.666096+0000","last_became_peered":"2024-03-21T08:28:36.666096+0000","last_unstale":"2024-03-21T08:30:06.646323+0000","last_undegraded":"2024-03-21T08:30:06.646323+0000","last_fullsized":"2024-03-21T08:30:06.646323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:58.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648234+0000","last_change":"2024-03-21T08:28:36.672164+0000","last_active":"2024-03-21T08:30:06.648234+0000","last_peered":"2024-03-21T08:30:06.648234+0000","last_clean":"2024-03-21T08:30:06.648234+0000","last_became_active":"2024-03-21T08:28:36.671812+0000","last_became_peered":"2024-03-21T08:28:36.671812+0000","last_unstale":"2024-03-21T08:30:06.648234+0000","last_undegraded":"2024-03-21T08:30:06.648234+0000","last_fullsized":"2024-03-21T08:30:06.648234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:24:19.687108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664568+0000","last_change":"2024-03-21T08:28:36.675056+0000","last_active":"2024-03-21T08:30:06.664568+0000","last_peered":"2024-03-21T08:30:06.664568+0000","last_clean":"2024-03-21T08:30:06.664568+0000","last_became_active":"2024-03-21T08:28:36.674586+0000","last_became_peered":"2024-03-21T08:28:36.674586+0000","last_unstale":"2024-03-21T08:30:06.664568+0000","last_undegraded":"2024-03-21T08:30:06.664568+0000","last_fullsized":"2024-03-21T08:30:06.664568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:04:51.207066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645873+0000","last_change":"2024-03-21T08:28:36.666370+0000","last_active":"2024-03-21T08:30:06.645873+0000","last_peered":"2024-03-21T08:30:06.645873+0000","last_clean":"2024-03-21T08:30:06.645873+0000","last_became_active":"2024-03-21T08:28:36.665950+0000","last_became_peered":"2024-03-21T08:28:36.665950+0000","last_unstale":"2024-03-21T08:30:06.645873+0000","last_undegraded":"2024-03-21T08:30:06.645873+0000","last_fullsized":"2024-03-21T08:30:06.645873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:49:31.945890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650190+0000","last_change":"2024-03-21T08:29:07.032862+0000","last_active":"2024-03-21T08:30:06.650190+0000","last_peered":"2024-03-21T08:30:06.650190+0000","last_clean":"2024-03-21T08:30:06.650190+0000","last_became_active":"2024-03-21T08:29:07.031784+0000","last_became_peered":"2024-03-21T08:29:07.031784+0000","last_unstale":"2024-03-21T08:30:06.650190+0000","last_undegraded":"2024-03-21T08:30:06.650190+0000","last_fullsized":"2024-03-21T08:30:06.650190+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:34.037495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641937+0000","last_change":"2024-03-21T08:28:36.660812+0000","last_active":"2024-03-21T08:30:06.641937+0000","last_peered":"2024-03-21T08:30:06.641937+0000","last_clean":"2024-03-21T08:30:06.641937+0000","last_became_active":"2024-03-21T08:28:36.660044+0000","last_became_peered":"2024-03-21T08:28:36.660044+0000","last_unstale":"2024-03-21T08:30:06.641937+0000","last_undegraded":"2024-03-21T08:30:06.641937+0000","last_fullsized":"2024-03-21T08:30:06.641937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:31.598678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656562+0000","last_change":"2024-03-21T08:28:36.669441+0000","last_active":"2024-03-21T08:30:06.656562+0000","last_peered":"2024-03-21T08:30:06.656562+0000","last_clean":"2024-03-21T08:30:06.656562+0000","last_became_active":"2024-03-21T08:28:36.668668+0000","last_became_peered":"2024-03-21T08:28:36.668668+0000","last_unstale":"2024-03-21T08:30:06.656562+0000","last_undegraded":"2024-03-21T08:30:06.656562+0000","last_fullsized":"2024-03-21T08:30:06.656562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:04.828297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66334,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":503,"num_read_kb":325,"num_write":163,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285607,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":534350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547245,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1781760,"data_stored":1183951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710772,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":538682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972412,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":533924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266756,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1699840,"data_stored":1130415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462987,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":529784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":592587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":531150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1179648,"data_stored":593746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1097728,"data_stored":529956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888150,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":534412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1703936,"data_stored":1125981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:30:31.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T08:30:32.268 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:32.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:31 smithi084 ceph-mon[35477]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:30:32.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:31 smithi171 ceph-mon[35388]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:30:32.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:31 smithi003 ceph-mon[31675]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:30:33.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:32 smithi003 ceph-mon[31675]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:33.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:32 smithi084 ceph-mon[35477]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:33.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:32 smithi171 ceph-mon[35388]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:34.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 -- 172.21.15.3:0/1414687002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c06a7e0 msgr2=0x7f220c10f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 --2- 172.21.15.3:0/1414687002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c06a7e0 0x7f220c10f020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f220400b0d0 tx=0x7f220400b3e0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 -- 172.21.15.3:0/1414687002 shutdown_connections 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 --2- 172.21.15.3:0/1414687002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c06a7e0 0x7f220c10f020 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 --2- 172.21.15.3:0/1414687002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220c1081e0 0x7f220c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 --2- 172.21.15.3:0/1414687002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f220c069e40 0x7f220c06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 -- 172.21.15.3:0/1414687002 >> 172.21.15.3:0/1414687002 conn(0x7f220c077fc0 msgr2=0x7f220c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 -- 172.21.15.3:0/1414687002 shutdown_connections 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.377+0000 7f2214600700 1 -- 172.21.15.3:0/1414687002 wait complete. 2024-03-21T08:30:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 Processor -- start 2024-03-21T08:30:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 -- start start 2024-03-21T08:30:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c069e40 0x7f220c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f220c06a7e0 0x7f220c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220c1081e0 0x7f220c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c112360 con 0x7f220c069e40 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c1121e0 con 0x7f220c1081e0 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.379+0000 7f2214600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c1124e0 con 0x7f220c06a7e0 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c069e40 0x7f220c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c069e40 0x7f220c1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50716/0 (socket says 172.21.15.3:50716) 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 -- 172.21.15.3:0/1421845408 learned_addr learned my addr 172.21.15.3:0/1421845408 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2212600700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220c1081e0 0x7f220c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211200700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f220c06a7e0 0x7f220c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 -- 172.21.15.3:0/1421845408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f220c06a7e0 msgr2=0x7f220c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f220c06a7e0 0x7f220c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 -- 172.21.15.3:0/1421845408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220c1081e0 msgr2=0x7f220c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220c1081e0 0x7f220c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 -- 172.21.15.3:0/1421845408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c19afe0 con 0x7f220c069e40 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.380+0000 7f2211c00700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c069e40 0x7f220c1a06c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f220000e9a0 tx=0x7f220000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.381+0000 7f220a000700 1 -- 172.21.15.3:0/1421845408 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220000cbd0 con 0x7f220c069e40 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.381+0000 7f220a000700 1 -- 172.21.15.3:0/1421845408 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f220000cd70 con 0x7f220c069e40 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.381+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c19b2d0 con 0x7f220c069e40 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.381+0000 7f220a000700 1 -- 172.21.15.3:0/1421845408 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200018680 con 0x7f220c069e40 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.381+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c1a7590 con 0x7f220c069e40 2024-03-21T08:30:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.382+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c068e50 con 0x7f220c069e40 2024-03-21T08:30:34.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.384+0000 7f220a000700 1 -- 172.21.15.3:0/1421845408 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 137904+0+0 (secure 0 0 0) 0x7f2200010430 con 0x7f220c069e40 2024-03-21T08:30:34.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.386+0000 7f220a000700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21f40a3040 0x7f21f40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:34.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.386+0000 7f220a000700 1 -- 172.21.15.3:0/1421845408 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f22000cfbc0 con 0x7f220c069e40 2024-03-21T08:30:34.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.386+0000 7f2211200700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21f40a3040 0x7f21f40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:34.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.387+0000 7f2211200700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21f40a3040 0x7f21f40a5500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f21f8005950 tx=0x7f21f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:34.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.390+0000 7f220a000700 1 -- 172.21.15.3:0/1421845408 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f220009c3f0 con 0x7f220c069e40 2024-03-21T08:30:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.613+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f220c10da80 con 0x7f21f40a3040 2024-03-21T08:30:34.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.643+0000 7f220a000700 1 -- 172.21.15.3:0/1421845408 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340359 (secure 0 0 0) 0x7f220c10da80 con 0x7f21f40a3040 2024-03-21T08:30:34.650 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21f40a3040 msgr2=0x7f21f40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21f40a3040 0x7f21f40a5500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f21f8005950 tx=0x7f21f80058e0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c069e40 msgr2=0x7f220c1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c069e40 0x7f220c1a06c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f220000e9a0 tx=0x7f220000ecb0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 shutdown_connections 2024-03-21T08:30:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21f40a3040 0x7f21f40a5500 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c069e40 0x7f220c1a06c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220c1081e0 0x7f220c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 --2- 172.21.15.3:0/1421845408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f220c06a7e0 0x7f220c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.650+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 >> 172.21.15.3:0/1421845408 conn(0x7f220c077fc0 msgr2=0x7f220c10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.651+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 shutdown_connections 2024-03-21T08:30:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:34.651+0000 7f2214600700 1 -- 172.21.15.3:0/1421845408 wait complete. 2024-03-21T08:30:34.654 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-03-21T08:30:35.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:34 smithi084 ceph-mon[35477]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:35.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:34 smithi171 ceph-mon[35388]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:35.416 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:34 smithi003 ceph-mon[31675]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:35.486 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-03-21T08:30:33.385728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656936,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":649,"num_read_kb":444,"num_write":346,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372408,"kb_used_data":14864,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569416,"statfs":{"total":1151940427776,"available":1151559081984,"internally_reserved":0,"allocated":15220736,"data_stored":8358938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006177"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657944+0000","last_change":"2024-03-21T08:28:36.667851+0000","last_active":"2024-03-21T08:30:06.657944+0000","last_peered":"2024-03-21T08:30:06.657944+0000","last_clean":"2024-03-21T08:30:06.657944+0000","last_became_active":"2024-03-21T08:28:36.665713+0000","last_became_peered":"2024-03-21T08:28:36.665713+0000","last_unstale":"2024-03-21T08:30:06.657944+0000","last_undegraded":"2024-03-21T08:30:06.657944+0000","last_fullsized":"2024-03-21T08:30:06.657944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:02.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653021+0000","last_change":"2024-03-21T08:28:36.650986+0000","last_active":"2024-03-21T08:30:06.653021+0000","last_peered":"2024-03-21T08:30:06.653021+0000","last_clean":"2024-03-21T08:30:06.653021+0000","last_became_active":"2024-03-21T08:28:36.650633+0000","last_became_peered":"2024-03-21T08:28:36.650633+0000","last_unstale":"2024-03-21T08:30:06.653021+0000","last_undegraded":"2024-03-21T08:30:06.653021+0000","last_fullsized":"2024-03-21T08:30:06.653021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:40:20.801814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652575+0000","last_change":"2024-03-21T08:28:36.662984+0000","last_active":"2024-03-21T08:30:06.652575+0000","last_peered":"2024-03-21T08:30:06.652575+0000","last_clean":"2024-03-21T08:30:06.652575+0000","last_became_active":"2024-03-21T08:28:36.662657+0000","last_became_peered":"2024-03-21T08:28:36.662657+0000","last_unstale":"2024-03-21T08:30:06.652575+0000","last_undegraded":"2024-03-21T08:30:06.652575+0000","last_fullsized":"2024-03-21T08:30:06.652575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:41.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664118+0000","last_change":"2024-03-21T08:28:36.665958+0000","last_active":"2024-03-21T08:30:06.664118+0000","last_peered":"2024-03-21T08:30:06.664118+0000","last_clean":"2024-03-21T08:30:06.664118+0000","last_became_active":"2024-03-21T08:28:36.664309+0000","last_became_peered":"2024-03-21T08:28:36.664309+0000","last_unstale":"2024-03-21T08:30:06.664118+0000","last_undegraded":"2024-03-21T08:30:06.664118+0000","last_fullsized":"2024-03-21T08:30:06.664118+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:16.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651421+0000","last_change":"2024-03-21T08:28:36.657767+0000","last_active":"2024-03-21T08:30:06.651421+0000","last_peered":"2024-03-21T08:30:06.651421+0000","last_clean":"2024-03-21T08:30:06.651421+0000","last_became_active":"2024-03-21T08:28:36.657350+0000","last_became_peered":"2024-03-21T08:28:36.657350+0000","last_unstale":"2024-03-21T08:30:06.651421+0000","last_undegraded":"2024-03-21T08:30:06.651421+0000","last_fullsized":"2024-03-21T08:30:06.651421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:47.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645358+0000","last_change":"2024-03-21T08:29:06.916971+0000","last_active":"2024-03-21T08:30:06.645358+0000","last_peered":"2024-03-21T08:30:06.645358+0000","last_clean":"2024-03-21T08:30:06.645358+0000","last_became_active":"2024-03-21T08:29:06.915997+0000","last_became_peered":"2024-03-21T08:29:06.915997+0000","last_unstale":"2024-03-21T08:30:06.645358+0000","last_undegraded":"2024-03-21T08:30:06.645358+0000","last_fullsized":"2024-03-21T08:30:06.645358+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:02.525190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655807+0000","last_change":"2024-03-21T08:28:36.667787+0000","last_active":"2024-03-21T08:30:06.655807+0000","last_peered":"2024-03-21T08:30:06.655807+0000","last_clean":"2024-03-21T08:30:06.655807+0000","last_became_active":"2024-03-21T08:28:36.667451+0000","last_became_peered":"2024-03-21T08:28:36.667451+0000","last_unstale":"2024-03-21T08:30:06.655807+0000","last_undegraded":"2024-03-21T08:30:06.655807+0000","last_fullsized":"2024-03-21T08:30:06.655807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:01.874897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647039+0000","last_change":"2024-03-21T08:28:36.662445+0000","last_active":"2024-03-21T08:30:06.647039+0000","last_peered":"2024-03-21T08:30:06.647039+0000","last_clean":"2024-03-21T08:30:06.647039+0000","last_became_active":"2024-03-21T08:28:36.662113+0000","last_became_peered":"2024-03-21T08:28:36.662113+0000","last_unstale":"2024-03-21T08:30:06.647039+0000","last_undegraded":"2024-03-21T08:30:06.647039+0000","last_fullsized":"2024-03-21T08:30:06.647039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:21.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648804+0000","last_change":"2024-03-21T08:28:36.666084+0000","last_active":"2024-03-21T08:30:06.648804+0000","last_peered":"2024-03-21T08:30:06.648804+0000","last_clean":"2024-03-21T08:30:06.648804+0000","last_became_active":"2024-03-21T08:28:36.665354+0000","last_became_peered":"2024-03-21T08:28:36.665354+0000","last_unstale":"2024-03-21T08:30:06.648804+0000","last_undegraded":"2024-03-21T08:30:06.648804+0000","last_fullsized":"2024-03-21T08:30:06.648804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:02:50.160765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645243+0000","last_change":"2024-03-21T08:29:06.919231+0000","last_active":"2024-03-21T08:30:06.645243+0000","last_peered":"2024-03-21T08:30:06.645243+0000","last_clean":"2024-03-21T08:30:06.645243+0000","last_became_active":"2024-03-21T08:29:06.918199+0000","last_became_peered":"2024-03-21T08:29:06.918199+0000","last_unstale":"2024-03-21T08:30:06.645243+0000","last_undegraded":"2024-03-21T08:30:06.645243+0000","last_fullsized":"2024-03-21T08:30:06.645243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:44.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.658150+0000","last_change":"2024-03-21T08:28:36.656385+0000","last_active":"2024-03-21T08:30:06.658150+0000","last_peered":"2024-03-21T08:30:06.658150+0000","last_clean":"2024-03-21T08:30:06.658150+0000","last_became_active":"2024-03-21T08:28:36.656014+0000","last_became_peered":"2024-03-21T08:28:36.656014+0000","last_unstale":"2024-03-21T08:30:06.658150+0000","last_undegraded":"2024-03-21T08:30:06.658150+0000","last_fullsized":"2024-03-21T08:30:06.658150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:46.000495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663994+0000","last_change":"2024-03-21T08:28:36.666254+0000","last_active":"2024-03-21T08:30:06.663994+0000","last_peered":"2024-03-21T08:30:06.663994+0000","last_clean":"2024-03-21T08:30:06.663994+0000","last_became_active":"2024-03-21T08:28:36.664698+0000","last_became_peered":"2024-03-21T08:28:36.664698+0000","last_unstale":"2024-03-21T08:30:06.663994+0000","last_undegraded":"2024-03-21T08:30:06.663994+0000","last_fullsized":"2024-03-21T08:30:06.663994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:03:08.358159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642152+0000","last_change":"2024-03-21T08:29:06.919930+0000","last_active":"2024-03-21T08:30:06.642152+0000","last_peered":"2024-03-21T08:30:06.642152+0000","last_clean":"2024-03-21T08:30:06.642152+0000","last_became_active":"2024-03-21T08:29:06.917852+0000","last_became_peered":"2024-03-21T08:29:06.917852+0000","last_unstale":"2024-03-21T08:30:06.642152+0000","last_undegraded":"2024-03-21T08:30:06.642152+0000","last_fullsized":"2024-03-21T08:30:06.642152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:05:01.185388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652424+0000","last_change":"2024-03-21T08:28:36.655229+0000","last_active":"2024-03-21T08:30:06.652424+0000","last_peered":"2024-03-21T08:30:06.652424+0000","last_clean":"2024-03-21T08:30:06.652424+0000","last_became_active":"2024-03-21T08:28:36.653949+0000","last_became_peered":"2024-03-21T08:28:36.653949+0000","last_unstale":"2024-03-21T08:30:06.652424+0000","last_undegraded":"2024-03-21T08:30:06.652424+0000","last_fullsized":"2024-03-21T08:30:06.652424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:57:38.370897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652132+0000","last_change":"2024-03-21T08:28:36.664894+0000","last_active":"2024-03-21T08:30:06.652132+0000","last_peered":"2024-03-21T08:30:06.652132+0000","last_clean":"2024-03-21T08:30:06.652132+0000","last_became_active":"2024-03-21T08:28:36.664523+0000","last_became_peered":"2024-03-21T08:28:36.664523+0000","last_unstale":"2024-03-21T08:30:06.652132+0000","last_undegraded":"2024-03-21T08:30:06.652132+0000","last_fullsized":"2024-03-21T08:30:06.652132+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:52:54.147459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648449+0000","last_change":"2024-03-21T08:28:36.662374+0000","last_active":"2024-03-21T08:30:06.648449+0000","last_peered":"2024-03-21T08:30:06.648449+0000","last_clean":"2024-03-21T08:30:06.648449+0000","last_became_active":"2024-03-21T08:28:36.662066+0000","last_became_peered":"2024-03-21T08:28:36.662066+0000","last_unstale":"2024-03-21T08:30:06.648449+0000","last_undegraded":"2024-03-21T08:30:06.648449+0000","last_fullsized":"2024-03-21T08:30:06.648449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:06.082925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654939+0000","last_change":"2024-03-21T08:28:36.672172+0000","last_active":"2024-03-21T08:30:06.654939+0000","last_peered":"2024-03-21T08:30:06.654939+0000","last_clean":"2024-03-21T08:30:06.654939+0000","last_became_active":"2024-03-21T08:28:36.671657+0000","last_became_peered":"2024-03-21T08:28:36.671657+0000","last_unstale":"2024-03-21T08:30:06.654939+0000","last_undegraded":"2024-03-21T08:30:06.654939+0000","last_fullsized":"2024-03-21T08:30:06.654939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:56.892818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649159+0000","last_change":"2024-03-21T08:28:36.666973+0000","last_active":"2024-03-21T08:30:06.649159+0000","last_peered":"2024-03-21T08:30:06.649159+0000","last_clean":"2024-03-21T08:30:06.649159+0000","last_became_active":"2024-03-21T08:28:36.663770+0000","last_became_peered":"2024-03-21T08:28:36.663770+0000","last_unstale":"2024-03-21T08:30:06.649159+0000","last_undegraded":"2024-03-21T08:30:06.649159+0000","last_fullsized":"2024-03-21T08:30:06.649159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:54:26.701009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646127+0000","last_change":"2024-03-21T08:28:36.668793+0000","last_active":"2024-03-21T08:30:06.646127+0000","last_peered":"2024-03-21T08:30:06.646127+0000","last_clean":"2024-03-21T08:30:06.646127+0000","last_became_active":"2024-03-21T08:28:36.668452+0000","last_became_peered":"2024-03-21T08:28:36.668452+0000","last_unstale":"2024-03-21T08:30:06.646127+0000","last_undegraded":"2024-03-21T08:30:06.646127+0000","last_fullsized":"2024-03-21T08:30:06.646127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:45.191849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644857+0000","last_change":"2024-03-21T08:28:40.687618+0000","last_active":"2024-03-21T08:30:06.644857+0000","last_peered":"2024-03-21T08:30:06.644857+0000","last_clean":"2024-03-21T08:30:06.644857+0000","last_became_active":"2024-03-21T08:28:40.687155+0000","last_became_peered":"2024-03-21T08:28:40.687155+0000","last_unstale":"2024-03-21T08:30:06.644857+0000","last_undegraded":"2024-03-21T08:30:06.644857+0000","last_fullsized":"2024-03-21T08:30:06.644857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:08.014790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'9","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672911+0000","last_change":"2024-03-21T08:30:06.672911+0000","last_active":"2024-03-21T08:30:06.672911+0000","last_peered":"2024-03-21T08:30:06.672911+0000","last_clean":"2024-03-21T08:30:06.672911+0000","last_became_active":"2024-03-21T08:30:06.661974+0000","last_became_peered":"2024-03-21T08:30:06.661974+0000","last_unstale":"2024-03-21T08:30:06.672911+0000","last_undegraded":"2024-03-21T08:30:06.672911+0000","last_fullsized":"2024-03-21T08:30:06.672911+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:37:25.848279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647791+0000","last_change":"2024-03-21T08:28:36.670325+0000","last_active":"2024-03-21T08:30:06.647791+0000","last_peered":"2024-03-21T08:30:06.647791+0000","last_clean":"2024-03-21T08:30:06.647791+0000","last_became_active":"2024-03-21T08:28:36.669783+0000","last_became_peered":"2024-03-21T08:28:36.669783+0000","last_unstale":"2024-03-21T08:30:06.647791+0000","last_undegraded":"2024-03-21T08:30:06.647791+0000","last_fullsized":"2024-03-21T08:30:06.647791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:24.166567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663660+0000","last_change":"2024-03-21T08:28:40.689849+0000","last_active":"2024-03-21T08:30:06.663660+0000","last_peered":"2024-03-21T08:30:06.663660+0000","last_clean":"2024-03-21T08:30:06.663660+0000","last_became_active":"2024-03-21T08:28:40.689307+0000","last_became_peered":"2024-03-21T08:28:40.689307+0000","last_unstale":"2024-03-21T08:30:06.663660+0000","last_undegraded":"2024-03-21T08:30:06.663660+0000","last_fullsized":"2024-03-21T08:30:06.663660+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:34:05.660317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647896+0000","last_change":"2024-03-21T08:28:35.674685+0000","last_active":"2024-03-21T08:30:06.647896+0000","last_peered":"2024-03-21T08:30:06.647896+0000","last_clean":"2024-03-21T08:30:06.647896+0000","last_became_active":"2024-03-21T08:28:35.674312+0000","last_became_peered":"2024-03-21T08:28:35.674312+0000","last_unstale":"2024-03-21T08:30:06.647896+0000","last_undegraded":"2024-03-21T08:30:06.647896+0000","last_fullsized":"2024-03-21T08:30:06.647896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:27.644111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652988+0000","last_change":"2024-03-21T08:28:36.667744+0000","last_active":"2024-03-21T08:30:06.652988+0000","last_peered":"2024-03-21T08:30:06.652988+0000","last_clean":"2024-03-21T08:30:06.652988+0000","last_became_active":"2024-03-21T08:28:36.667279+0000","last_became_peered":"2024-03-21T08:28:36.667279+0000","last_unstale":"2024-03-21T08:30:06.652988+0000","last_undegraded":"2024-03-21T08:30:06.652988+0000","last_fullsized":"2024-03-21T08:30:06.652988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:14:24.974683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664150+0000","last_change":"2024-03-21T08:28:40.688864+0000","last_active":"2024-03-21T08:30:06.664150+0000","last_peered":"2024-03-21T08:30:06.664150+0000","last_clean":"2024-03-21T08:30:06.664150+0000","last_became_active":"2024-03-21T08:28:40.688029+0000","last_became_peered":"2024-03-21T08:28:40.688029+0000","last_unstale":"2024-03-21T08:30:06.664150+0000","last_undegraded":"2024-03-21T08:30:06.664150+0000","last_fullsized":"2024-03-21T08:30:06.664150+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:39.235807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.680224+0000","last_change":"2024-03-21T08:30:06.680224+0000","last_active":"2024-03-21T08:30:06.680224+0000","last_peered":"2024-03-21T08:30:06.680224+0000","last_clean":"2024-03-21T08:30:06.680224+0000","last_became_active":"2024-03-21T08:30:06.665898+0000","last_became_peered":"2024-03-21T08:30:06.665898+0000","last_unstale":"2024-03-21T08:30:06.680224+0000","last_undegraded":"2024-03-21T08:30:06.680224+0000","last_fullsized":"2024-03-21T08:30:06.680224+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:21.690576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650614+0000","last_change":"2024-03-21T08:28:36.667244+0000","last_active":"2024-03-21T08:30:06.650614+0000","last_peered":"2024-03-21T08:30:06.650614+0000","last_clean":"2024-03-21T08:30:06.650614+0000","last_became_active":"2024-03-21T08:28:36.666829+0000","last_became_peered":"2024-03-21T08:28:36.666829+0000","last_unstale":"2024-03-21T08:30:06.650614+0000","last_undegraded":"2024-03-21T08:30:06.650614+0000","last_fullsized":"2024-03-21T08:30:06.650614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:12:56.842084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657724+0000","last_change":"2024-03-21T08:28:40.692621+0000","last_active":"2024-03-21T08:30:06.657724+0000","last_peered":"2024-03-21T08:30:06.657724+0000","last_clean":"2024-03-21T08:30:06.657724+0000","last_became_active":"2024-03-21T08:28:40.691984+0000","last_became_peered":"2024-03-21T08:28:40.691984+0000","last_unstale":"2024-03-21T08:30:06.657724+0000","last_undegraded":"2024-03-21T08:30:06.657724+0000","last_fullsized":"2024-03-21T08:30:06.657724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:29.330702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651177+0000","last_change":"2024-03-21T08:28:35.676421+0000","last_active":"2024-03-21T08:30:06.651177+0000","last_peered":"2024-03-21T08:30:06.651177+0000","last_clean":"2024-03-21T08:30:06.651177+0000","last_became_active":"2024-03-21T08:28:35.674957+0000","last_became_peered":"2024-03-21T08:28:35.674957+0000","last_unstale":"2024-03-21T08:30:06.651177+0000","last_undegraded":"2024-03-21T08:30:06.651177+0000","last_fullsized":"2024-03-21T08:30:06.651177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:01:38.397757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666420+0000","last_change":"2024-03-21T08:28:36.664662+0000","last_active":"2024-03-21T08:30:06.666420+0000","last_peered":"2024-03-21T08:30:06.666420+0000","last_clean":"2024-03-21T08:30:06.666420+0000","last_became_active":"2024-03-21T08:28:36.664324+0000","last_became_peered":"2024-03-21T08:28:36.664324+0000","last_unstale":"2024-03-21T08:30:06.666420+0000","last_undegraded":"2024-03-21T08:30:06.666420+0000","last_fullsized":"2024-03-21T08:30:06.666420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:57.325869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645339+0000","last_change":"2024-03-21T08:28:40.676297+0000","last_active":"2024-03-21T08:30:06.645339+0000","last_peered":"2024-03-21T08:30:06.645339+0000","last_clean":"2024-03-21T08:30:06.645339+0000","last_became_active":"2024-03-21T08:28:40.675337+0000","last_became_peered":"2024-03-21T08:28:40.675337+0000","last_unstale":"2024-03-21T08:30:06.645339+0000","last_undegraded":"2024-03-21T08:30:06.645339+0000","last_fullsized":"2024-03-21T08:30:06.645339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:05.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":124,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:31.568677+0000","last_change":"2024-03-21T08:28:35.687651+0000","last_active":"2024-03-21T08:30:31.568677+0000","last_peered":"2024-03-21T08:30:31.568677+0000","last_clean":"2024-03-21T08:30:31.568677+0000","last_became_active":"2024-03-21T08:28:35.686284+0000","last_became_peered":"2024-03-21T08:28:35.686284+0000","last_unstale":"2024-03-21T08:30:31.568677+0000","last_undegraded":"2024-03-21T08:30:31.568677+0000","last_fullsized":"2024-03-21T08:30:31.568677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:56.121798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651915+0000","last_change":"2024-03-21T08:29:07.033436+0000","last_active":"2024-03-21T08:30:06.651915+0000","last_peered":"2024-03-21T08:30:06.651915+0000","last_clean":"2024-03-21T08:30:06.651915+0000","last_became_active":"2024-03-21T08:29:07.032012+0000","last_became_peered":"2024-03-21T08:29:07.032012+0000","last_unstale":"2024-03-21T08:30:06.651915+0000","last_undegraded":"2024-03-21T08:30:06.651915+0000","last_fullsized":"2024-03-21T08:30:06.651915+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:44:47.009078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645632+0000","last_change":"2024-03-21T08:29:06.882449+0000","last_active":"2024-03-21T08:30:06.645632+0000","last_peered":"2024-03-21T08:30:06.645632+0000","last_clean":"2024-03-21T08:30:06.645632+0000","last_became_active":"2024-03-21T08:29:06.880682+0000","last_became_peered":"2024-03-21T08:29:06.880682+0000","last_unstale":"2024-03-21T08:30:06.645632+0000","last_undegraded":"2024-03-21T08:30:06.645632+0000","last_fullsized":"2024-03-21T08:30:06.645632+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:09.464388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.659993+0000","last_change":"2024-03-21T08:30:06.659993+0000","last_active":"2024-03-21T08:30:06.659993+0000","last_peered":"2024-03-21T08:30:06.659993+0000","last_clean":"2024-03-21T08:30:06.659993+0000","last_became_active":"2024-03-21T08:30:06.653383+0000","last_became_peered":"2024-03-21T08:30:06.653383+0000","last_unstale":"2024-03-21T08:30:06.659993+0000","last_undegraded":"2024-03-21T08:30:06.659993+0000","last_fullsized":"2024-03-21T08:30:06.659993+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:43.836881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.667643+0000","last_change":"2024-03-21T08:28:36.670176+0000","last_active":"2024-03-21T08:30:06.667643+0000","last_peered":"2024-03-21T08:30:06.667643+0000","last_clean":"2024-03-21T08:30:06.667643+0000","last_became_active":"2024-03-21T08:28:36.669477+0000","last_became_peered":"2024-03-21T08:28:36.669477+0000","last_unstale":"2024-03-21T08:30:06.667643+0000","last_undegraded":"2024-03-21T08:30:06.667643+0000","last_fullsized":"2024-03-21T08:30:06.667643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:10:21.289484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654633+0000","last_change":"2024-03-21T08:29:06.784996+0000","last_active":"2024-03-21T08:30:06.654633+0000","last_peered":"2024-03-21T08:30:06.654633+0000","last_clean":"2024-03-21T08:30:06.654633+0000","last_became_active":"2024-03-21T08:29:06.783403+0000","last_became_peered":"2024-03-21T08:29:06.783403+0000","last_unstale":"2024-03-21T08:30:06.654633+0000","last_undegraded":"2024-03-21T08:30:06.654633+0000","last_fullsized":"2024-03-21T08:30:06.654633+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:18:59.912956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641054+0000","last_change":"2024-03-21T08:28:35.667502+0000","last_active":"2024-03-21T08:30:06.641054+0000","last_peered":"2024-03-21T08:30:06.641054+0000","last_clean":"2024-03-21T08:30:06.641054+0000","last_became_active":"2024-03-21T08:28:35.666667+0000","last_became_peered":"2024-03-21T08:28:35.666667+0000","last_unstale":"2024-03-21T08:30:06.641054+0000","last_undegraded":"2024-03-21T08:30:06.641054+0000","last_fullsized":"2024-03-21T08:30:06.641054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:42.196361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646415+0000","last_change":"2024-03-21T08:28:36.670855+0000","last_active":"2024-03-21T08:30:06.646415+0000","last_peered":"2024-03-21T08:30:06.646415+0000","last_clean":"2024-03-21T08:30:06.646415+0000","last_became_active":"2024-03-21T08:28:36.666012+0000","last_became_peered":"2024-03-21T08:28:36.666012+0000","last_unstale":"2024-03-21T08:30:06.646415+0000","last_undegraded":"2024-03-21T08:30:06.646415+0000","last_fullsized":"2024-03-21T08:30:06.646415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:19.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652857+0000","last_change":"2024-03-21T08:28:40.700037+0000","last_active":"2024-03-21T08:30:06.652857+0000","last_peered":"2024-03-21T08:30:06.652857+0000","last_clean":"2024-03-21T08:30:06.652857+0000","last_became_active":"2024-03-21T08:28:40.698127+0000","last_became_peered":"2024-03-21T08:28:40.698127+0000","last_unstale":"2024-03-21T08:30:06.652857+0000","last_undegraded":"2024-03-21T08:30:06.652857+0000","last_fullsized":"2024-03-21T08:30:06.652857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:48:57.135690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664454+0000","last_change":"2024-03-21T08:28:35.677071+0000","last_active":"2024-03-21T08:30:06.664454+0000","last_peered":"2024-03-21T08:30:06.664454+0000","last_clean":"2024-03-21T08:30:06.664454+0000","last_became_active":"2024-03-21T08:28:35.676652+0000","last_became_peered":"2024-03-21T08:28:35.676652+0000","last_unstale":"2024-03-21T08:30:06.664454+0000","last_undegraded":"2024-03-21T08:30:06.664454+0000","last_fullsized":"2024-03-21T08:30:06.664454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:24:54.097952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647294+0000","last_change":"2024-03-21T08:28:36.677981+0000","last_active":"2024-03-21T08:30:06.647294+0000","last_peered":"2024-03-21T08:30:06.647294+0000","last_clean":"2024-03-21T08:30:06.647294+0000","last_became_active":"2024-03-21T08:28:36.677512+0000","last_became_peered":"2024-03-21T08:28:36.677512+0000","last_unstale":"2024-03-21T08:30:06.647294+0000","last_undegraded":"2024-03-21T08:30:06.647294+0000","last_fullsized":"2024-03-21T08:30:06.647294+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:50.742928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646682+0000","last_change":"2024-03-21T08:28:40.692672+0000","last_active":"2024-03-21T08:30:06.646682+0000","last_peered":"2024-03-21T08:30:06.646682+0000","last_clean":"2024-03-21T08:30:06.646682+0000","last_became_active":"2024-03-21T08:28:40.688940+0000","last_became_peered":"2024-03-21T08:28:40.688940+0000","last_unstale":"2024-03-21T08:30:06.646682+0000","last_undegraded":"2024-03-21T08:30:06.646682+0000","last_fullsized":"2024-03-21T08:30:06.646682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:39.569709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":114,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:27.555744+0000","last_change":"2024-03-21T08:28:35.681970+0000","last_active":"2024-03-21T08:30:27.555744+0000","last_peered":"2024-03-21T08:30:27.555744+0000","last_clean":"2024-03-21T08:30:27.555744+0000","last_became_active":"2024-03-21T08:28:35.681252+0000","last_became_peered":"2024-03-21T08:28:35.681252+0000","last_unstale":"2024-03-21T08:30:27.555744+0000","last_undegraded":"2024-03-21T08:30:27.555744+0000","last_fullsized":"2024-03-21T08:30:27.555744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:46.345359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648867+0000","last_change":"2024-03-21T08:28:36.675419+0000","last_active":"2024-03-21T08:30:06.648867+0000","last_peered":"2024-03-21T08:30:06.648867+0000","last_clean":"2024-03-21T08:30:06.648867+0000","last_became_active":"2024-03-21T08:28:36.674039+0000","last_became_peered":"2024-03-21T08:28:36.674039+0000","last_unstale":"2024-03-21T08:30:06.648867+0000","last_undegraded":"2024-03-21T08:30:06.648867+0000","last_fullsized":"2024-03-21T08:30:06.648867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:09.667634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648534+0000","last_change":"2024-03-21T08:28:40.690076+0000","last_active":"2024-03-21T08:30:06.648534+0000","last_peered":"2024-03-21T08:30:06.648534+0000","last_clean":"2024-03-21T08:30:06.648534+0000","last_became_active":"2024-03-21T08:28:40.683817+0000","last_became_peered":"2024-03-21T08:28:40.683817+0000","last_unstale":"2024-03-21T08:30:06.648534+0000","last_undegraded":"2024-03-21T08:30:06.648534+0000","last_fullsized":"2024-03-21T08:30:06.648534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:33:00.948879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654037+0000","last_change":"2024-03-21T08:28:35.690693+0000","last_active":"2024-03-21T08:30:06.654037+0000","last_peered":"2024-03-21T08:30:06.654037+0000","last_clean":"2024-03-21T08:30:06.654037+0000","last_became_active":"2024-03-21T08:28:35.690250+0000","last_became_peered":"2024-03-21T08:28:35.690250+0000","last_unstale":"2024-03-21T08:30:06.654037+0000","last_undegraded":"2024-03-21T08:30:06.654037+0000","last_fullsized":"2024-03-21T08:30:06.654037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:44:21.307772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653094+0000","last_change":"2024-03-21T08:28:36.661872+0000","last_active":"2024-03-21T08:30:06.653094+0000","last_peered":"2024-03-21T08:30:06.653094+0000","last_clean":"2024-03-21T08:30:06.653094+0000","last_became_active":"2024-03-21T08:28:36.661483+0000","last_became_peered":"2024-03-21T08:28:36.661483+0000","last_unstale":"2024-03-21T08:30:06.653094+0000","last_undegraded":"2024-03-21T08:30:06.653094+0000","last_fullsized":"2024-03-21T08:30:06.653094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:03:35.630141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664958+0000","last_change":"2024-03-21T08:28:40.682753+0000","last_active":"2024-03-21T08:30:06.664958+0000","last_peered":"2024-03-21T08:30:06.664958+0000","last_clean":"2024-03-21T08:30:06.664958+0000","last_became_active":"2024-03-21T08:28:40.682278+0000","last_became_peered":"2024-03-21T08:28:40.682278+0000","last_unstale":"2024-03-21T08:30:06.664958+0000","last_undegraded":"2024-03-21T08:30:06.664958+0000","last_fullsized":"2024-03-21T08:30:06.664958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:20.280442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666203+0000","last_change":"2024-03-21T08:28:35.672524+0000","last_active":"2024-03-21T08:30:06.666203+0000","last_peered":"2024-03-21T08:30:06.666203+0000","last_clean":"2024-03-21T08:30:06.666203+0000","last_became_active":"2024-03-21T08:28:35.672051+0000","last_became_peered":"2024-03-21T08:28:35.672051+0000","last_unstale":"2024-03-21T08:30:06.666203+0000","last_undegraded":"2024-03-21T08:30:06.666203+0000","last_fullsized":"2024-03-21T08:30:06.666203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:53.336320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651096+0000","last_change":"2024-03-21T08:29:06.532172+0000","last_active":"2024-03-21T08:30:06.651096+0000","last_peered":"2024-03-21T08:30:06.651096+0000","last_clean":"2024-03-21T08:30:06.651096+0000","last_became_active":"2024-03-21T08:29:06.531439+0000","last_became_peered":"2024-03-21T08:29:06.531439+0000","last_unstale":"2024-03-21T08:30:06.651096+0000","last_undegraded":"2024-03-21T08:30:06.651096+0000","last_fullsized":"2024-03-21T08:30:06.651096+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:27.222359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648559+0000","last_change":"2024-03-21T08:28:40.682294+0000","last_active":"2024-03-21T08:30:06.648559+0000","last_peered":"2024-03-21T08:30:06.648559+0000","last_clean":"2024-03-21T08:30:06.648559+0000","last_became_active":"2024-03-21T08:28:40.681855+0000","last_became_peered":"2024-03-21T08:28:40.681855+0000","last_unstale":"2024-03-21T08:30:06.648559+0000","last_undegraded":"2024-03-21T08:30:06.648559+0000","last_fullsized":"2024-03-21T08:30:06.648559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:49.756322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'34","reported_seq":234,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:28.558857+0000","last_change":"2024-03-21T08:28:35.682947+0000","last_active":"2024-03-21T08:30:28.558857+0000","last_peered":"2024-03-21T08:30:28.558857+0000","last_clean":"2024-03-21T08:30:28.558857+0000","last_became_active":"2024-03-21T08:28:35.682395+0000","last_became_peered":"2024-03-21T08:28:35.682395+0000","last_unstale":"2024-03-21T08:30:28.558857+0000","last_undegraded":"2024-03-21T08:30:28.558857+0000","last_fullsized":"2024-03-21T08:30:28.558857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:07:48.072988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62333,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":169,"num_read_kb":146,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653737+0000","last_change":"2024-03-21T08:28:36.651140+0000","last_active":"2024-03-21T08:30:06.653737+0000","last_peered":"2024-03-21T08:30:06.653737+0000","last_clean":"2024-03-21T08:30:06.653737+0000","last_became_active":"2024-03-21T08:28:36.650551+0000","last_became_peered":"2024-03-21T08:28:36.650551+0000","last_unstale":"2024-03-21T08:30:06.653737+0000","last_undegraded":"2024-03-21T08:30:06.653737+0000","last_fullsized":"2024-03-21T08:30:06.653737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:53.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.667005+0000","last_change":"2024-03-21T08:28:40.691814+0000","last_active":"2024-03-21T08:30:06.667005+0000","last_peered":"2024-03-21T08:30:06.667005+0000","last_clean":"2024-03-21T08:30:06.667005+0000","last_became_active":"2024-03-21T08:28:40.691372+0000","last_became_peered":"2024-03-21T08:28:40.691372+0000","last_unstale":"2024-03-21T08:30:06.667005+0000","last_undegraded":"2024-03-21T08:30:06.667005+0000","last_fullsized":"2024-03-21T08:30:06.667005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:21:42.612037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645605+0000","last_change":"2024-03-21T08:28:35.687178+0000","last_active":"2024-03-21T08:30:06.645605+0000","last_peered":"2024-03-21T08:30:06.645605+0000","last_clean":"2024-03-21T08:30:06.645605+0000","last_became_active":"2024-03-21T08:28:35.686771+0000","last_became_peered":"2024-03-21T08:28:35.686771+0000","last_unstale":"2024-03-21T08:30:06.645605+0000","last_undegraded":"2024-03-21T08:30:06.645605+0000","last_fullsized":"2024-03-21T08:30:06.645605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:54.665866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641400+0000","last_change":"2024-03-21T08:28:36.662952+0000","last_active":"2024-03-21T08:30:06.641400+0000","last_peered":"2024-03-21T08:30:06.641400+0000","last_clean":"2024-03-21T08:30:06.641400+0000","last_became_active":"2024-03-21T08:28:36.662468+0000","last_became_peered":"2024-03-21T08:28:36.662468+0000","last_unstale":"2024-03-21T08:30:06.641400+0000","last_undegraded":"2024-03-21T08:30:06.641400+0000","last_fullsized":"2024-03-21T08:30:06.641400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:40.486998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656265+0000","last_change":"2024-03-21T08:28:40.693999+0000","last_active":"2024-03-21T08:30:06.656265+0000","last_peered":"2024-03-21T08:30:06.656265+0000","last_clean":"2024-03-21T08:30:06.656265+0000","last_became_active":"2024-03-21T08:28:40.693507+0000","last_became_peered":"2024-03-21T08:28:40.693507+0000","last_unstale":"2024-03-21T08:30:06.656265+0000","last_undegraded":"2024-03-21T08:30:06.656265+0000","last_fullsized":"2024-03-21T08:30:06.656265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:14.159379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.668638+0000","last_change":"2024-03-21T08:28:35.677536+0000","last_active":"2024-03-21T08:30:06.668638+0000","last_peered":"2024-03-21T08:30:06.668638+0000","last_clean":"2024-03-21T08:30:06.668638+0000","last_became_active":"2024-03-21T08:28:35.677100+0000","last_became_peered":"2024-03-21T08:28:35.677100+0000","last_unstale":"2024-03-21T08:30:06.668638+0000","last_undegraded":"2024-03-21T08:30:06.668638+0000","last_fullsized":"2024-03-21T08:30:06.668638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:52:47.275869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656491+0000","last_change":"2024-03-21T08:28:36.667190+0000","last_active":"2024-03-21T08:30:06.656491+0000","last_peered":"2024-03-21T08:30:06.656491+0000","last_clean":"2024-03-21T08:30:06.656491+0000","last_became_active":"2024-03-21T08:28:36.665336+0000","last_became_peered":"2024-03-21T08:28:36.665336+0000","last_unstale":"2024-03-21T08:30:06.656491+0000","last_undegraded":"2024-03-21T08:30:06.656491+0000","last_fullsized":"2024-03-21T08:30:06.656491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:21.888426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649671+0000","last_change":"2024-03-21T08:28:40.696582+0000","last_active":"2024-03-21T08:30:06.649671+0000","last_peered":"2024-03-21T08:30:06.649671+0000","last_clean":"2024-03-21T08:30:06.649671+0000","last_became_active":"2024-03-21T08:28:40.689583+0000","last_became_peered":"2024-03-21T08:28:40.689583+0000","last_unstale":"2024-03-21T08:30:06.649671+0000","last_undegraded":"2024-03-21T08:30:06.649671+0000","last_fullsized":"2024-03-21T08:30:06.649671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:57.329533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":117,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:28.560112+0000","last_change":"2024-03-21T08:28:35.675560+0000","last_active":"2024-03-21T08:30:28.560112+0000","last_peered":"2024-03-21T08:30:28.560112+0000","last_clean":"2024-03-21T08:30:28.560112+0000","last_became_active":"2024-03-21T08:28:35.673496+0000","last_became_peered":"2024-03-21T08:28:35.673496+0000","last_unstale":"2024-03-21T08:30:28.560112+0000","last_undegraded":"2024-03-21T08:30:28.560112+0000","last_fullsized":"2024-03-21T08:30:28.560112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:41:27.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653646+0000","last_change":"2024-03-21T08:28:36.660869+0000","last_active":"2024-03-21T08:30:06.653646+0000","last_peered":"2024-03-21T08:30:06.653646+0000","last_clean":"2024-03-21T08:30:06.653646+0000","last_became_active":"2024-03-21T08:28:36.659291+0000","last_became_peered":"2024-03-21T08:28:36.659291+0000","last_unstale":"2024-03-21T08:30:06.653646+0000","last_undegraded":"2024-03-21T08:30:06.653646+0000","last_fullsized":"2024-03-21T08:30:06.653646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:06.534214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.643105+0000","last_change":"2024-03-21T08:28:40.701351+0000","last_active":"2024-03-21T08:30:06.643105+0000","last_peered":"2024-03-21T08:30:06.643105+0000","last_clean":"2024-03-21T08:30:06.643105+0000","last_became_active":"2024-03-21T08:28:40.700989+0000","last_became_peered":"2024-03-21T08:28:40.700989+0000","last_unstale":"2024-03-21T08:30:06.643105+0000","last_undegraded":"2024-03-21T08:30:06.643105+0000","last_fullsized":"2024-03-21T08:30:06.643105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:35.697013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650372+0000","last_change":"2024-03-21T08:28:35.685024+0000","last_active":"2024-03-21T08:30:06.650372+0000","last_peered":"2024-03-21T08:30:06.650372+0000","last_clean":"2024-03-21T08:30:06.650372+0000","last_became_active":"2024-03-21T08:28:35.684597+0000","last_became_peered":"2024-03-21T08:28:35.684597+0000","last_unstale":"2024-03-21T08:30:06.650372+0000","last_undegraded":"2024-03-21T08:30:06.650372+0000","last_fullsized":"2024-03-21T08:30:06.650372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:09:46.262782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664546+0000","last_change":"2024-03-21T08:28:36.664688+0000","last_active":"2024-03-21T08:30:06.664546+0000","last_peered":"2024-03-21T08:30:06.664546+0000","last_clean":"2024-03-21T08:30:06.664546+0000","last_became_active":"2024-03-21T08:28:36.663990+0000","last_became_peered":"2024-03-21T08:28:36.663990+0000","last_unstale":"2024-03-21T08:30:06.664546+0000","last_undegraded":"2024-03-21T08:30:06.664546+0000","last_fullsized":"2024-03-21T08:30:06.664546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:39.656290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664326+0000","last_change":"2024-03-21T08:28:40.695269+0000","last_active":"2024-03-21T08:30:06.664326+0000","last_peered":"2024-03-21T08:30:06.664326+0000","last_clean":"2024-03-21T08:30:06.664326+0000","last_became_active":"2024-03-21T08:28:40.694820+0000","last_became_peered":"2024-03-21T08:28:40.694820+0000","last_unstale":"2024-03-21T08:30:06.664326+0000","last_undegraded":"2024-03-21T08:30:06.664326+0000","last_fullsized":"2024-03-21T08:30:06.664326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:47:25.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672662+0000","last_change":"2024-03-21T08:30:06.672662+0000","last_active":"2024-03-21T08:30:06.672662+0000","last_peered":"2024-03-21T08:30:06.672662+0000","last_clean":"2024-03-21T08:30:06.672662+0000","last_became_active":"2024-03-21T08:30:06.660161+0000","last_became_peered":"2024-03-21T08:30:06.660161+0000","last_unstale":"2024-03-21T08:30:06.672662+0000","last_undegraded":"2024-03-21T08:30:06.672662+0000","last_fullsized":"2024-03-21T08:30:06.672662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:41.163709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,4],"acting":[8,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655691+0000","last_change":"2024-03-21T08:28:36.667605+0000","last_active":"2024-03-21T08:30:06.655691+0000","last_peered":"2024-03-21T08:30:06.655691+0000","last_clean":"2024-03-21T08:30:06.655691+0000","last_became_active":"2024-03-21T08:28:36.667300+0000","last_became_peered":"2024-03-21T08:28:36.667300+0000","last_unstale":"2024-03-21T08:30:06.655691+0000","last_undegraded":"2024-03-21T08:30:06.655691+0000","last_fullsized":"2024-03-21T08:30:06.655691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:49.268935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652450+0000","last_change":"2024-03-21T08:28:40.884964+0000","last_active":"2024-03-21T08:30:06.652450+0000","last_peered":"2024-03-21T08:30:06.652450+0000","last_clean":"2024-03-21T08:30:06.652450+0000","last_became_active":"2024-03-21T08:28:40.883445+0000","last_became_peered":"2024-03-21T08:28:40.883445+0000","last_unstale":"2024-03-21T08:30:06.652450+0000","last_undegraded":"2024-03-21T08:30:06.652450+0000","last_fullsized":"2024-03-21T08:30:06.652450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:36:54.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647907+0000","last_change":"2024-03-21T08:28:35.682581+0000","last_active":"2024-03-21T08:30:06.647907+0000","last_peered":"2024-03-21T08:30:06.647907+0000","last_clean":"2024-03-21T08:30:06.647907+0000","last_became_active":"2024-03-21T08:28:35.682150+0000","last_became_peered":"2024-03-21T08:28:35.682150+0000","last_unstale":"2024-03-21T08:30:06.647907+0000","last_undegraded":"2024-03-21T08:30:06.647907+0000","last_fullsized":"2024-03-21T08:30:06.647907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:08.729710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645727+0000","last_change":"2024-03-21T08:28:36.666632+0000","last_active":"2024-03-21T08:30:06.645727+0000","last_peered":"2024-03-21T08:30:06.645727+0000","last_clean":"2024-03-21T08:30:06.645727+0000","last_became_active":"2024-03-21T08:28:36.666305+0000","last_became_peered":"2024-03-21T08:28:36.666305+0000","last_unstale":"2024-03-21T08:30:06.645727+0000","last_undegraded":"2024-03-21T08:30:06.645727+0000","last_fullsized":"2024-03-21T08:30:06.645727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:15.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642839+0000","last_change":"2024-03-21T08:28:40.700382+0000","last_active":"2024-03-21T08:30:06.642839+0000","last_peered":"2024-03-21T08:30:06.642839+0000","last_clean":"2024-03-21T08:30:06.642839+0000","last_became_active":"2024-03-21T08:28:40.699762+0000","last_became_peered":"2024-03-21T08:28:40.699762+0000","last_unstale":"2024-03-21T08:30:06.642839+0000","last_undegraded":"2024-03-21T08:30:06.642839+0000","last_fullsized":"2024-03-21T08:30:06.642839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:53.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657040+0000","last_change":"2024-03-21T08:28:35.680310+0000","last_active":"2024-03-21T08:30:06.657040+0000","last_peered":"2024-03-21T08:30:06.657040+0000","last_clean":"2024-03-21T08:30:06.657040+0000","last_became_active":"2024-03-21T08:28:35.678634+0000","last_became_peered":"2024-03-21T08:28:35.678634+0000","last_unstale":"2024-03-21T08:30:06.657040+0000","last_undegraded":"2024-03-21T08:30:06.657040+0000","last_fullsized":"2024-03-21T08:30:06.657040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:56:42.872521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644184+0000","last_change":"2024-03-21T08:28:36.669527+0000","last_active":"2024-03-21T08:30:06.644184+0000","last_peered":"2024-03-21T08:30:06.644184+0000","last_clean":"2024-03-21T08:30:06.644184+0000","last_became_active":"2024-03-21T08:28:36.669196+0000","last_became_peered":"2024-03-21T08:28:36.669196+0000","last_unstale":"2024-03-21T08:30:06.644184+0000","last_undegraded":"2024-03-21T08:30:06.644184+0000","last_fullsized":"2024-03-21T08:30:06.644184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:07:34.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644005+0000","last_change":"2024-03-21T08:28:40.882949+0000","last_active":"2024-03-21T08:30:06.644005+0000","last_peered":"2024-03-21T08:30:06.644005+0000","last_clean":"2024-03-21T08:30:06.644005+0000","last_became_active":"2024-03-21T08:28:40.882432+0000","last_became_peered":"2024-03-21T08:28:40.882432+0000","last_unstale":"2024-03-21T08:30:06.644005+0000","last_undegraded":"2024-03-21T08:30:06.644005+0000","last_fullsized":"2024-03-21T08:30:06.644005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:51.171365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664621+0000","last_change":"2024-03-21T08:28:36.665164+0000","last_active":"2024-03-21T08:30:06.664621+0000","last_peered":"2024-03-21T08:30:06.664621+0000","last_clean":"2024-03-21T08:30:06.664621+0000","last_became_active":"2024-03-21T08:28:36.664794+0000","last_became_peered":"2024-03-21T08:28:36.664794+0000","last_unstale":"2024-03-21T08:30:06.664621+0000","last_undegraded":"2024-03-21T08:30:06.664621+0000","last_fullsized":"2024-03-21T08:30:06.664621+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:25:39.355633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":115,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:27.555595+0000","last_change":"2024-03-21T08:28:35.671754+0000","last_active":"2024-03-21T08:30:27.555595+0000","last_peered":"2024-03-21T08:30:27.555595+0000","last_clean":"2024-03-21T08:30:27.555595+0000","last_became_active":"2024-03-21T08:28:35.671363+0000","last_became_peered":"2024-03-21T08:28:35.671363+0000","last_unstale":"2024-03-21T08:30:27.555595+0000","last_undegraded":"2024-03-21T08:30:27.555595+0000","last_fullsized":"2024-03-21T08:30:27.555595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:22:11.147325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657802+0000","last_change":"2024-03-21T08:28:40.689038+0000","last_active":"2024-03-21T08:30:06.657802+0000","last_peered":"2024-03-21T08:30:06.657802+0000","last_clean":"2024-03-21T08:30:06.657802+0000","last_became_active":"2024-03-21T08:28:40.688538+0000","last_became_peered":"2024-03-21T08:28:40.688538+0000","last_unstale":"2024-03-21T08:30:06.657802+0000","last_undegraded":"2024-03-21T08:30:06.657802+0000","last_fullsized":"2024-03-21T08:30:06.657802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:14.833594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646707+0000","last_change":"2024-03-21T08:28:36.667725+0000","last_active":"2024-03-21T08:30:06.646707+0000","last_peered":"2024-03-21T08:30:06.646707+0000","last_clean":"2024-03-21T08:30:06.646707+0000","last_became_active":"2024-03-21T08:28:36.667303+0000","last_became_peered":"2024-03-21T08:28:36.667303+0000","last_unstale":"2024-03-21T08:30:06.646707+0000","last_undegraded":"2024-03-21T08:30:06.646707+0000","last_fullsized":"2024-03-21T08:30:06.646707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:32:18.990943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664189+0000","last_change":"2024-03-21T08:28:35.673497+0000","last_active":"2024-03-21T08:30:06.664189+0000","last_peered":"2024-03-21T08:30:06.664189+0000","last_clean":"2024-03-21T08:30:06.664189+0000","last_became_active":"2024-03-21T08:28:35.673045+0000","last_became_peered":"2024-03-21T08:28:35.673045+0000","last_unstale":"2024-03-21T08:30:06.664189+0000","last_undegraded":"2024-03-21T08:30:06.664189+0000","last_fullsized":"2024-03-21T08:30:06.664189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:23.972672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653331+0000","last_change":"2024-03-21T08:26:58.139094+0000","last_active":"2024-03-21T08:30:06.653331+0000","last_peered":"2024-03-21T08:30:06.653331+0000","last_clean":"2024-03-21T08:30:06.653331+0000","last_became_active":"2024-03-21T08:26:58.137648+0000","last_became_peered":"2024-03-21T08:26:58.137648+0000","last_unstale":"2024-03-21T08:30:06.653331+0000","last_undegraded":"2024-03-21T08:30:06.653331+0000","last_fullsized":"2024-03-21T08:30:06.653331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_clean_scrub_stamp":"2024-03-21T08:22:17.977314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:57.950111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665685+0000","last_change":"2024-03-21T08:28:40.887865+0000","last_active":"2024-03-21T08:30:06.665685+0000","last_peered":"2024-03-21T08:30:06.665685+0000","last_clean":"2024-03-21T08:30:06.665685+0000","last_became_active":"2024-03-21T08:28:40.885538+0000","last_became_peered":"2024-03-21T08:28:40.885538+0000","last_unstale":"2024-03-21T08:30:06.665685+0000","last_undegraded":"2024-03-21T08:30:06.665685+0000","last_fullsized":"2024-03-21T08:30:06.665685+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:26.532005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641113+0000","last_change":"2024-03-21T08:28:36.668600+0000","last_active":"2024-03-21T08:30:06.641113+0000","last_peered":"2024-03-21T08:30:06.641113+0000","last_clean":"2024-03-21T08:30:06.641113+0000","last_became_active":"2024-03-21T08:28:36.668188+0000","last_became_peered":"2024-03-21T08:28:36.668188+0000","last_unstale":"2024-03-21T08:30:06.641113+0000","last_undegraded":"2024-03-21T08:30:06.641113+0000","last_fullsized":"2024-03-21T08:30:06.641113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:13.536003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.674753+0000","last_change":"2024-03-21T08:30:06.673425+0000","last_active":"2024-03-21T08:30:06.674753+0000","last_peered":"2024-03-21T08:30:06.674753+0000","last_clean":"2024-03-21T08:30:06.674753+0000","last_became_active":"2024-03-21T08:30:06.666858+0000","last_became_peered":"2024-03-21T08:30:06.666858+0000","last_unstale":"2024-03-21T08:30:06.674753+0000","last_undegraded":"2024-03-21T08:30:06.674753+0000","last_fullsized":"2024-03-21T08:30:06.674753+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:37.159978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655430+0000","last_change":"2024-03-21T08:29:07.032622+0000","last_active":"2024-03-21T08:30:06.655430+0000","last_peered":"2024-03-21T08:30:06.655430+0000","last_clean":"2024-03-21T08:30:06.655430+0000","last_became_active":"2024-03-21T08:29:07.030884+0000","last_became_peered":"2024-03-21T08:29:07.030884+0000","last_unstale":"2024-03-21T08:30:06.655430+0000","last_undegraded":"2024-03-21T08:30:06.655430+0000","last_fullsized":"2024-03-21T08:30:06.655430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:47.615776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653948+0000","last_change":"2024-03-21T08:28:35.672948+0000","last_active":"2024-03-21T08:30:06.653948+0000","last_peered":"2024-03-21T08:30:06.653948+0000","last_clean":"2024-03-21T08:30:06.653948+0000","last_became_active":"2024-03-21T08:28:35.671576+0000","last_became_peered":"2024-03-21T08:28:35.671576+0000","last_unstale":"2024-03-21T08:30:06.653948+0000","last_undegraded":"2024-03-21T08:30:06.653948+0000","last_fullsized":"2024-03-21T08:30:06.653948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:36:25.620392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642652+0000","last_change":"2024-03-21T08:28:36.671173+0000","last_active":"2024-03-21T08:30:06.642652+0000","last_peered":"2024-03-21T08:30:06.642652+0000","last_clean":"2024-03-21T08:30:06.642652+0000","last_became_active":"2024-03-21T08:28:36.666264+0000","last_became_peered":"2024-03-21T08:28:36.666264+0000","last_unstale":"2024-03-21T08:30:06.642652+0000","last_undegraded":"2024-03-21T08:30:06.642652+0000","last_fullsized":"2024-03-21T08:30:06.642652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:35.081487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646298+0000","last_change":"2024-03-21T08:28:40.883801+0000","last_active":"2024-03-21T08:30:06.646298+0000","last_peered":"2024-03-21T08:30:06.646298+0000","last_clean":"2024-03-21T08:30:06.646298+0000","last_became_active":"2024-03-21T08:28:40.882556+0000","last_became_peered":"2024-03-21T08:28:40.882556+0000","last_unstale":"2024-03-21T08:30:06.646298+0000","last_undegraded":"2024-03-21T08:30:06.646298+0000","last_fullsized":"2024-03-21T08:30:06.646298+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:24.211625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'8","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649233+0000","last_change":"2024-03-21T08:28:35.668657+0000","last_active":"2024-03-21T08:30:06.649233+0000","last_peered":"2024-03-21T08:30:06.649233+0000","last_clean":"2024-03-21T08:30:06.649233+0000","last_became_active":"2024-03-21T08:28:35.668284+0000","last_became_peered":"2024-03-21T08:28:35.668284+0000","last_unstale":"2024-03-21T08:30:06.649233+0000","last_undegraded":"2024-03-21T08:30:06.649233+0000","last_fullsized":"2024-03-21T08:30:06.649233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:26:07.791388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648082+0000","last_change":"2024-03-21T08:28:36.652383+0000","last_active":"2024-03-21T08:30:06.648082+0000","last_peered":"2024-03-21T08:30:06.648082+0000","last_clean":"2024-03-21T08:30:06.648082+0000","last_became_active":"2024-03-21T08:28:36.652046+0000","last_became_peered":"2024-03-21T08:28:36.652046+0000","last_unstale":"2024-03-21T08:30:06.648082+0000","last_undegraded":"2024-03-21T08:30:06.648082+0000","last_fullsized":"2024-03-21T08:30:06.648082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:11:08.699391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644328+0000","last_change":"2024-03-21T08:28:40.686122+0000","last_active":"2024-03-21T08:30:06.644328+0000","last_peered":"2024-03-21T08:30:06.644328+0000","last_clean":"2024-03-21T08:30:06.644328+0000","last_became_active":"2024-03-21T08:28:40.685736+0000","last_became_peered":"2024-03-21T08:28:40.685736+0000","last_unstale":"2024-03-21T08:30:06.644328+0000","last_undegraded":"2024-03-21T08:30:06.644328+0000","last_fullsized":"2024-03-21T08:30:06.644328+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:52.941896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665195+0000","last_change":"2024-03-21T08:28:35.675197+0000","last_active":"2024-03-21T08:30:06.665195+0000","last_peered":"2024-03-21T08:30:06.665195+0000","last_clean":"2024-03-21T08:30:06.665195+0000","last_became_active":"2024-03-21T08:28:35.674688+0000","last_became_peered":"2024-03-21T08:28:35.674688+0000","last_unstale":"2024-03-21T08:30:06.665195+0000","last_undegraded":"2024-03-21T08:30:06.665195+0000","last_fullsized":"2024-03-21T08:30:06.665195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:01:11.489294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653149+0000","last_change":"2024-03-21T08:28:36.668160+0000","last_active":"2024-03-21T08:30:06.653149+0000","last_peered":"2024-03-21T08:30:06.653149+0000","last_clean":"2024-03-21T08:30:06.653149+0000","last_became_active":"2024-03-21T08:28:36.665152+0000","last_became_peered":"2024-03-21T08:28:36.665152+0000","last_unstale":"2024-03-21T08:30:06.653149+0000","last_undegraded":"2024-03-21T08:30:06.653149+0000","last_fullsized":"2024-03-21T08:30:06.653149+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:43:41.803822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663541+0000","last_change":"2024-03-21T08:28:40.884365+0000","last_active":"2024-03-21T08:30:06.663541+0000","last_peered":"2024-03-21T08:30:06.663541+0000","last_clean":"2024-03-21T08:30:06.663541+0000","last_became_active":"2024-03-21T08:28:40.883654+0000","last_became_peered":"2024-03-21T08:28:40.883654+0000","last_unstale":"2024-03-21T08:30:06.663541+0000","last_undegraded":"2024-03-21T08:30:06.663541+0000","last_fullsized":"2024-03-21T08:30:06.663541+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:46:43.888121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672111+0000","last_change":"2024-03-21T08:30:06.672111+0000","last_active":"2024-03-21T08:30:06.672111+0000","last_peered":"2024-03-21T08:30:06.672111+0000","last_clean":"2024-03-21T08:30:06.672111+0000","last_became_active":"2024-03-21T08:30:06.665627+0000","last_became_peered":"2024-03-21T08:30:06.665627+0000","last_unstale":"2024-03-21T08:30:06.672111+0000","last_undegraded":"2024-03-21T08:30:06.672111+0000","last_fullsized":"2024-03-21T08:30:06.672111+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:05.512276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650465+0000","last_change":"2024-03-21T08:28:36.653797+0000","last_active":"2024-03-21T08:30:06.650465+0000","last_peered":"2024-03-21T08:30:06.650465+0000","last_clean":"2024-03-21T08:30:06.650465+0000","last_became_active":"2024-03-21T08:28:36.651872+0000","last_became_peered":"2024-03-21T08:28:36.651872+0000","last_unstale":"2024-03-21T08:30:06.650465+0000","last_undegraded":"2024-03-21T08:30:06.650465+0000","last_fullsized":"2024-03-21T08:30:06.650465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:09.467573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649671+0000","last_change":"2024-03-21T08:28:40.692996+0000","last_active":"2024-03-21T08:30:06.649671+0000","last_peered":"2024-03-21T08:30:06.649671+0000","last_clean":"2024-03-21T08:30:06.649671+0000","last_became_active":"2024-03-21T08:28:40.687598+0000","last_became_peered":"2024-03-21T08:28:40.687598+0000","last_unstale":"2024-03-21T08:30:06.649671+0000","last_undegraded":"2024-03-21T08:30:06.649671+0000","last_fullsized":"2024-03-21T08:30:06.649671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:09.557272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645874+0000","last_change":"2024-03-21T08:28:35.672567+0000","last_active":"2024-03-21T08:30:06.645874+0000","last_peered":"2024-03-21T08:30:06.645874+0000","last_clean":"2024-03-21T08:30:06.645874+0000","last_became_active":"2024-03-21T08:28:35.671849+0000","last_became_peered":"2024-03-21T08:28:35.671849+0000","last_unstale":"2024-03-21T08:30:06.645874+0000","last_undegraded":"2024-03-21T08:30:06.645874+0000","last_fullsized":"2024-03-21T08:30:06.645874+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:13.050227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656580+0000","last_change":"2024-03-21T08:28:36.664110+0000","last_active":"2024-03-21T08:30:06.656580+0000","last_peered":"2024-03-21T08:30:06.656580+0000","last_clean":"2024-03-21T08:30:06.656580+0000","last_became_active":"2024-03-21T08:28:36.663297+0000","last_became_peered":"2024-03-21T08:28:36.663297+0000","last_unstale":"2024-03-21T08:30:06.656580+0000","last_undegraded":"2024-03-21T08:30:06.656580+0000","last_fullsized":"2024-03-21T08:30:06.656580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:55.450399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663781+0000","last_change":"2024-03-21T08:28:40.682371+0000","last_active":"2024-03-21T08:30:06.663781+0000","last_peered":"2024-03-21T08:30:06.663781+0000","last_clean":"2024-03-21T08:30:06.663781+0000","last_became_active":"2024-03-21T08:28:40.681786+0000","last_became_peered":"2024-03-21T08:28:40.681786+0000","last_unstale":"2024-03-21T08:30:06.663781+0000","last_undegraded":"2024-03-21T08:30:06.663781+0000","last_fullsized":"2024-03-21T08:30:06.663781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:15:38.130807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644923+0000","last_change":"2024-03-21T08:28:35.685312+0000","last_active":"2024-03-21T08:30:06.644923+0000","last_peered":"2024-03-21T08:30:06.644923+0000","last_clean":"2024-03-21T08:30:06.644923+0000","last_became_active":"2024-03-21T08:28:35.684327+0000","last_became_peered":"2024-03-21T08:28:35.684327+0000","last_unstale":"2024-03-21T08:30:06.644923+0000","last_undegraded":"2024-03-21T08:30:06.644923+0000","last_fullsized":"2024-03-21T08:30:06.644923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:12.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647398+0000","last_change":"2024-03-21T08:28:36.671186+0000","last_active":"2024-03-21T08:30:06.647398+0000","last_peered":"2024-03-21T08:30:06.647398+0000","last_clean":"2024-03-21T08:30:06.647398+0000","last_became_active":"2024-03-21T08:28:36.662726+0000","last_became_peered":"2024-03-21T08:28:36.662726+0000","last_unstale":"2024-03-21T08:30:06.647398+0000","last_undegraded":"2024-03-21T08:30:06.647398+0000","last_fullsized":"2024-03-21T08:30:06.647398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:29.129014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663706+0000","last_change":"2024-03-21T08:28:40.885203+0000","last_active":"2024-03-21T08:30:06.663706+0000","last_peered":"2024-03-21T08:30:06.663706+0000","last_clean":"2024-03-21T08:30:06.663706+0000","last_became_active":"2024-03-21T08:28:40.883810+0000","last_became_peered":"2024-03-21T08:28:40.883810+0000","last_unstale":"2024-03-21T08:30:06.663706+0000","last_undegraded":"2024-03-21T08:30:06.663706+0000","last_fullsized":"2024-03-21T08:30:06.663706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:12:58.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651040+0000","last_change":"2024-03-21T08:28:35.690494+0000","last_active":"2024-03-21T08:30:06.651040+0000","last_peered":"2024-03-21T08:30:06.651040+0000","last_clean":"2024-03-21T08:30:06.651040+0000","last_became_active":"2024-03-21T08:28:35.690071+0000","last_became_peered":"2024-03-21T08:28:35.690071+0000","last_unstale":"2024-03-21T08:30:06.651040+0000","last_undegraded":"2024-03-21T08:30:06.651040+0000","last_fullsized":"2024-03-21T08:30:06.651040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:04:46.713006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656927+0000","last_change":"2024-03-21T08:28:36.665135+0000","last_active":"2024-03-21T08:30:06.656927+0000","last_peered":"2024-03-21T08:30:06.656927+0000","last_clean":"2024-03-21T08:30:06.656927+0000","last_became_active":"2024-03-21T08:28:36.663700+0000","last_became_peered":"2024-03-21T08:28:36.663700+0000","last_unstale":"2024-03-21T08:30:06.656927+0000","last_undegraded":"2024-03-21T08:30:06.656927+0000","last_fullsized":"2024-03-21T08:30:06.656927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:50:00.604958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649936+0000","last_change":"2024-03-21T08:28:40.696076+0000","last_active":"2024-03-21T08:30:06.649936+0000","last_peered":"2024-03-21T08:30:06.649936+0000","last_clean":"2024-03-21T08:30:06.649936+0000","last_became_active":"2024-03-21T08:28:40.695556+0000","last_became_peered":"2024-03-21T08:28:40.695556+0000","last_unstale":"2024-03-21T08:30:06.649936+0000","last_undegraded":"2024-03-21T08:30:06.649936+0000","last_fullsized":"2024-03-21T08:30:06.649936+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:42.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646280+0000","last_change":"2024-03-21T08:28:35.685545+0000","last_active":"2024-03-21T08:30:06.646280+0000","last_peered":"2024-03-21T08:30:06.646280+0000","last_clean":"2024-03-21T08:30:06.646280+0000","last_became_active":"2024-03-21T08:28:35.684795+0000","last_became_peered":"2024-03-21T08:28:35.684795+0000","last_unstale":"2024-03-21T08:30:06.646280+0000","last_undegraded":"2024-03-21T08:30:06.646280+0000","last_fullsized":"2024-03-21T08:30:06.646280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:40.308014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.658263+0000","last_change":"2024-03-21T08:28:36.656276+0000","last_active":"2024-03-21T08:30:06.658263+0000","last_peered":"2024-03-21T08:30:06.658263+0000","last_clean":"2024-03-21T08:30:06.658263+0000","last_became_active":"2024-03-21T08:28:36.655871+0000","last_became_peered":"2024-03-21T08:28:36.655871+0000","last_unstale":"2024-03-21T08:30:06.658263+0000","last_undegraded":"2024-03-21T08:30:06.658263+0000","last_fullsized":"2024-03-21T08:30:06.658263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:22.362165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651916+0000","last_change":"2024-03-21T08:29:06.783155+0000","last_active":"2024-03-21T08:30:06.651916+0000","last_peered":"2024-03-21T08:30:06.651916+0000","last_clean":"2024-03-21T08:30:06.651916+0000","last_became_active":"2024-03-21T08:29:06.782546+0000","last_became_peered":"2024-03-21T08:29:06.782546+0000","last_unstale":"2024-03-21T08:30:06.651916+0000","last_undegraded":"2024-03-21T08:30:06.651916+0000","last_fullsized":"2024-03-21T08:30:06.651916+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:16.489888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666587+0000","last_change":"2024-03-21T08:28:35.672783+0000","last_active":"2024-03-21T08:30:06.666587+0000","last_peered":"2024-03-21T08:30:06.666587+0000","last_clean":"2024-03-21T08:30:06.666587+0000","last_became_active":"2024-03-21T08:28:35.672161+0000","last_became_peered":"2024-03-21T08:28:35.672161+0000","last_unstale":"2024-03-21T08:30:06.666587+0000","last_undegraded":"2024-03-21T08:30:06.666587+0000","last_fullsized":"2024-03-21T08:30:06.666587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:37.180625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657422+0000","last_change":"2024-03-21T08:28:36.659447+0000","last_active":"2024-03-21T08:30:06.657422+0000","last_peered":"2024-03-21T08:30:06.657422+0000","last_clean":"2024-03-21T08:30:06.657422+0000","last_became_active":"2024-03-21T08:28:36.657773+0000","last_became_peered":"2024-03-21T08:28:36.657773+0000","last_unstale":"2024-03-21T08:30:06.657422+0000","last_undegraded":"2024-03-21T08:30:06.657422+0000","last_fullsized":"2024-03-21T08:30:06.657422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:05.324498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664177+0000","last_change":"2024-03-21T08:28:40.679806+0000","last_active":"2024-03-21T08:30:06.664177+0000","last_peered":"2024-03-21T08:30:06.664177+0000","last_clean":"2024-03-21T08:30:06.664177+0000","last_became_active":"2024-03-21T08:28:40.679329+0000","last_became_peered":"2024-03-21T08:28:40.679329+0000","last_unstale":"2024-03-21T08:30:06.664177+0000","last_undegraded":"2024-03-21T08:30:06.664177+0000","last_fullsized":"2024-03-21T08:30:06.664177+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:30:58.297581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.675069+0000","last_change":"2024-03-21T08:30:06.675069+0000","last_active":"2024-03-21T08:30:06.675069+0000","last_peered":"2024-03-21T08:30:06.675069+0000","last_clean":"2024-03-21T08:30:06.675069+0000","last_became_active":"2024-03-21T08:30:06.665699+0000","last_became_peered":"2024-03-21T08:30:06.665699+0000","last_unstale":"2024-03-21T08:30:06.675069+0000","last_undegraded":"2024-03-21T08:30:06.675069+0000","last_fullsized":"2024-03-21T08:30:06.675069+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:30:07.336423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655166+0000","last_change":"2024-03-21T08:28:36.649194+0000","last_active":"2024-03-21T08:30:06.655166+0000","last_peered":"2024-03-21T08:30:06.655166+0000","last_clean":"2024-03-21T08:30:06.655166+0000","last_became_active":"2024-03-21T08:28:36.648663+0000","last_became_peered":"2024-03-21T08:28:36.648663+0000","last_unstale":"2024-03-21T08:30:06.655166+0000","last_undegraded":"2024-03-21T08:30:06.655166+0000","last_fullsized":"2024-03-21T08:30:06.655166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:16.296886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656934+0000","last_change":"2024-03-21T08:28:36.664473+0000","last_active":"2024-03-21T08:30:06.656934+0000","last_peered":"2024-03-21T08:30:06.656934+0000","last_clean":"2024-03-21T08:30:06.656934+0000","last_became_active":"2024-03-21T08:28:36.663785+0000","last_became_peered":"2024-03-21T08:28:36.663785+0000","last_unstale":"2024-03-21T08:30:06.656934+0000","last_undegraded":"2024-03-21T08:30:06.656934+0000","last_fullsized":"2024-03-21T08:30:06.656934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:00.409224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645357+0000","last_change":"2024-03-21T08:28:36.668372+0000","last_active":"2024-03-21T08:30:06.645357+0000","last_peered":"2024-03-21T08:30:06.645357+0000","last_clean":"2024-03-21T08:30:06.645357+0000","last_became_active":"2024-03-21T08:28:36.666227+0000","last_became_peered":"2024-03-21T08:28:36.666227+0000","last_unstale":"2024-03-21T08:30:06.645357+0000","last_undegraded":"2024-03-21T08:30:06.645357+0000","last_fullsized":"2024-03-21T08:30:06.645357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:40:56.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644522+0000","last_change":"2024-03-21T08:28:36.670770+0000","last_active":"2024-03-21T08:30:06.644522+0000","last_peered":"2024-03-21T08:30:06.644522+0000","last_clean":"2024-03-21T08:30:06.644522+0000","last_became_active":"2024-03-21T08:28:36.670467+0000","last_became_peered":"2024-03-21T08:28:36.670467+0000","last_unstale":"2024-03-21T08:30:06.644522+0000","last_undegraded":"2024-03-21T08:30:06.644522+0000","last_fullsized":"2024-03-21T08:30:06.644522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:48.075189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652220+0000","last_change":"2024-03-21T08:28:36.669395+0000","last_active":"2024-03-21T08:30:06.652220+0000","last_peered":"2024-03-21T08:30:06.652220+0000","last_clean":"2024-03-21T08:30:06.652220+0000","last_became_active":"2024-03-21T08:28:36.668788+0000","last_became_peered":"2024-03-21T08:28:36.668788+0000","last_unstale":"2024-03-21T08:30:06.652220+0000","last_undegraded":"2024-03-21T08:30:06.652220+0000","last_fullsized":"2024-03-21T08:30:06.652220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:03.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655160+0000","last_change":"2024-03-21T08:30:06.655160+0000","last_active":"2024-03-21T08:30:06.655160+0000","last_peered":"2024-03-21T08:30:06.655160+0000","last_clean":"2024-03-21T08:30:06.655160+0000","last_became_active":"2024-03-21T08:30:06.654744+0000","last_became_peered":"2024-03-21T08:30:06.654744+0000","last_unstale":"2024-03-21T08:30:06.655160+0000","last_undegraded":"2024-03-21T08:30:06.655160+0000","last_fullsized":"2024-03-21T08:30:06.655160+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:19:54.690562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655090+0000","last_change":"2024-03-21T08:28:36.666836+0000","last_active":"2024-03-21T08:30:06.655090+0000","last_peered":"2024-03-21T08:30:06.655090+0000","last_clean":"2024-03-21T08:30:06.655090+0000","last_became_active":"2024-03-21T08:28:36.666538+0000","last_became_peered":"2024-03-21T08:28:36.666538+0000","last_unstale":"2024-03-21T08:30:06.655090+0000","last_undegraded":"2024-03-21T08:30:06.655090+0000","last_fullsized":"2024-03-21T08:30:06.655090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:43.353269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646323+0000","last_change":"2024-03-21T08:28:36.667823+0000","last_active":"2024-03-21T08:30:06.646323+0000","last_peered":"2024-03-21T08:30:06.646323+0000","last_clean":"2024-03-21T08:30:06.646323+0000","last_became_active":"2024-03-21T08:28:36.666096+0000","last_became_peered":"2024-03-21T08:28:36.666096+0000","last_unstale":"2024-03-21T08:30:06.646323+0000","last_undegraded":"2024-03-21T08:30:06.646323+0000","last_fullsized":"2024-03-21T08:30:06.646323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:58.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648234+0000","last_change":"2024-03-21T08:28:36.672164+0000","last_active":"2024-03-21T08:30:06.648234+0000","last_peered":"2024-03-21T08:30:06.648234+0000","last_clean":"2024-03-21T08:30:06.648234+0000","last_became_active":"2024-03-21T08:28:36.671812+0000","last_became_peered":"2024-03-21T08:28:36.671812+0000","last_unstale":"2024-03-21T08:30:06.648234+0000","last_undegraded":"2024-03-21T08:30:06.648234+0000","last_fullsized":"2024-03-21T08:30:06.648234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:24:19.687108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664568+0000","last_change":"2024-03-21T08:28:36.675056+0000","last_active":"2024-03-21T08:30:06.664568+0000","last_peered":"2024-03-21T08:30:06.664568+0000","last_clean":"2024-03-21T08:30:06.664568+0000","last_became_active":"2024-03-21T08:28:36.674586+0000","last_became_peered":"2024-03-21T08:28:36.674586+0000","last_unstale":"2024-03-21T08:30:06.664568+0000","last_undegraded":"2024-03-21T08:30:06.664568+0000","last_fullsized":"2024-03-21T08:30:06.664568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:04:51.207066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645873+0000","last_change":"2024-03-21T08:28:36.666370+0000","last_active":"2024-03-21T08:30:06.645873+0000","last_peered":"2024-03-21T08:30:06.645873+0000","last_clean":"2024-03-21T08:30:06.645873+0000","last_became_active":"2024-03-21T08:28:36.665950+0000","last_became_peered":"2024-03-21T08:28:36.665950+0000","last_unstale":"2024-03-21T08:30:06.645873+0000","last_undegraded":"2024-03-21T08:30:06.645873+0000","last_fullsized":"2024-03-21T08:30:06.645873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:49:31.945890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650190+0000","last_change":"2024-03-21T08:29:07.032862+0000","last_active":"2024-03-21T08:30:06.650190+0000","last_peered":"2024-03-21T08:30:06.650190+0000","last_clean":"2024-03-21T08:30:06.650190+0000","last_became_active":"2024-03-21T08:29:07.031784+0000","last_became_peered":"2024-03-21T08:29:07.031784+0000","last_unstale":"2024-03-21T08:30:06.650190+0000","last_undegraded":"2024-03-21T08:30:06.650190+0000","last_fullsized":"2024-03-21T08:30:06.650190+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:34.037495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641937+0000","last_change":"2024-03-21T08:28:36.660812+0000","last_active":"2024-03-21T08:30:06.641937+0000","last_peered":"2024-03-21T08:30:06.641937+0000","last_clean":"2024-03-21T08:30:06.641937+0000","last_became_active":"2024-03-21T08:28:36.660044+0000","last_became_peered":"2024-03-21T08:28:36.660044+0000","last_unstale":"2024-03-21T08:30:06.641937+0000","last_undegraded":"2024-03-21T08:30:06.641937+0000","last_fullsized":"2024-03-21T08:30:06.641937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:31.598678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656562+0000","last_change":"2024-03-21T08:28:36.669441+0000","last_active":"2024-03-21T08:30:06.656562+0000","last_peered":"2024-03-21T08:30:06.656562+0000","last_clean":"2024-03-21T08:30:06.656562+0000","last_became_active":"2024-03-21T08:28:36.668668+0000","last_became_peered":"2024-03-21T08:28:36.668668+0000","last_unstale":"2024-03-21T08:30:06.656562+0000","last_undegraded":"2024-03-21T08:30:06.656562+0000","last_fullsized":"2024-03-21T08:30:06.656562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:04.828297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66334,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":513,"num_read_kb":325,"num_write":163,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285608,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":534350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547245,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1781760,"data_stored":1183951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710773,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":538682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972412,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":533924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266756,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1699840,"data_stored":1130415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462987,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":529784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":592587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":531150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1179648,"data_stored":593746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149791,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1097728,"data_stored":529956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888151,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":534412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1703936,"data_stored":1125981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:30:35.492 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T08:30:35.492 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T08:30:35.492 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T08:30:35.492 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T08:30:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:36.329 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:35 smithi003 ceph-mon[31675]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:30:36.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:35 smithi084 ceph-mon[35477]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:30:36.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:35 smithi171 ceph-mon[35388]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:30:37.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:36 smithi003 ceph-mon[31675]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:37.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:36 smithi003 ceph-mon[31675]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-21T08:30:37.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:36 smithi084 ceph-mon[35477]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:37.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:36 smithi084 ceph-mon[35477]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-21T08:30:37.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:36 smithi171 ceph-mon[35388]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:37.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:36 smithi171 ceph-mon[35388]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 -- 172.21.15.3:0/2811183161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63881074f0 msgr2=0x7f638810f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 --2- 172.21.15.3:0/2811183161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63881074f0 0x7f638810f0c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f638000b0d0 tx=0x7f638000b3e0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 -- 172.21.15.3:0/2811183161 shutdown_connections 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 --2- 172.21.15.3:0/2811183161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63881074f0 0x7f638810f0c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 --2- 172.21.15.3:0/2811183161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388102410 0x7f63881027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 --2- 172.21.15.3:0/2811183161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6388102dc0 0x7f6388106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 -- 172.21.15.3:0/2811183161 >> 172.21.15.3:0/2811183161 conn(0x7f63880fdc60 msgr2=0x7f6388100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 -- 172.21.15.3:0/2811183161 shutdown_connections 2024-03-21T08:30:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.107+0000 7f638d600700 1 -- 172.21.15.3:0/2811183161 wait complete. 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.108+0000 7f638d600700 1 Processor -- start 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f638d600700 1 -- start start 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f638d600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6388102410 0x7f638819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f638d600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388102dc0 0x7f638819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f638d600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63881074f0 0x7f6388198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f638d600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881123b0 con 0x7f6388102410 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f638d600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388112230 con 0x7f6388102dc0 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f638d600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388112530 con 0x7f63881074f0 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f6386a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6388102410 0x7f638819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f6386a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6388102410 0x7f638819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50730/0 (socket says 172.21.15.3:50730) 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.109+0000 7f6386a00700 1 -- 172.21.15.3:0/1825019663 learned_addr learned my addr 172.21.15.3:0/1825019663 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f637fe00700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388102dc0 0x7f638819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f6387400700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63881074f0 0x7f6388198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f6386a00700 1 -- 172.21.15.3:0/1825019663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63881074f0 msgr2=0x7f6388198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f6386a00700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63881074f0 0x7f6388198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f6386a00700 1 -- 172.21.15.3:0/1825019663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388102dc0 msgr2=0x7f638819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f6386a00700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388102dc0 0x7f638819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f6386a00700 1 -- 172.21.15.3:0/1825019663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388198f50 con 0x7f6388102410 2024-03-21T08:30:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f6386a00700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6388102410 0x7f638819e5a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f637800d950 tx=0x7f637800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.110+0000 7f637f400700 1 -- 172.21.15.3:0/1825019663 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378016070 con 0x7f6388102410 2024-03-21T08:30:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.111+0000 7f637f400700 1 -- 172.21.15.3:0/1825019663 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6378010460 con 0x7f6388102410 2024-03-21T08:30:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.111+0000 7f637f400700 1 -- 172.21.15.3:0/1825019663 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637800f620 con 0x7f6388102410 2024-03-21T08:30:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.111+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388199240 con 0x7f6388102410 2024-03-21T08:30:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.111+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63881a5520 con 0x7f6388102410 2024-03-21T08:30:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.112+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63881038f0 con 0x7f6388102410 2024-03-21T08:30:38.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.114+0000 7f637f400700 1 -- 172.21.15.3:0/1825019663 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f6378019070 con 0x7f6388102410 2024-03-21T08:30:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.115+0000 7f637f400700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f636c0a2fb0 0x7f636c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.115+0000 7f637f400700 1 -- 172.21.15.3:0/1825019663 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f63780cf490 con 0x7f6388102410 2024-03-21T08:30:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.116+0000 7f637fe00700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f636c0a2fb0 0x7f636c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.116+0000 7f637fe00700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f636c0a2fb0 0x7f636c0a5470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6388199e30 tx=0x7f637000b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:38.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.119+0000 7f637f400700 1 -- 172.21.15.3:0/1825019663 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f637809be80 con 0x7f6388102410 2024-03-21T08:30:38.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.395+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6388199ba0 con 0x7f6388102410 2024-03-21T08:30:38.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.396+0000 7f637f400700 1 -- 172.21.15.3:0/1825019663 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f637809b780 con 0x7f6388102410 2024-03-21T08:30:38.398 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:38.398 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T08:30:38.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f636c0a2fb0 msgr2=0x7f636c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:38.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f636c0a2fb0 0x7f636c0a5470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6388199e30 tx=0x7f637000b590 comp rx=0 tx=0).stop 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6388102410 msgr2=0x7f638819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6388102410 0x7f638819e5a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f637800d950 tx=0x7f637800dd10 comp rx=0 tx=0).stop 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 shutdown_connections 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f636c0a2fb0 0x7f636c0a5470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6388102410 0x7f638819e5a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63881074f0 0x7f6388198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 --2- 172.21.15.3:0/1825019663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388102dc0 0x7f638819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.399+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 >> 172.21.15.3:0/1825019663 conn(0x7f63880fdc60 msgr2=0x7f63880fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.400+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 shutdown_connections 2024-03-21T08:30:38.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:38.400+0000 7f638d600700 1 -- 172.21.15.3:0/1825019663 wait complete. 2024-03-21T08:30:39.081 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:38 smithi003 ceph-mon[31675]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:39.082 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:38 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1825019663' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T08:30:39.143 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T08:30:39.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:30:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:38 smithi084 ceph-mon[35477]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:39.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:38 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1825019663' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T08:30:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 -- 172.21.15.3:0/1589835557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea80756e0 msgr2=0x7f9ea8073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 --2- 172.21.15.3:0/1589835557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea80756e0 0x7f9ea8073b30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009930 tx=0x7f9e9802bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 -- 172.21.15.3:0/1589835557 shutdown_connections 2024-03-21T08:30:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 --2- 172.21.15.3:0/1589835557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea810dd00 0x7f9ea81100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 --2- 172.21.15.3:0/1589835557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea8074100 0x7f9ea8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 --2- 172.21.15.3:0/1589835557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea80756e0 0x7f9ea8073b30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 -- 172.21.15.3:0/1589835557 >> 172.21.15.3:0/1589835557 conn(0x7f9ea80fccc0 msgr2=0x7f9ea80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 -- 172.21.15.3:0/1589835557 shutdown_connections 2024-03-21T08:30:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.375+0000 7f9ead000640 1 -- 172.21.15.3:0/1589835557 wait complete. 2024-03-21T08:30:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 Processor -- start 2024-03-21T08:30:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 -- start start 2024-03-21T08:30:39.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8074100 0x7f9ea819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea80756e0 0x7f9ea819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea810dd00 0x7f9ea81a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81137e0 con 0x7f9ea810dd00 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8113660 con 0x7f9ea80756e0 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ead000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8113960 con 0x7f9ea8074100 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.376+0000 7f9ea6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea810dd00 0x7f9ea81a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea810dd00 0x7f9ea81a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50748/0 (socket says 172.21.15.3:50748) 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 -- 172.21.15.3:0/3807261896 learned_addr learned my addr 172.21.15.3:0/3807261896 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea5e00640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8074100 0x7f9ea819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea5400640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea80756e0 0x7f9ea819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 -- 172.21.15.3:0/3807261896 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8074100 msgr2=0x7f9ea819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8074100 0x7f9ea819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 -- 172.21.15.3:0/3807261896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea80756e0 msgr2=0x7f9ea819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea80756e0 0x7f9ea819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 -- 172.21.15.3:0/3807261896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea81a5b20 con 0x7f9ea810dd00 2024-03-21T08:30:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.377+0000 7f9ea6800640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea810dd00 0x7f9ea81a54b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000eaa0 tx=0x7f9ea000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.378+0000 7f9e9ea00640 1 -- 172.21.15.3:0/3807261896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000cdb0 con 0x7f9ea810dd00 2024-03-21T08:30:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.378+0000 7f9e9ea00640 1 -- 172.21.15.3:0/3807261896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ea0005020 con 0x7f9ea810dd00 2024-03-21T08:30:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.378+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea81a5e10 con 0x7f9ea810dd00 2024-03-21T08:30:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.378+0000 7f9e9ea00640 1 -- 172.21.15.3:0/3807261896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00116d0 con 0x7f9ea810dd00 2024-03-21T08:30:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.378+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea81a62d0 con 0x7f9ea810dd00 2024-03-21T08:30:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.379+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea8103ed0 con 0x7f9ea810dd00 2024-03-21T08:30:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:38 smithi171 ceph-mon[35388]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:39.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:38 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1825019663' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T08:30:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.381+0000 7f9e9ea00640 1 -- 172.21.15.3:0/3807261896 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f9ea00051d0 con 0x7f9ea810dd00 2024-03-21T08:30:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.383+0000 7f9e9ea00640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e8c0a2f20 0x7f9e8c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.383+0000 7f9e9ea00640 1 -- 172.21.15.3:0/3807261896 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f9ea00cfaf0 con 0x7f9ea810dd00 2024-03-21T08:30:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.383+0000 7f9ea5e00640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e8c0a2f20 0x7f9e8c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.384+0000 7f9ea5e00640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e8c0a2f20 0x7f9e8c0a53e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9e98006010 tx=0x7f9e9802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.387+0000 7f9e9ea00640 1 -- 172.21.15.3:0/3807261896 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ea0016030 con 0x7f9ea810dd00 2024-03-21T08:30:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.593+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ea8062340 con 0x7f9ea810dd00 2024-03-21T08:30:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.596+0000 7f9e9ea00640 1 -- 172.21.15.3:0/3807261896 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7f9ea009c480 con 0x7f9ea810dd00 2024-03-21T08:30:39.598 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:39.599 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:30:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e8c0a2f20 msgr2=0x7f9e8c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e8c0a2f20 0x7f9e8c0a53e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9e98006010 tx=0x7f9e9802c800 comp rx=0 tx=0).stop 2024-03-21T08:30:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea810dd00 msgr2=0x7f9ea81a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea810dd00 0x7f9ea81a54b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000eaa0 tx=0x7f9ea000ef60 comp rx=0 tx=0).stop 2024-03-21T08:30:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 shutdown_connections 2024-03-21T08:30:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ea810dd00 0x7f9ea81a54b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea80756e0 0x7f9ea819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9e8c0a2f20 0x7f9e8c0a53e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 --2- 172.21.15.3:0/3807261896 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8074100 0x7f9ea819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 >> 172.21.15.3:0/3807261896 conn(0x7f9ea80fccc0 msgr2=0x7f9ea81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 shutdown_connections 2024-03-21T08:30:39.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.599+0000 7f9ead000640 1 -- 172.21.15.3:0/3807261896 wait complete. 2024-03-21T08:30:39.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:30:39.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:30:39.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 -- 172.21.15.3:0/4291910259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc105480 msgr2=0x7f0cbc10bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/4291910259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc105480 0x7f0cbc10bd10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009ca0 tx=0x7f0cb402c100 comp rx=0 tx=0).stop 2024-03-21T08:30:39.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 -- 172.21.15.3:0/4291910259 shutdown_connections 2024-03-21T08:30:39.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/4291910259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc105480 0x7f0cbc10bd10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/4291910259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc104ac0 0x7f0cbc104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/4291910259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc1038d0 0x7f0cbc103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 -- 172.21.15.3:0/4291910259 >> 172.21.15.3:0/4291910259 conn(0x7f0cbc0fee60 msgr2=0x7f0cbc101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:39.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 -- 172.21.15.3:0/4291910259 shutdown_connections 2024-03-21T08:30:39.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.851+0000 7f0cc3e00640 1 -- 172.21.15.3:0/4291910259 wait complete. 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.852+0000 7f0cc3e00640 1 Processor -- start 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.852+0000 7f0cc3e00640 1 -- start start 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc1038d0 0x7f0cbc199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc104ac0 0x7f0cbc199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc105480 0x7f0cbc1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc3e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc10f400 con 0x7f0cbc104ac0 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc10f280 con 0x7f0cbc105480 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc3e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc10f580 con 0x7f0cbc1038d0 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc104ac0 0x7f0cbc199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc104ac0 0x7f0cbc199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50766/0 (socket says 172.21.15.3:50766) 2024-03-21T08:30:39.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc0a00640 1 -- 172.21.15.3:0/1658297530 learned_addr learned my addr 172.21.15.3:0/1658297530 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc1e00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc105480 0x7f0cbc1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.853+0000 7f0cc1400640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc1038d0 0x7f0cbc199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc0a00640 1 -- 172.21.15.3:0/1658297530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc1038d0 msgr2=0x7f0cbc199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc0a00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc1038d0 0x7f0cbc199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc0a00640 1 -- 172.21.15.3:0/1658297530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc105480 msgr2=0x7f0cbc1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc0a00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc105480 0x7f0cbc1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc0a00640 1 -- 172.21.15.3:0/1658297530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc1a16c0 con 0x7f0cbc104ac0 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc0a00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc104ac0 0x7f0cbc199fe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800d950 tx=0x7f0ca800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cba000640 1 -- 172.21.15.3:0/1658297530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8017070 con 0x7f0cbc104ac0 2024-03-21T08:30:39.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cba000640 1 -- 172.21.15.3:0/1658297530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0ca8009c30 con 0x7f0cbc104ac0 2024-03-21T08:30:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc1a19b0 con 0x7f0cbc104ac0 2024-03-21T08:30:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cba000640 1 -- 172.21.15.3:0/1658297530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800f620 con 0x7f0cbc104ac0 2024-03-21T08:30:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.854+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc1a1f00 con 0x7f0cbc104ac0 2024-03-21T08:30:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.856+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc10abf0 con 0x7f0cbc104ac0 2024-03-21T08:30:39.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.858+0000 7f0cba000640 1 -- 172.21.15.3:0/1658297530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f0ca80109a0 con 0x7f0cbc104ac0 2024-03-21T08:30:39.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.860+0000 7f0cba000640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0ca40a2f20 0x7f0ca40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:39.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.861+0000 7f0cba000640 1 -- 172.21.15.3:0/1658297530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0ca80cf550 con 0x7f0cbc104ac0 2024-03-21T08:30:39.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.861+0000 7f0cc1400640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0ca40a2f20 0x7f0ca40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:39.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.861+0000 7f0cc1400640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0ca40a2f20 0x7f0ca40a53e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0005fd0 tx=0x7f0cb0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:39.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:39.865+0000 7f0cba000640 1 -- 172.21.15.3:0/1658297530 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ca8009de0 con 0x7f0cbc104ac0 2024-03-21T08:30:40.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.072+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0cbc0622d0 con 0x7f0cbc104ac0 2024-03-21T08:30:40.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.073+0000 7f0cba000640 1 -- 172.21.15.3:0/1658297530 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ca809bee0 con 0x7f0cbc104ac0 2024-03-21T08:30:40.075 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0ca40a2f20 msgr2=0x7f0ca40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0ca40a2f20 0x7f0ca40a53e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0005fd0 tx=0x7f0cb0005850 comp rx=0 tx=0).stop 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc104ac0 msgr2=0x7f0cbc199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc104ac0 0x7f0cbc199fe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800d950 tx=0x7f0ca800de10 comp rx=0 tx=0).stop 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 shutdown_connections 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc105480 0x7f0cbc1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cbc104ac0 0x7f0cbc199fe0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f0ca40a2f20 0x7f0ca40a53e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 --2- 172.21.15.3:0/1658297530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc1038d0 0x7f0cbc199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 >> 172.21.15.3:0/1658297530 conn(0x7f0cbc0fee60 msgr2=0x7f0cbc107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:40.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 shutdown_connections 2024-03-21T08:30:40.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.076+0000 7f0cc3e00640 1 -- 172.21.15.3:0/1658297530 wait complete. 2024-03-21T08:30:40.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:30:40.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 -- 172.21.15.3:0/747225057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181016c0 msgr2=0x7f8218101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 --2- 172.21.15.3:0/747225057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181016c0 0x7f8218101ae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f820c009900 tx=0x7f820c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 -- 172.21.15.3:0/747225057 shutdown_connections 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 --2- 172.21.15.3:0/747225057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8218103270 0x7f8218109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 --2- 172.21.15.3:0/747225057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82181028b0 0x7f8218102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 --2- 172.21.15.3:0/747225057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181016c0 0x7f8218101ae0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 -- 172.21.15.3:0/747225057 >> 172.21.15.3:0/747225057 conn(0x7f82180fcc30 msgr2=0x7f82180ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 -- 172.21.15.3:0/747225057 shutdown_connections 2024-03-21T08:30:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.319+0000 7f8220000640 1 -- 172.21.15.3:0/747225057 wait complete. 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.320+0000 7f8220000640 1 Processor -- start 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f8220000640 1 -- start start 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f8220000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82181016c0 0x7f8218199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f8220000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181028b0 0x7f821819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f8220000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8218103270 0x7f82181a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f8220000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821810d230 con 0x7f82181028b0 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f8220000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821810d0b0 con 0x7f8218103270 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f8220000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821810d3b0 con 0x7f82181016c0 2024-03-21T08:30:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f821cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181028b0 0x7f821819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f821cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181028b0 0x7f821819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34198/0 (socket says 172.21.15.3:34198) 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.321+0000 7f821cc00640 1 -- 172.21.15.3:0/887029155 learned_addr learned my addr 172.21.15.3:0/887029155 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821e000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8218103270 0x7f82181a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821d600640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82181016c0 0x7f8218199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821cc00640 1 -- 172.21.15.3:0/887029155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82181016c0 msgr2=0x7f8218199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821cc00640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82181016c0 0x7f8218199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821cc00640 1 -- 172.21.15.3:0/887029155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8218103270 msgr2=0x7f82181a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821cc00640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8218103270 0x7f82181a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821cc00640 1 -- 172.21.15.3:0/887029155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82181a1780 con 0x7f82181028b0 2024-03-21T08:30:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.322+0000 7f821cc00640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181028b0 0x7f821819a0f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f820400d770 tx=0x7f820400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.323+0000 7f8216000640 1 -- 172.21.15.3:0/887029155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204017070 con 0x7f82181028b0 2024-03-21T08:30:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.323+0000 7f8216000640 1 -- 172.21.15.3:0/887029155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82040054d0 con 0x7f82181028b0 2024-03-21T08:30:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.323+0000 7f8216000640 1 -- 172.21.15.3:0/887029155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204004cf0 con 0x7f82181028b0 2024-03-21T08:30:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.323+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82181a1a70 con 0x7f82181028b0 2024-03-21T08:30:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.323+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82181a1f70 con 0x7f82181028b0 2024-03-21T08:30:40.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.324+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82181a21a0 con 0x7f82181028b0 2024-03-21T08:30:40.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.329+0000 7f8216000640 1 -- 172.21.15.3:0/887029155 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8204004e90 con 0x7f82181028b0 2024-03-21T08:30:40.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.332+0000 7f8216000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82000a2fa0 0x7f82000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.332+0000 7f8216000640 1 -- 172.21.15.3:0/887029155 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f82040cf1a0 con 0x7f82181028b0 2024-03-21T08:30:40.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.332+0000 7f821d600640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82000a2fa0 0x7f82000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.333+0000 7f821d600640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82000a2fa0 0x7f82000a5460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f820c02c240 tx=0x7f820c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:40.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.338+0000 7f8216000640 1 -- 172.21.15.3:0/887029155 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f820409b4e0 con 0x7f82181028b0 2024-03-21T08:30:40.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:39 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3807261896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:40.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:39 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3807261896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:40.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:39 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3807261896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:40.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.551+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82181a2370 con 0x7f82181028b0 2024-03-21T08:30:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.552+0000 7f8216000640 1 -- 172.21.15.3:0/887029155 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f820409f580 con 0x7f82181028b0 2024-03-21T08:30:40.554 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:30:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82000a2fa0 msgr2=0x7f82000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82000a2fa0 0x7f82000a5460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f820c02c240 tx=0x7f820c02c7e0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181028b0 msgr2=0x7f821819a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181028b0 0x7f821819a0f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f820400d770 tx=0x7f820400dc30 comp rx=0 tx=0).stop 2024-03-21T08:30:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 shutdown_connections 2024-03-21T08:30:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8218103270 0x7f82181a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82181028b0 0x7f821819a0f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f82000a2fa0 0x7f82000a5460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 --2- 172.21.15.3:0/887029155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82181016c0 0x7f8218199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 >> 172.21.15.3:0/887029155 conn(0x7f82180fcc30 msgr2=0x7f82180738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 shutdown_connections 2024-03-21T08:30:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.555+0000 7f8220000640 1 -- 172.21.15.3:0/887029155 wait complete. 2024-03-21T08:30:40.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:30:40.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 -- 172.21.15.3:0/3240359558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0756e0 msgr2=0x7f307c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 --2- 172.21.15.3:0/3240359558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0756e0 0x7f307c073b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f306c009930 tx=0x7f306c02bc20 comp rx=0 tx=0).stop 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 -- 172.21.15.3:0/3240359558 shutdown_connections 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 --2- 172.21.15.3:0/3240359558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f307c10dad0 0x7f307c10fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 --2- 172.21.15.3:0/3240359558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307c074100 0x7f307c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 --2- 172.21.15.3:0/3240359558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c0756e0 0x7f307c073b30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 -- 172.21.15.3:0/3240359558 >> 172.21.15.3:0/3240359558 conn(0x7f307c0fcab0 msgr2=0x7f307c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 -- 172.21.15.3:0/3240359558 shutdown_connections 2024-03-21T08:30:40.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.784+0000 7f3080a00640 1 -- 172.21.15.3:0/3240359558 wait complete. 2024-03-21T08:30:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.785+0000 7f3080a00640 1 Processor -- start 2024-03-21T08:30:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.785+0000 7f3080a00640 1 -- start start 2024-03-21T08:30:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f3080a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c074100 0x7f307c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f3080a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307c0756e0 0x7f307c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f3080a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f307c10dad0 0x7f307c1a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f3080a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c113650 con 0x7f307c074100 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f3080a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1134d0 con 0x7f307c0756e0 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f3080a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1137d0 con 0x7f307c10dad0 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f307a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c074100 0x7f307c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f307a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c074100 0x7f307c19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34206/0 (socket says 172.21.15.3:34206) 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f307a800640 1 -- 172.21.15.3:0/1253120050 learned_addr learned my addr 172.21.15.3:0/1253120050 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f3079e00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307c0756e0 0x7f307c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.786+0000 7f307b200640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f307c10dad0 0x7f307c1a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f307a800640 1 -- 172.21.15.3:0/1253120050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f307c10dad0 msgr2=0x7f307c1a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f307a800640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f307c10dad0 0x7f307c1a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f307a800640 1 -- 172.21.15.3:0/1253120050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307c0756e0 msgr2=0x7f307c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f307a800640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307c0756e0 0x7f307c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f307a800640 1 -- 172.21.15.3:0/1253120050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c1a5930 con 0x7f307c074100 2024-03-21T08:30:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f3079e00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307c0756e0 0x7f307c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f307a800640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c074100 0x7f307c19dd20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f306c000c00 tx=0x7f306c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f3073400640 1 -- 172.21.15.3:0/1253120050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c043020 con 0x7f307c074100 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f3073400640 1 -- 172.21.15.3:0/1253120050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f306c0044c0 con 0x7f307c074100 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c1a5bc0 con 0x7f307c074100 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.787+0000 7f3073400640 1 -- 172.21.15.3:0/1253120050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c003c60 con 0x7f307c074100 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.788+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c1a6160 con 0x7f307c074100 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.789+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c04f400 con 0x7f307c074100 2024-03-21T08:30:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.791+0000 7f3073400640 1 -- 172.21.15.3:0/1253120050 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f306c046020 con 0x7f307c074100 2024-03-21T08:30:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.793+0000 7f3073400640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f30600a2fa0 0x7f30600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.793+0000 7f3073400640 1 -- 172.21.15.3:0/1253120050 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f306c0f1a20 con 0x7f307c074100 2024-03-21T08:30:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.793+0000 7f3079e00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f30600a2fa0 0x7f30600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.794+0000 7f3079e00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f30600a2fa0 0x7f30600a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fc0 tx=0x7f3068005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:40.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:40.797+0000 7f3073400640 1 -- 172.21.15.3:0/1253120050 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f306c04d020 con 0x7f307c074100 2024-03-21T08:30:41.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.001+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f307c0622a0 con 0x7f307c074100 2024-03-21T08:30:41.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.002+0000 7f3073400640 1 -- 172.21.15.3:0/1253120050 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f306c0a8070 con 0x7f307c074100 2024-03-21T08:30:41.004 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:30:41.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.005+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f30600a2fa0 msgr2=0x7f30600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.005+0000 7f3080a00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f30600a2fa0 0x7f30600a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fc0 tx=0x7f3068005f90 comp rx=0 tx=0).stop 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.005+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c074100 msgr2=0x7f307c19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.005+0000 7f3080a00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c074100 0x7f307c19dd20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f306c000c00 tx=0x7f306c0040c0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 shutdown_connections 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f30600a2fa0 0x7f30600a5460 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f307c10dad0 0x7f307c1a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307c0756e0 0x7f307c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 --2- 172.21.15.3:0/1253120050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f307c074100 0x7f307c19dd20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 >> 172.21.15.3:0/1253120050 conn(0x7f307c0fcab0 msgr2=0x7f307c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 shutdown_connections 2024-03-21T08:30:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.006+0000 7f3080a00640 1 -- 172.21.15.3:0/1253120050 wait complete. 2024-03-21T08:30:41.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:30:41.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:40 smithi003 ceph-mon[31675]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:41.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:40 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1658297530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:30:41.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:40 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/887029155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:30:41.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 -- 172.21.15.3:0/57298158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630102700 msgr2=0x7f9630102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 --2- 172.21.15.3:0/57298158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630102700 0x7f9630102b80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f961c009900 tx=0x7f961c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:41.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 -- 172.21.15.3:0/57298158 shutdown_connections 2024-03-21T08:30:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 --2- 172.21.15.3:0/57298158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96301030c0 0x7f9630109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 --2- 172.21.15.3:0/57298158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630102700 0x7f9630102b80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 --2- 172.21.15.3:0/57298158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9630101510 0x7f9630101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 -- 172.21.15.3:0/57298158 >> 172.21.15.3:0/57298158 conn(0x7f96300fca80 msgr2=0x7f96300feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 -- 172.21.15.3:0/57298158 shutdown_connections 2024-03-21T08:30:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.219+0000 7f9635a00640 1 -- 172.21.15.3:0/57298158 wait complete. 2024-03-21T08:30:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 Processor -- start 2024-03-21T08:30:41.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 -- start start 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630101510 0x7f9630075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9630102700 0x7f9630073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96301030c0 0x7f9630073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963010d0a0 con 0x7f9630101510 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963010cf20 con 0x7f9630102700 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.220+0000 7f9635a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963010d220 con 0x7f96301030c0 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630101510 0x7f9630075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630101510 0x7f9630075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34222/0 (socket says 172.21.15.3:34222) 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 -- 172.21.15.3:0/818549046 learned_addr learned my addr 172.21.15.3:0/818549046 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962f400640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9630102700 0x7f9630073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f9635000640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96301030c0 0x7f9630073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 -- 172.21.15.3:0/818549046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96301030c0 msgr2=0x7f9630073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96301030c0 0x7f9630073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 -- 172.21.15.3:0/818549046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9630102700 msgr2=0x7f9630073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9630102700 0x7f9630073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f962fe00640 1 -- 172.21.15.3:0/818549046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630074860 con 0x7f9630101510 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.221+0000 7f9635000640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96301030c0 0x7f9630073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.222+0000 7f962fe00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630101510 0x7f9630075450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f962000ebc0 tx=0x7f962000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.222+0000 7f962cc00640 1 -- 172.21.15.3:0/818549046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620003b70 con 0x7f9630101510 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.222+0000 7f962cc00640 1 -- 172.21.15.3:0/818549046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9620003d10 con 0x7f9630101510 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.222+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96301a17c0 con 0x7f9630101510 2024-03-21T08:30:41.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.222+0000 7f962cc00640 1 -- 172.21.15.3:0/818549046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620019840 con 0x7f9630101510 2024-03-21T08:30:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.222+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96301a1ce0 con 0x7f9630101510 2024-03-21T08:30:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.223+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963004f400 con 0x7f9630101510 2024-03-21T08:30:41.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.227+0000 7f962cc00640 1 -- 172.21.15.3:0/818549046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f96200199e0 con 0x7f9630101510 2024-03-21T08:30:41.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.228+0000 7f962cc00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96140a2f20 0x7f96140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.228+0000 7f962cc00640 1 -- 172.21.15.3:0/818549046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f96200d03a0 con 0x7f9630101510 2024-03-21T08:30:41.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.228+0000 7f962f400640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96140a2f20 0x7f96140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.229+0000 7f962f400640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96140a2f20 0x7f96140a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f961c006010 tx=0x7f961c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:41.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.231+0000 7f962cc00640 1 -- 172.21.15.3:0/818549046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f962009ce20 con 0x7f9630101510 2024-03-21T08:30:41.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:40 smithi084 ceph-mon[35477]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:41.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:40 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1658297530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:30:41.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:40 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/887029155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:30:41.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:40 smithi171 ceph-mon[35388]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:41.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:40 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1658297530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:30:41.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:40 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/887029155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:30:41.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.430+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f963002d070 con 0x7f9630101510 2024-03-21T08:30:41.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.431+0000 7f962cc00640 1 -- 172.21.15.3:0/818549046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f962009c720 con 0x7f9630101510 2024-03-21T08:30:41.433 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.433+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96140a2f20 msgr2=0x7f96140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.433+0000 7f9635a00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96140a2f20 0x7f96140a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f961c006010 tx=0x7f961c000bc0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.433+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630101510 msgr2=0x7f9630075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.433+0000 7f9635a00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630101510 0x7f9630075450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f962000ebc0 tx=0x7f962000c6a0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 shutdown_connections 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96140a2f20 0x7f96140a53e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96301030c0 0x7f9630073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9630102700 0x7f9630073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 --2- 172.21.15.3:0/818549046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9630101510 0x7f9630075450 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 >> 172.21.15.3:0/818549046 conn(0x7f96300fca80 msgr2=0x7f96301059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 shutdown_connections 2024-03-21T08:30:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.434+0000 7f9635a00640 1 -- 172.21.15.3:0/818549046 wait complete. 2024-03-21T08:30:41.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:30:41.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 -- 172.21.15.3:0/3063613992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370100780 msgr2=0x7f2370100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 --2- 172.21.15.3:0/3063613992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370100780 0x7f2370100be0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ca0 tx=0x7f236002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 -- 172.21.15.3:0/3063613992 shutdown_connections 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 --2- 172.21.15.3:0/3063613992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2370101120 0x7f23701015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 --2- 172.21.15.3:0/3063613992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370100780 0x7f2370100be0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 --2- 172.21.15.3:0/3063613992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23700ffe20 0x7f2370100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 -- 172.21.15.3:0/3063613992 >> 172.21.15.3:0/3063613992 conn(0x7f23700fba20 msgr2=0x7f23700fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 -- 172.21.15.3:0/3063613992 shutdown_connections 2024-03-21T08:30:41.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.660+0000 7f2375e00640 1 -- 172.21.15.3:0/3063613992 wait complete. 2024-03-21T08:30:41.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.661+0000 7f2375e00640 1 Processor -- start 2024-03-21T08:30:41.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.661+0000 7f2375e00640 1 -- start start 2024-03-21T08:30:41.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23700ffe20 0x7f23701998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2370100780 0x7f2370199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370101120 0x7f23701a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237010f2a0 con 0x7f2370101120 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237010f120 con 0x7f2370100780 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237010f420 con 0x7f23700ffe20 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370101120 0x7f23701a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370101120 0x7f23701a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34246/0 (socket says 172.21.15.3:34246) 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2375400640 1 -- 172.21.15.3:0/3201536700 learned_addr learned my addr 172.21.15.3:0/3201536700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f236fe00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2370100780 0x7f2370199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.662+0000 7f2374a00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23700ffe20 0x7f23701998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f2375400640 1 -- 172.21.15.3:0/3201536700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23700ffe20 msgr2=0x7f23701998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f2375400640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23700ffe20 0x7f23701998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f2375400640 1 -- 172.21.15.3:0/3201536700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2370100780 msgr2=0x7f2370199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f2375400640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2370100780 0x7f2370199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f2375400640 1 -- 172.21.15.3:0/3201536700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23701a1500 con 0x7f2370101120 2024-03-21T08:30:41.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f2375400640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370101120 0x7f23701a0e90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f236800c970 tx=0x7f236800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:41.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f236d600640 1 -- 172.21.15.3:0/3201536700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368007bb0 con 0x7f2370101120 2024-03-21T08:30:41.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f236d600640 1 -- 172.21.15.3:0/3201536700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2368007d50 con 0x7f2370101120 2024-03-21T08:30:41.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f236d600640 1 -- 172.21.15.3:0/3201536700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368018700 con 0x7f2370101120 2024-03-21T08:30:41.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.663+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23701a17f0 con 0x7f2370101120 2024-03-21T08:30:41.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.664+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23701a1cc0 con 0x7f2370101120 2024-03-21T08:30:41.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.665+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237010ab70 con 0x7f2370101120 2024-03-21T08:30:41.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.667+0000 7f236d600640 1 -- 172.21.15.3:0/3201536700 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f236800fa20 con 0x7f2370101120 2024-03-21T08:30:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.669+0000 7f236d600640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f23540a2f90 0x7f23540a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.669+0000 7f236d600640 1 -- 172.21.15.3:0/3201536700 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f23680cf8f0 con 0x7f2370101120 2024-03-21T08:30:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.669+0000 7f2374a00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f23540a2f90 0x7f23540a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.670+0000 7f2374a00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f23540a2f90 0x7f23540a5450 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2358005e10 tx=0x7f2358005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:41.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.672+0000 7f236d600640 1 -- 172.21.15.3:0/3201536700 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2368014030 con 0x7f2370101120 2024-03-21T08:30:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.914+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f237002d070 con 0x7f2370101120 2024-03-21T08:30:41.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.915+0000 7f236d600640 1 -- 172.21.15.3:0/3201536700 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f236809c280 con 0x7f2370101120 2024-03-21T08:30:41.917 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:41.918 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:29:09.323523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34409,34472]},"id":1}]} 2024-03-21T08:30:41.918 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-03-21T08:30:41.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.918+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f23540a2f90 msgr2=0x7f23540a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.918+0000 7f2375e00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f23540a2f90 0x7f23540a5450 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2358005e10 tx=0x7f2358005da0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.918+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370101120 msgr2=0x7f23701a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.918+0000 7f2375e00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370101120 0x7f23701a0e90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f236800c970 tx=0x7f236800ce30 comp rx=0 tx=0).stop 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 shutdown_connections 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2370101120 0x7f23701a0e90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2370100780 0x7f2370199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f23540a2f90 0x7f23540a5450 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 --2- 172.21.15.3:0/3201536700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23700ffe20 0x7f23701998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 >> 172.21.15.3:0/3201536700 conn(0x7f23700fba20 msgr2=0x7f23700fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 shutdown_connections 2024-03-21T08:30:41.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:41.919+0000 7f2375e00640 1 -- 172.21.15.3:0/3201536700 wait complete. 2024-03-21T08:30:41.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:30:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 -- 172.21.15.3:0/2895071037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83401038d0 msgr2=0x7f8340103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 --2- 172.21.15.3:0/2895071037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83401038d0 0x7f8340103cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8330009900 tx=0x7f833002bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 -- 172.21.15.3:0/2895071037 shutdown_connections 2024-03-21T08:30:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 --2- 172.21.15.3:0/2895071037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8340105480 0x7f834010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 --2- 172.21.15.3:0/2895071037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 0x7f8340104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 --2- 172.21.15.3:0/2895071037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83401038d0 0x7f8340103cf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 -- 172.21.15.3:0/2895071037 >> 172.21.15.3:0/2895071037 conn(0x7f83400fee60 msgr2=0x7f8340101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 -- 172.21.15.3:0/2895071037 shutdown_connections 2024-03-21T08:30:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.162+0000 7f8346c00640 1 -- 172.21.15.3:0/2895071037 wait complete. 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.163+0000 7f8346c00640 1 Processor -- start 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8346c00640 1 -- start start 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8346c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83401038d0 0x7f8340199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8346c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 0x7f834019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8346c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8340105480 0x7f83401a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8346c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834010f400 con 0x7f8340104ac0 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8346c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834010f280 con 0x7f8340105480 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8346c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834010f580 con 0x7f83401038d0 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8337e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 0x7f834019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8337e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 0x7f834019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34262/0 (socket says 172.21.15.3:34262) 2024-03-21T08:30:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.164+0000 7f8337e00640 1 -- 172.21.15.3:0/287420801 learned_addr learned my addr 172.21.15.3:0/287420801 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f833fe00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83401038d0 0x7f8340199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8344c00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8340105480 0x7f83401a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8337e00640 1 -- 172.21.15.3:0/287420801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83401038d0 msgr2=0x7f8340199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8337e00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83401038d0 0x7f8340199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8337e00640 1 -- 172.21.15.3:0/287420801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8340105480 msgr2=0x7f83401a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8337e00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8340105480 0x7f83401a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8337e00640 1 -- 172.21.15.3:0/287420801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83401a1700 con 0x7f8340104ac0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8337e00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 0x7f834019a020 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f832800ea60 tx=0x7f832800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f833d600640 1 -- 172.21.15.3:0/287420801 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800cde0 con 0x7f8340104ac0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f833d600640 1 -- 172.21.15.3:0/287420801 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83280035f0 con 0x7f8340104ac0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.165+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83401a19f0 con 0x7f8340104ac0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.166+0000 7f833d600640 1 -- 172.21.15.3:0/287420801 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328010690 con 0x7f8340104ac0 2024-03-21T08:30:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.166+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83401a1f40 con 0x7f8340104ac0 2024-03-21T08:30:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.167+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834010ac70 con 0x7f8340104ac0 2024-03-21T08:30:42.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.169+0000 7f833d600640 1 -- 172.21.15.3:0/287420801 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f83280037a0 con 0x7f8340104ac0 2024-03-21T08:30:42.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.171+0000 7f833d600640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f83240a2f20 0x7f83240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.171+0000 7f833d600640 1 -- 172.21.15.3:0/287420801 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f83280cfc20 con 0x7f8340104ac0 2024-03-21T08:30:42.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.172+0000 7f833fe00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f83240a2f20 0x7f83240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:42.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.172+0000 7f833fe00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f83240a2f20 0x7f83240a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8330006010 tx=0x7f8330005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:42.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.176+0000 7f833d600640 1 -- 172.21.15.3:0/287420801 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f832809c610 con 0x7f8340104ac0 2024-03-21T08:30:42.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:41 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1253120050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:30:42.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:41 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/818549046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:30:42.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:41 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3201536700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:42.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:41 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1253120050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:30:42.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:41 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/818549046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:30:42.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:41 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3201536700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:42.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.381+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83400622a0 con 0x7f8340104ac0 2024-03-21T08:30:42.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.384+0000 7f833d600640 1 -- 172.21.15.3:0/287420801 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7f832809bf10 con 0x7f8340104ac0 2024-03-21T08:30:42.387 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:42.388 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:30:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f83240a2f20 msgr2=0x7f83240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f83240a2f20 0x7f83240a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8330006010 tx=0x7f8330005be0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 msgr2=0x7f834019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 0x7f834019a020 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f832800ea60 tx=0x7f832800ef20 comp rx=0 tx=0).stop 2024-03-21T08:30:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 shutdown_connections 2024-03-21T08:30:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8340105480 0x7f83401a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8340104ac0 0x7f834019a020 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f83240a2f20 0x7f83240a53e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 --2- 172.21.15.3:0/287420801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83401038d0 0x7f8340199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 >> 172.21.15.3:0/287420801 conn(0x7f83400fee60 msgr2=0x7f8340107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 shutdown_connections 2024-03-21T08:30:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.387+0000 7f8346c00640 1 -- 172.21.15.3:0/287420801 wait complete. 2024-03-21T08:30:42.405 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:41 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1253120050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:30:42.405 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:41 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/818549046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:30:42.405 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:41 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3201536700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:42.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:30:42.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:30:42.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 -- 172.21.15.3:0/3705216132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf410b780 msgr2=0x7fecf410db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 --2- 172.21.15.3:0/3705216132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf410b780 0x7fecf410db70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009900 tx=0x7fecdc02bc40 comp rx=0 tx=0).stop 2024-03-21T08:30:42.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 -- 172.21.15.3:0/3705216132 shutdown_connections 2024-03-21T08:30:42.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 --2- 172.21.15.3:0/3705216132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fecf410e1e0 0x7fecf4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 --2- 172.21.15.3:0/3705216132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf410b780 0x7fecf410db70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 --2- 172.21.15.3:0/3705216132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf406a3e0 0x7fecf410b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 -- 172.21.15.3:0/3705216132 >> 172.21.15.3:0/3705216132 conn(0x7fecf406e880 msgr2=0x7fecf4070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:42.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 -- 172.21.15.3:0/3705216132 shutdown_connections 2024-03-21T08:30:42.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.634+0000 7fecf9e00640 1 -- 172.21.15.3:0/3705216132 wait complete. 2024-03-21T08:30:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 Processor -- start 2024-03-21T08:30:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 -- start start 2024-03-21T08:30:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf406a3e0 0x7fecf41a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fecf410b780 0x7fecf41a2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf410e1e0 0x7fecf41a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4115a50 con 0x7fecf406a3e0 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf41158d0 con 0x7fecf410e1e0 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.635+0000 7fecf9e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4115bd0 con 0x7fecf410b780 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf406a3e0 0x7fecf41a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf406a3e0 0x7fecf41a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34286/0 (socket says 172.21.15.3:34286) 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 -- 172.21.15.3:0/433581269 learned_addr learned my addr 172.21.15.3:0/433581269 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf3400640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf410e1e0 0x7fecf41a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 -- 172.21.15.3:0/433581269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fecf410b780 msgr2=0x7fecf41a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fecf410b780 0x7fecf41a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 -- 172.21.15.3:0/433581269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf410e1e0 msgr2=0x7fecf41a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf410e1e0 0x7fecf41a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 -- 172.21.15.3:0/433581269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf41a9f50 con 0x7fecf406a3e0 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.636+0000 7fecf2a00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf406a3e0 0x7fecf41a2330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fece400d770 tx=0x7fece400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.637+0000 7fecf8a00640 1 -- 172.21.15.3:0/433581269 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4017070 con 0x7fecf406a3e0 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.637+0000 7fecf8a00640 1 -- 172.21.15.3:0/433581269 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fece4009c30 con 0x7fecf406a3e0 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.637+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf41aa240 con 0x7fecf406a3e0 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.637+0000 7fecf8a00640 1 -- 172.21.15.3:0/433581269 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4003d00 con 0x7fecf406a3e0 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.637+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf41aa7e0 con 0x7fecf406a3e0 2024-03-21T08:30:42.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.638+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf410fd70 con 0x7fecf406a3e0 2024-03-21T08:30:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.640+0000 7fecf8a00640 1 -- 172.21.15.3:0/433581269 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fece4003ea0 con 0x7fecf406a3e0 2024-03-21T08:30:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.641+0000 7fecf8a00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecd80a2fa0 0x7fecd80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.641+0000 7fecf8a00640 1 -- 172.21.15.3:0/433581269 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fece40cf580 con 0x7fecf406a3e0 2024-03-21T08:30:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.642+0000 7fecea000640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecd80a2fa0 0x7fecd80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.642+0000 7fecea000640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecd80a2fa0 0x7fecd80a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fecdc036040 tx=0x7fecdc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:42.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.646+0000 7fecf8a00640 1 -- 172.21.15.3:0/433581269 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fece409bf70 con 0x7fecf406a3e0 2024-03-21T08:30:42.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.849+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecf40622d0 con 0x7fecf406a3e0 2024-03-21T08:30:42.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.849+0000 7fecf8a00640 1 -- 172.21.15.3:0/433581269 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fece4009de0 con 0x7fecf406a3e0 2024-03-21T08:30:42.851 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecd80a2fa0 msgr2=0x7fecd80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecd80a2fa0 0x7fecd80a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fecdc036040 tx=0x7fecdc000bc0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf406a3e0 msgr2=0x7fecf41a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf406a3e0 0x7fecf41a2330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fece400d770 tx=0x7fece400dc30 comp rx=0 tx=0).stop 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 shutdown_connections 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecf410e1e0 0x7fecf41a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fecd80a2fa0 0x7fecd80a5460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fecf410b780 0x7fecf41a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 --2- 172.21.15.3:0/433581269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf406a3e0 0x7fecf41a2330 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:42.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 >> 172.21.15.3:0/433581269 conn(0x7fecf406e880 msgr2=0x7fecf4113990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.852+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 shutdown_connections 2024-03-21T08:30:42.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:42.853+0000 7fecf9e00640 1 -- 172.21.15.3:0/433581269 wait complete. 2024-03-21T08:30:42.876 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 -- 172.21.15.3:0/1504429074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481052b0 msgr2=0x7fee4810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1504429074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481052b0 0x7fee4810bb40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fee40009900 tx=0x7fee4002bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 -- 172.21.15.3:0/1504429074 shutdown_connections 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1504429074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481052b0 0x7fee4810bb40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1504429074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee481048f0 0x7fee48104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1504429074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee48103700 0x7fee48103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 -- 172.21.15.3:0/1504429074 >> 172.21.15.3:0/1504429074 conn(0x7fee480fec70 msgr2=0x7fee481010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 -- 172.21.15.3:0/1504429074 shutdown_connections 2024-03-21T08:30:43.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.109+0000 7fee4ec00640 1 -- 172.21.15.3:0/1504429074 wait complete. 2024-03-21T08:30:43.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 Processor -- start 2024-03-21T08:30:43.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 -- start start 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee48103700 0x7fee48199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481048f0 0x7fee48199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee481052b0 0x7fee481a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4810f2e0 con 0x7fee481048f0 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4810f160 con 0x7fee48103700 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.110+0000 7fee4ec00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4810f460 con 0x7fee481052b0 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481048f0 0x7fee48199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481048f0 0x7fee48199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34294/0 (socket says 172.21.15.3:34294) 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 -- 172.21.15.3:0/1434808650 learned_addr learned my addr 172.21.15.3:0/1434808650 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4d800640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee48103700 0x7fee48199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4e200640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee481052b0 0x7fee481a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 -- 172.21.15.3:0/1434808650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee481052b0 msgr2=0x7fee481a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee481052b0 0x7fee481a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 -- 172.21.15.3:0/1434808650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee48103700 msgr2=0x7fee48199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee48103700 0x7fee48199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 -- 172.21.15.3:0/1434808650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee481a1580 con 0x7fee481048f0 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.111+0000 7fee4ce00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481048f0 0x7fee48199ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fee3400b4a0 tx=0x7fee3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.112+0000 7fee46000640 1 -- 172.21.15.3:0/1434808650 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34014070 con 0x7fee481048f0 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.112+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee481a1870 con 0x7fee481048f0 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.112+0000 7fee46000640 1 -- 172.21.15.3:0/1434808650 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fee34010ac0 con 0x7fee481048f0 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.112+0000 7fee46000640 1 -- 172.21.15.3:0/1434808650 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3401e060 con 0x7fee481048f0 2024-03-21T08:30:43.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.112+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee481a1e40 con 0x7fee481048f0 2024-03-21T08:30:43.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.113+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4810aaf0 con 0x7fee481048f0 2024-03-21T08:30:43.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.116+0000 7fee46000640 1 -- 172.21.15.3:0/1434808650 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fee34017070 con 0x7fee481048f0 2024-03-21T08:30:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.117+0000 7fee46000640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fee300a2f20 0x7fee300a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.117+0000 7fee46000640 1 -- 172.21.15.3:0/1434808650 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fee340cefe0 con 0x7fee481048f0 2024-03-21T08:30:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.118+0000 7fee4d800640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fee300a2f20 0x7fee300a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.118+0000 7fee4d800640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fee300a2f20 0x7fee300a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee48104750 tx=0x7fee3c009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:43.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.121+0000 7fee46000640 1 -- 172.21.15.3:0/1434808650 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee3409b9d0 con 0x7fee481048f0 2024-03-21T08:30:43.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:42 smithi003 ceph-mon[31675]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-21T08:30:43.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:42 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/287420801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:43.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:42 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/433581269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:30:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.324+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee48062340 con 0x7fee481048f0 2024-03-21T08:30:43.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.324+0000 7fee46000640 1 -- 172.21.15.3:0/1434808650 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee3409b2d0 con 0x7fee481048f0 2024-03-21T08:30:43.326 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:30:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fee300a2f20 msgr2=0x7fee300a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fee300a2f20 0x7fee300a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee48104750 tx=0x7fee3c009870 comp rx=0 tx=0).stop 2024-03-21T08:30:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481048f0 msgr2=0x7fee48199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481048f0 0x7fee48199ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fee3400b4a0 tx=0x7fee3400b960 comp rx=0 tx=0).stop 2024-03-21T08:30:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 shutdown_connections 2024-03-21T08:30:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fee300a2f20 0x7fee300a53e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee481052b0 0x7fee481a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee481048f0 0x7fee48199ea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 --2- 172.21.15.3:0/1434808650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee48103700 0x7fee48199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 >> 172.21.15.3:0/1434808650 conn(0x7fee480fec70 msgr2=0x7fee48107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.327+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 shutdown_connections 2024-03-21T08:30:43.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.328+0000 7fee4ec00640 1 -- 172.21.15.3:0/1434808650 wait complete. 2024-03-21T08:30:43.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:30:43.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:42 smithi084 ceph-mon[35477]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-21T08:30:43.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:42 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/287420801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:43.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:42 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/433581269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:30:43.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:42 smithi171 ceph-mon[35388]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-21T08:30:43.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:42 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/287420801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:43.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:42 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/433581269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:30:43.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 -- 172.21.15.3:0/2844155776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a90109370 msgr2=0x7f5a9010b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 --2- 172.21.15.3:0/2844155776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a90109370 0x7f5a9010b810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009ca0 tx=0x7f5a8802c180 comp rx=0 tx=0).stop 2024-03-21T08:30:43.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 -- 172.21.15.3:0/2844155776 shutdown_connections 2024-03-21T08:30:43.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 --2- 172.21.15.3:0/2844155776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a90109370 0x7f5a9010b810 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 --2- 172.21.15.3:0/2844155776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a900740a0 0x7f5a90074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 --2- 172.21.15.3:0/2844155776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a90075680 0x7f5a90073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 -- 172.21.15.3:0/2844155776 >> 172.21.15.3:0/2844155776 conn(0x7f5a900fca50 msgr2=0x7f5a900fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:43.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 -- 172.21.15.3:0/2844155776 shutdown_connections 2024-03-21T08:30:43.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.575+0000 7f5a97000640 1 -- 172.21.15.3:0/2844155776 wait complete. 2024-03-21T08:30:43.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.576+0000 7f5a97000640 1 Processor -- start 2024-03-21T08:30:43.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.576+0000 7f5a97000640 1 -- start start 2024-03-21T08:30:43.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a97000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a900740a0 0x7f5a901998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a97000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a90075680 0x7f5a90199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a97000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a90109370 0x7f5a901a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a97000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9010f1f0 con 0x7f5a900740a0 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a97000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9010f070 con 0x7f5a90075680 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a97000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9010f370 con 0x7f5a90109370 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a95c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a900740a0 0x7f5a901998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a95c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a900740a0 0x7f5a901998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34310/0 (socket says 172.21.15.3:34310) 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a95c00640 1 -- 172.21.15.3:0/2594391107 learned_addr learned my addr 172.21.15.3:0/2594391107 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a95200640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a90075680 0x7f5a90199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.577+0000 7f5a96600640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a90109370 0x7f5a901a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a95c00640 1 -- 172.21.15.3:0/2594391107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a90109370 msgr2=0x7f5a901a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a95c00640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a90109370 0x7f5a901a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a95c00640 1 -- 172.21.15.3:0/2594391107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a90075680 msgr2=0x7f5a90199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a95c00640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a90075680 0x7f5a90199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a95c00640 1 -- 172.21.15.3:0/2594391107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a901a14c0 con 0x7f5a900740a0 2024-03-21T08:30:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a95c00640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a900740a0 0x7f5a901998a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000d950 tx=0x7f5a8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a8e000640 1 -- 172.21.15.3:0/2594391107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80017070 con 0x7f5a900740a0 2024-03-21T08:30:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a8e000640 1 -- 172.21.15.3:0/2594391107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a80009c30 con 0x7f5a900740a0 2024-03-21T08:30:43.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a901a17b0 con 0x7f5a900740a0 2024-03-21T08:30:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.578+0000 7f5a8e000640 1 -- 172.21.15.3:0/2594391107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000f620 con 0x7f5a900740a0 2024-03-21T08:30:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.579+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a901a1d00 con 0x7f5a900740a0 2024-03-21T08:30:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.580+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a90108130 con 0x7f5a900740a0 2024-03-21T08:30:43.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.582+0000 7f5a8e000640 1 -- 172.21.15.3:0/2594391107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f5a800109a0 con 0x7f5a900740a0 2024-03-21T08:30:43.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.583+0000 7f5a8e000640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5a740a2f20 0x7f5a740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.583+0000 7f5a8e000640 1 -- 172.21.15.3:0/2594391107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f5a800cf4d0 con 0x7f5a900740a0 2024-03-21T08:30:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.583+0000 7f5a95200640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5a740a2f20 0x7f5a740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.584+0000 7f5a95200640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5a740a2f20 0x7f5a740a53e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a9019adb0 tx=0x7f5a7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:43.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.588+0000 7f5a8e000640 1 -- 172.21.15.3:0/2594391107 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a80009de0 con 0x7f5a900740a0 2024-03-21T08:30:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.795+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a900622d0 con 0x7f5a900740a0 2024-03-21T08:30:43.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.796+0000 7f5a8e000640 1 -- 172.21.15.3:0/2594391107 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a8009bec0 con 0x7f5a900740a0 2024-03-21T08:30:43.798 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:30:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5a740a2f20 msgr2=0x7f5a740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5a740a2f20 0x7f5a740a53e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a9019adb0 tx=0x7f5a7c005f90 comp rx=0 tx=0).stop 2024-03-21T08:30:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a900740a0 msgr2=0x7f5a901998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a900740a0 0x7f5a901998a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000d950 tx=0x7f5a8000de10 comp rx=0 tx=0).stop 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 shutdown_connections 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f5a740a2f20 0x7f5a740a53e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a90109370 0x7f5a901a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a90075680 0x7f5a90199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 --2- 172.21.15.3:0/2594391107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a900740a0 0x7f5a901998a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.798+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 >> 172.21.15.3:0/2594391107 conn(0x7f5a900fca50 msgr2=0x7f5a90105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.799+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 shutdown_connections 2024-03-21T08:30:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:43.799+0000 7f5a97000640 1 -- 172.21.15.3:0/2594391107 wait complete. 2024-03-21T08:30:43.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.052+0000 7f649e600640 1 -- 172.21.15.3:0/1420651054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6498074100 msgr2=0x7f6498074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.052+0000 7f649e600640 1 --2- 172.21.15.3:0/1420651054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6498074100 0x7f6498074580 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6484009930 tx=0x7f648402bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.053+0000 7f649e600640 1 -- 172.21.15.3:0/1420651054 shutdown_connections 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.053+0000 7f649e600640 1 --2- 172.21.15.3:0/1420651054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f649810dab0 0x7f649810fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.053+0000 7f649e600640 1 --2- 172.21.15.3:0/1420651054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6498074100 0x7f6498074580 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.053+0000 7f649e600640 1 --2- 172.21.15.3:0/1420651054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64980756e0 0x7f6498073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.053+0000 7f649e600640 1 -- 172.21.15.3:0/1420651054 >> 172.21.15.3:0/1420651054 conn(0x7f64980fca50 msgr2=0x7f64980fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.053+0000 7f649e600640 1 -- 172.21.15.3:0/1420651054 shutdown_connections 2024-03-21T08:30:44.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.053+0000 7f649e600640 1 -- 172.21.15.3:0/1420651054 wait complete. 2024-03-21T08:30:44.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.054+0000 7f649e600640 1 Processor -- start 2024-03-21T08:30:44.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.054+0000 7f649e600640 1 -- start start 2024-03-21T08:30:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649e600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6498074100 0x7f649819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649e600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64980756e0 0x7f649819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649810dab0 0x7f64981a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649e600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6498113600 con 0x7f649810dab0 2024-03-21T08:30:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649e600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6498113480 con 0x7f6498074100 2024-03-21T08:30:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649e600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6498113780 con 0x7f64980756e0 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649810dab0 0x7f64981a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649810dab0 0x7f64981a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34320/0 (socket says 172.21.15.3:34320) 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649dc00640 1 -- 172.21.15.3:0/1253992262 learned_addr learned my addr 172.21.15.3:0/1253992262 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f649d200640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6498074100 0x7f649819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.055+0000 7f6497e00640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64980756e0 0x7f649819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f649dc00640 1 -- 172.21.15.3:0/1253992262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64980756e0 msgr2=0x7f649819e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f649dc00640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64980756e0 0x7f649819e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f649dc00640 1 -- 172.21.15.3:0/1253992262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6498074100 msgr2=0x7f649819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f649dc00640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6498074100 0x7f649819de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f649dc00640 1 -- 172.21.15.3:0/1253992262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64981a59d0 con 0x7f649810dab0 2024-03-21T08:30:44.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f649dc00640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649810dab0 0x7f64981a5330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f649000eb10 tx=0x7f649000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:44.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f6495600640 1 -- 172.21.15.3:0/1253992262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490004b20 con 0x7f649810dab0 2024-03-21T08:30:44.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f6495600640 1 -- 172.21.15.3:0/1253992262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6490004cc0 con 0x7f649810dab0 2024-03-21T08:30:44.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64981a5cc0 con 0x7f649810dab0 2024-03-21T08:30:44.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.056+0000 7f6495600640 1 -- 172.21.15.3:0/1253992262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490010690 con 0x7f649810dab0 2024-03-21T08:30:44.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.057+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64981a61c0 con 0x7f649810dab0 2024-03-21T08:30:44.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.058+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6498103b70 con 0x7f649810dab0 2024-03-21T08:30:44.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.060+0000 7f6495600640 1 -- 172.21.15.3:0/1253992262 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f6490005020 con 0x7f649810dab0 2024-03-21T08:30:44.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.062+0000 7f6495600640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f64800a2fa0 0x7f64800a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.062+0000 7f6495600640 1 -- 172.21.15.3:0/1253992262 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f64900cfea0 con 0x7f649810dab0 2024-03-21T08:30:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.062+0000 7f649d200640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f64800a2fa0 0x7f64800a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.063+0000 7f649d200640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f64800a2fa0 0x7f64800a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f649819f290 tx=0x7f648c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.066+0000 7f6495600640 1 -- 172.21.15.3:0/1253992262 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f649009c830 con 0x7f649810dab0 2024-03-21T08:30:44.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:43 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1434808650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:30:44.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:43 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2594391107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:30:44.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.269+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6498062340 con 0x7f649810dab0 2024-03-21T08:30:44.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.270+0000 7f6495600640 1 -- 172.21.15.3:0/1253992262 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f649009c130 con 0x7f649810dab0 2024-03-21T08:30:44.273 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:30:44.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f64800a2fa0 msgr2=0x7f64800a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f64800a2fa0 0x7f64800a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f649819f290 tx=0x7f648c00b390 comp rx=0 tx=0).stop 2024-03-21T08:30:44.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649810dab0 msgr2=0x7f64981a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649810dab0 0x7f64981a5330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f649000eb10 tx=0x7f649000c2d0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 shutdown_connections 2024-03-21T08:30:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649810dab0 0x7f64981a5330 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f64800a2fa0 0x7f64800a5460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64980756e0 0x7f649819e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 --2- 172.21.15.3:0/1253992262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6498074100 0x7f649819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 >> 172.21.15.3:0/1253992262 conn(0x7f64980fca50 msgr2=0x7f6498100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.273+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 shutdown_connections 2024-03-21T08:30:44.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.274+0000 7f649e600640 1 -- 172.21.15.3:0/1253992262 wait complete. 2024-03-21T08:30:44.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:30:44.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:43 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1434808650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:30:44.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:43 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2594391107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:30:44.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:43 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1434808650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:30:44.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:43 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2594391107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:30:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.524+0000 7f374e000640 1 -- 172.21.15.3:0/2542497264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748077c40 msgr2=0x7f37480784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.524+0000 7f374e000640 1 --2- 172.21.15.3:0/2542497264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748077c40 0x7f37480784e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3740009ca0 tx=0x7f374002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.525+0000 7f374e000640 1 -- 172.21.15.3:0/2542497264 shutdown_connections 2024-03-21T08:30:44.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.525+0000 7f374e000640 1 --2- 172.21.15.3:0/2542497264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748077c40 0x7f37480784e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.525+0000 7f374e000640 1 --2- 172.21.15.3:0/2542497264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37480772a0 0x7f3748077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.525+0000 7f374e000640 1 --2- 172.21.15.3:0/2542497264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3748076060 0x7f3748076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.525+0000 7f374e000640 1 -- 172.21.15.3:0/2542497264 >> 172.21.15.3:0/2542497264 conn(0x7f37480fee60 msgr2=0x7f3748101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.525+0000 7f374e000640 1 -- 172.21.15.3:0/2542497264 shutdown_connections 2024-03-21T08:30:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.525+0000 7f374e000640 1 -- 172.21.15.3:0/2542497264 wait complete. 2024-03-21T08:30:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 Processor -- start 2024-03-21T08:30:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 -- start start 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3748076060 0x7f374819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37480772a0 0x7f374819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3748077c40 0x7f37481a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748113810 con 0x7f37480772a0 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748113690 con 0x7f3748077c40 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.526+0000 7f374e000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748113990 con 0x7f3748076060 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37480772a0 0x7f374819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37480772a0 0x7f374819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34328/0 (socket says 172.21.15.3:34328) 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 -- 172.21.15.3:0/3173487186 learned_addr learned my addr 172.21.15.3:0/3173487186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f3747400640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3748076060 0x7f374819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f3747e00640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3748077c40 0x7f37481a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 -- 172.21.15.3:0/3173487186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3748076060 msgr2=0x7f374819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3748076060 0x7f374819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 -- 172.21.15.3:0/3173487186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3748077c40 msgr2=0x7f37481a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3748077c40 0x7f37481a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 -- 172.21.15.3:0/3173487186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37481a5ac0 con 0x7f37480772a0 2024-03-21T08:30:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.527+0000 7f373fe00640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37480772a0 0x7f374819e3e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f373000e9f0 tx=0x7f373000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.528+0000 7f3744c00640 1 -- 172.21.15.3:0/3173487186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373000cd80 con 0x7f37480772a0 2024-03-21T08:30:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.528+0000 7f3744c00640 1 -- 172.21.15.3:0/3173487186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f37300054d0 con 0x7f37480772a0 2024-03-21T08:30:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.528+0000 7f3744c00640 1 -- 172.21.15.3:0/3173487186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730010490 con 0x7f37480772a0 2024-03-21T08:30:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.528+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37481a5db0 con 0x7f37480772a0 2024-03-21T08:30:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.528+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37481a6220 con 0x7f37480772a0 2024-03-21T08:30:44.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.529+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3748110420 con 0x7f37480772a0 2024-03-21T08:30:44.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.531+0000 7f3744c00640 1 -- 172.21.15.3:0/3173487186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f3730005020 con 0x7f37480772a0 2024-03-21T08:30:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.532+0000 7f3744c00640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37280a2fa0 0x7f37280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.533+0000 7f3744c00640 1 -- 172.21.15.3:0/3173487186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f37300cfcc0 con 0x7f37480772a0 2024-03-21T08:30:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.533+0000 7f3747400640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37280a2fa0 0x7f37280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:44.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.533+0000 7f3747400640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37280a2fa0 0x7f37280a5460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3738009b60 tx=0x7f3738005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:44.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.536+0000 7f3744c00640 1 -- 172.21.15.3:0/3173487186 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f37300d6080 con 0x7f37480772a0 2024-03-21T08:30:44.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.787+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f374802d070 con 0x7f37480772a0 2024-03-21T08:30:44.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.789+0000 7f3744c00640 1 -- 172.21.15.3:0/3173487186 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f373009c650 con 0x7f37480772a0 2024-03-21T08:30:44.792 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:44.792 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:29:09.323523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34409,34472]},"id":1}]} 2024-03-21T08:30:44.792 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-03-21T08:30:44.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37280a2fa0 msgr2=0x7f37280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37280a2fa0 0x7f37280a5460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3738009b60 tx=0x7f3738005f90 comp rx=0 tx=0).stop 2024-03-21T08:30:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37480772a0 msgr2=0x7f374819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37480772a0 0x7f374819e3e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f373000e9f0 tx=0x7f373000edf0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 shutdown_connections 2024-03-21T08:30:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3748077c40 0x7f37481a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37480772a0 0x7f374819e3e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37280a2fa0 0x7f37280a5460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.792+0000 7f374e000640 1 --2- 172.21.15.3:0/3173487186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3748076060 0x7f374819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:44.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.793+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 >> 172.21.15.3:0/3173487186 conn(0x7f37480fee60 msgr2=0x7f3748100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:44.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.793+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 shutdown_connections 2024-03-21T08:30:44.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:44.793+0000 7f374e000640 1 -- 172.21.15.3:0/3173487186 wait complete. 2024-03-21T08:30:44.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:30:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.043+0000 7f9460a00640 1 -- 172.21.15.3:0/2229857913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 msgr2=0x7f945c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.043+0000 7f9460a00640 1 --2- 172.21.15.3:0/2229857913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c103be0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f944c009900 tx=0x7f944c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.043+0000 7f9460a00640 1 -- 172.21.15.3:0/2229857913 shutdown_connections 2024-03-21T08:30:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.043+0000 7f9460a00640 1 --2- 172.21.15.3:0/2229857913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f945c105370 0x7f945c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.043+0000 7f9460a00640 1 --2- 172.21.15.3:0/2229857913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f945c1049b0 0x7f945c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.043+0000 7f9460a00640 1 --2- 172.21.15.3:0/2229857913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c103be0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.044+0000 7f9460a00640 1 -- 172.21.15.3:0/2229857913 >> 172.21.15.3:0/2229857913 conn(0x7f945c0fed50 msgr2=0x7f945c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:45.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.044+0000 7f9460a00640 1 -- 172.21.15.3:0/2229857913 shutdown_connections 2024-03-21T08:30:45.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.044+0000 7f9460a00640 1 -- 172.21.15.3:0/2229857913 wait complete. 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.045+0000 7f9460a00640 1 Processor -- start 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.045+0000 7f9460a00640 1 -- start start 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.045+0000 7f9460a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.045+0000 7f9460a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f945c1049b0 0x7f945c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f9460a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f945c105370 0x7f945c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f9460a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c10f2c0 con 0x7f945c1037c0 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f9460a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c10f140 con 0x7f945c1049b0 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f9460a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c10f440 con 0x7f945c105370 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34342/0 (socket says 172.21.15.3:34342) 2024-03-21T08:30:45.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 -- 172.21.15.3:0/3247579652 learned_addr learned my addr 172.21.15.3:0/3247579652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f9453e00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f945c1049b0 0x7f945c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 -- 172.21.15.3:0/3247579652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f945c105370 msgr2=0x7f945c1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945b200640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f945c105370 0x7f945c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f945c105370 0x7f945c1a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 -- 172.21.15.3:0/3247579652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f945c1049b0 msgr2=0x7f945c199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f945c1049b0 0x7f945c199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945a800640 1 -- 172.21.15.3:0/3247579652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c1a1610 con 0x7f945c1037c0 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f945b200640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f945c105370 0x7f945c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.046+0000 7f9453e00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f945c1049b0 0x7f945c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.047+0000 7f945a800640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c1999f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f944c000c00 tx=0x7f944c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.047+0000 7f9453400640 1 -- 172.21.15.3:0/3247579652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c043020 con 0x7f945c1037c0 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.047+0000 7f9453400640 1 -- 172.21.15.3:0/3247579652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f944c004440 con 0x7f945c1037c0 2024-03-21T08:30:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.047+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c1a18a0 con 0x7f945c1037c0 2024-03-21T08:30:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.047+0000 7f9453400640 1 -- 172.21.15.3:0/3247579652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c003c90 con 0x7f945c1037c0 2024-03-21T08:30:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.047+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c1a1e40 con 0x7f945c1037c0 2024-03-21T08:30:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.048+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c10ab50 con 0x7f945c1037c0 2024-03-21T08:30:45.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.050+0000 7f9453400640 1 -- 172.21.15.3:0/3247579652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f944c046020 con 0x7f945c1037c0 2024-03-21T08:30:45.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.053+0000 7f9453400640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f94400a2f90 0x7f94400a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.053+0000 7f9453400640 1 -- 172.21.15.3:0/3247579652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f944c0f1b40 con 0x7f945c1037c0 2024-03-21T08:30:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.053+0000 7f9453e00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f94400a2f90 0x7f94400a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.054+0000 7f9453e00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f94400a2f90 0x7f94400a5450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9444005d00 tx=0x7f9444005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.057+0000 7f9453400640 1 -- 172.21.15.3:0/3247579652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f944c04d020 con 0x7f945c1037c0 2024-03-21T08:30:45.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:45 smithi003 ceph-mon[31675]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-21T08:30:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:45 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1253992262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:30:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:45 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3173487186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.267+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f945c0622a0 con 0x7f945c1037c0 2024-03-21T08:30:45.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.272+0000 7f9453400640 1 -- 172.21.15.3:0/3247579652 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7f944c0be4d0 con 0x7f945c1037c0 2024-03-21T08:30:45.274 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:45.274 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:30:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f94400a2f90 msgr2=0x7f94400a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f94400a2f90 0x7f94400a5450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9444005d00 tx=0x7f9444005c70 comp rx=0 tx=0).stop 2024-03-21T08:30:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 msgr2=0x7f945c1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c1999f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f944c000c00 tx=0x7f944c004040 comp rx=0 tx=0).stop 2024-03-21T08:30:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 shutdown_connections 2024-03-21T08:30:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f94400a2f90 0x7f94400a5450 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f945c105370 0x7f945c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f945c1049b0 0x7f945c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 --2- 172.21.15.3:0/3247579652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f945c1037c0 0x7f945c1999f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 >> 172.21.15.3:0/3247579652 conn(0x7f945c0fed50 msgr2=0x7f945c107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 shutdown_connections 2024-03-21T08:30:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.275+0000 7f9460a00640 1 -- 172.21.15.3:0/3247579652 wait complete. 2024-03-21T08:30:45.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:30:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:45 smithi084 ceph-mon[35477]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-21T08:30:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:45 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1253992262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:30:45.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:45 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3173487186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:45.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:45 smithi171 ceph-mon[35388]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-21T08:30:45.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:45 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1253992262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:30:45.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:45 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3173487186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 -- 172.21.15.3:0/2631036457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 msgr2=0x7f779c102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 --2- 172.21.15.3:0/2631036457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c102c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7784009900 tx=0x7f778402bc50 comp rx=0 tx=0).stop 2024-03-21T08:30:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 -- 172.21.15.3:0/2631036457 shutdown_connections 2024-03-21T08:30:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 --2- 172.21.15.3:0/2631036457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f779c103170 0x7f779c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 --2- 172.21.15.3:0/2631036457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c102c30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 --2- 172.21.15.3:0/2631036457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f779c1015c0 0x7f779c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 -- 172.21.15.3:0/2631036457 >> 172.21.15.3:0/2631036457 conn(0x7f779c0fcb30 msgr2=0x7f779c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 -- 172.21.15.3:0/2631036457 shutdown_connections 2024-03-21T08:30:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.512+0000 7f779be00640 1 -- 172.21.15.3:0/2631036457 wait complete. 2024-03-21T08:30:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 Processor -- start 2024-03-21T08:30:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 -- start start 2024-03-21T08:30:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f779c1015c0 0x7f779c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f779c103170 0x7f779c1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c10d0f0 con 0x7f779c1027b0 2024-03-21T08:30:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c10cf70 con 0x7f779c1015c0 2024-03-21T08:30:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.513+0000 7f779be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c10d270 con 0x7f779c103170 2024-03-21T08:30:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34356/0 (socket says 172.21.15.3:34356) 2024-03-21T08:30:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 -- 172.21.15.3:0/2667841410 learned_addr learned my addr 172.21.15.3:0/2667841410 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779aa00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f779c1015c0 0x7f779c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779b400640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f779c103170 0x7f779c1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 -- 172.21.15.3:0/2667841410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f779c103170 msgr2=0x7f779c1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f779c103170 0x7f779c1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 -- 172.21.15.3:0/2667841410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f779c1015c0 msgr2=0x7f779c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f779c1015c0 0x7f779c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779a000640 1 -- 172.21.15.3:0/2667841410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c1a1660 con 0x7f779c1027b0 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.514+0000 7f779b400640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f779c103170 0x7f779c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.515+0000 7f779a000640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c199fd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7784006010 tx=0x7f77840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.515+0000 7f77a0a00640 1 -- 172.21.15.3:0/2667841410 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784043020 con 0x7f779c1027b0 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.515+0000 7f77a0a00640 1 -- 172.21.15.3:0/2667841410 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7784004aa0 con 0x7f779c1027b0 2024-03-21T08:30:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.515+0000 7f77a0a00640 1 -- 172.21.15.3:0/2667841410 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784003c60 con 0x7f779c1027b0 2024-03-21T08:30:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.515+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c1a18f0 con 0x7f779c1027b0 2024-03-21T08:30:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.515+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c1a1df0 con 0x7f779c1027b0 2024-03-21T08:30:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.516+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c1a1ff0 con 0x7f779c1027b0 2024-03-21T08:30:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.519+0000 7f77a0a00640 1 -- 172.21.15.3:0/2667841410 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f7784046020 con 0x7f779c1027b0 2024-03-21T08:30:45.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.520+0000 7f77a0a00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f77800a2f90 0x7f77800a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.520+0000 7f77a0a00640 1 -- 172.21.15.3:0/2667841410 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f77840f1cc0 con 0x7f779c1027b0 2024-03-21T08:30:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.520+0000 7f779aa00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f77800a2f90 0x7f77800a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.521+0000 7f779aa00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f77800a2f90 0x7f77800a5450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f779c102610 tx=0x7f7794009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:45.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.524+0000 7f77a0a00640 1 -- 172.21.15.3:0/2667841410 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f77840f8080 con 0x7f779c1027b0 2024-03-21T08:30:45.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.771+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f779c1a21c0 con 0x7f779c1027b0 2024-03-21T08:30:45.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.772+0000 7f77a0a00640 1 -- 172.21.15.3:0/2667841410 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f778404d020 con 0x7f779c1027b0 2024-03-21T08:30:45.775 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:30:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:29:09.323523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34409,34472]},"id":1}]} 2024-03-21T08:30:45.775 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-03-21T08:30:45.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f77800a2f90 msgr2=0x7f77800a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f77800a2f90 0x7f77800a5450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f779c102610 tx=0x7f7794009870 comp rx=0 tx=0).stop 2024-03-21T08:30:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 msgr2=0x7f779c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c199fd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7784006010 tx=0x7f77840042f0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 shutdown_connections 2024-03-21T08:30:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f77800a2f90 0x7f77800a5450 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f779c103170 0x7f779c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f779c1027b0 0x7f779c199fd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 --2- 172.21.15.3:0/2667841410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f779c1015c0 0x7f779c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:45.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 >> 172.21.15.3:0/2667841410 conn(0x7f779c0fcb30 msgr2=0x7f779c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:45.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 shutdown_connections 2024-03-21T08:30:45.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:45.775+0000 7f779be00640 1 -- 172.21.15.3:0/2667841410 wait complete. 2024-03-21T08:30:45.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T08:28:36.113879+0000', 'modified': '2024-03-21T08:29:09.323523+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34472, 'mds_1': 34439, 'mds_2': 34409}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6835/1859190507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 1859190507}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 1859190507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/2662792128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 2662792128}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 2662792128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6839/1010033316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1010033316}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1010033316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34409': {'gid': 34409, 'name': 'k', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.84:6833/4091779255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 4091779255}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 4091779255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34439': {'gid': 34439, 'name': 'e', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.84:6835/2576104068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 2576104068}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 2576104068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.84:6837/3332280865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 3332280865}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 3332280865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34472, 'qdb_cluster': [34439, 34409, 34472]} 2024-03-21T08:30:45.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-21T08:30:45.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-21T08:30:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 -- 172.21.15.3:0/1659253795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341026e0 msgr2=0x7f0534102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 --2- 172.21.15.3:0/1659253795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341026e0 0x7f0534102b60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0524009900 tx=0x7f052402bc30 comp rx=0 tx=0).stop 2024-03-21T08:30:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 -- 172.21.15.3:0/1659253795 shutdown_connections 2024-03-21T08:30:46.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 --2- 172.21.15.3:0/1659253795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05341030a0 0x7f0534109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 --2- 172.21.15.3:0/1659253795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341026e0 0x7f0534102b60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 --2- 172.21.15.3:0/1659253795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05341014f0 0x7f0534101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 -- 172.21.15.3:0/1659253795 >> 172.21.15.3:0/1659253795 conn(0x7f05340fca60 msgr2=0x7f05340feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 -- 172.21.15.3:0/1659253795 shutdown_connections 2024-03-21T08:30:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.015+0000 7f0533200640 1 -- 172.21.15.3:0/1659253795 wait complete. 2024-03-21T08:30:46.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.016+0000 7f0533200640 1 Processor -- start 2024-03-21T08:30:46.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.016+0000 7f0533200640 1 -- start start 2024-03-21T08:30:46.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.016+0000 7f0533200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341014f0 0x7f0534199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.016+0000 7f0533200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05341026e0 0x7f0534199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0533200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05341030a0 0x7f05341a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0533200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053410d080 con 0x7f05341014f0 2024-03-21T08:30:46.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0533200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053410cf00 con 0x7f05341030a0 2024-03-21T08:30:46.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0533200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053410d200 con 0x7f05341026e0 2024-03-21T08:30:46.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341014f0 0x7f0534199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341014f0 0x7f0534199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34388/0 (socket says 172.21.15.3:34388) 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 -- 172.21.15.3:0/3952818784 learned_addr learned my addr 172.21.15.3:0/3952818784 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531400640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05341026e0 0x7f0534199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0532800640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05341030a0 0x7f05341a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 -- 172.21.15.3:0/3952818784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05341026e0 msgr2=0x7f0534199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05341026e0 0x7f0534199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 -- 172.21.15.3:0/3952818784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05341030a0 msgr2=0x7f05341a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05341030a0 0x7f05341a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.017+0000 7f0531e00640 1 -- 172.21.15.3:0/3952818784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05341a1560 con 0x7f05341014f0 2024-03-21T08:30:46.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.018+0000 7f0531e00640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341014f0 0x7f0534199940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e9c0 tx=0x7f051c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.021+0000 7f052aa00640 1 -- 172.21.15.3:0/3952818784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00cd80 con 0x7f05341014f0 2024-03-21T08:30:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.021+0000 7f052aa00640 1 -- 172.21.15.3:0/3952818784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f051c0035f0 con 0x7f05341014f0 2024-03-21T08:30:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.021+0000 7f052aa00640 1 -- 172.21.15.3:0/3952818784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c010690 con 0x7f05341014f0 2024-03-21T08:30:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.021+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05341a1850 con 0x7f05341014f0 2024-03-21T08:30:46.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.021+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341a1dc0 con 0x7f05341014f0 2024-03-21T08:30:46.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.024+0000 7f052aa00640 1 -- 172.21.15.3:0/3952818784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f051c0037a0 con 0x7f05341014f0 2024-03-21T08:30:46.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.024+0000 7f053bdfd740 1 Processor -- start 2024-03-21T08:30:46.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.024+0000 7f053bdfd740 1 -- start start 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca7601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca760770 0x557fca765270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca767d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f053bdfd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca6c4c90 con 0x557fca7658e0 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f053bdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca66d980 con 0x557fca760770 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f053bdfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca6c0d30 con 0x557fca75fd80 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f0531400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca767d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f0531400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca767d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34404/0 (socket says 172.21.15.3:34404) 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f0531400640 1 -- 172.21.15.3:0/1322603089 learned_addr learned my addr 172.21.15.3:0/1322603089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f0532800640 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca7601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.025+0000 7f0531e00640 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca760770 0x557fca765270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0531400640 1 -- 172.21.15.3:0/1322603089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 msgr2=0x557fca7601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0531400640 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca7601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0531400640 1 -- 172.21.15.3:0/1322603089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca760770 msgr2=0x557fca765270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0531400640 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca760770 0x557fca765270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0531400640 1 -- 172.21.15.3:0/1322603089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fca685510 con 0x557fca7658e0 2024-03-21T08:30:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f052aa00640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05180a2fa0 0x7f05180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f052aa00640 1 -- 172.21.15.3:0/3952818784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f051c0cfcb0 con 0x7f05341014f0 2024-03-21T08:30:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0531400640 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca767d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f052402c840 tx=0x7f0524009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef79ed671dbe1c8d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0532800640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05180a2fa0 0x7f05180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.026+0000 7f0523400640 1 -- 172.21.15.3:0/1322603089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524044020 con 0x557fca7658e0 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.027+0000 7f0523400640 1 -- 172.21.15.3:0/1322603089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f052402cae0 con 0x557fca7658e0 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.027+0000 7f0523400640 1 -- 172.21.15.3:0/1322603089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524004920 con 0x557fca7658e0 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.027+0000 7f053bdfd740 1 -- 172.21.15.3:0/1322603089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 msgr2=0x557fca767d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.027+0000 7f0532800640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05180a2fa0 0x7f05180a5460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557fca760fd0 tx=0x7f052c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.027+0000 7f053bdfd740 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca767d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f052402c840 tx=0x7f0524009670 comp rx=0 tx=0).stop 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 -- 172.21.15.3:0/1322603089 shutdown_connections 2024-03-21T08:30:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca767d70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca760770 0x557fca765270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 --2- 172.21.15.3:0/1322603089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca7601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 -- 172.21.15.3:0/1322603089 >> 172.21.15.3:0/1322603089 conn(0x557fca75b2f0 msgr2=0x557fca75b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 -- 172.21.15.3:0/1322603089 shutdown_connections 2024-03-21T08:30:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 -- 172.21.15.3:0/1322603089 wait complete. 2024-03-21T08:30:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 -- start start 2024-03-21T08:30:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.028+0000 7f053bdfd740 1 Processor -- start 2024-03-21T08:30:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca76a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca7658e0 0x557fca771780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f053bdfd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca768830 con 0x557fca760770 2024-03-21T08:30:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f053bdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca680dd0 con 0x557fca75fd80 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f053bdfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca7689b0 con 0x557fca7658e0 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f053bdfd740 10 client.0 _ll_register_callbacks cb 0x557fca3d8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f0531400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f0531400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34412/0 (socket says 172.21.15.3:34412) 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f0531400640 1 -- 172.21.15.3:0/4151462222 learned_addr learned my addr 172.21.15.3:0/4151462222 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f0531e00640 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca76a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.029+0000 7f0532800640 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca7658e0 0x557fca771780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0531400640 1 -- 172.21.15.3:0/4151462222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca7658e0 msgr2=0x557fca771780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0531400640 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca7658e0 0x557fca771780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0531400640 1 -- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 msgr2=0x557fca76a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0531400640 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca76a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0531400640 1 -- 172.21.15.3:0/4151462222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fca3d92d0 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0531400640 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76a740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f052402c240 tx=0x7f052402c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a60bc9a40bc6d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0517e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0517e00640 1 -- 172.21.15.3:0/4151462222 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524044030 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f0517e00640 1 -- 172.21.15.3:0/4151462222 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0524003800 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.030+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f052402cae0 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f0517e00640 1 -- 172.21.15.3:0/4151462222 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f052405a050 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 10 client.15243 fetch_fsmap learned FSMap version 22 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557fca4a36b0 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f0517e00640 1 -- 172.21.15.3:0/4151462222 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f052402cae0 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f0517e00640 1 -- 172.21.15.3:0/4151462222 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0524040bc0 con 0x557fca760770 2024-03-21T08:30:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 10 client.15243 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 10 client.15243 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 20 client.15243 populate_metadata read hostname 'smithi003' 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca776460 0x557fca77a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 4 client.15243 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.031+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fca666520 con 0x557fca776460 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.032+0000 7f0531e00640 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca776460 0x557fca77a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.033+0000 7f0531e00640 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca776460 0x557fca77a8e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51074a10a70333fb server_cookie=40c59570d4970d0e in_seq=0 out_seq=0 2024-03-21T08:30:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.033+0000 7f0517e00640 10 client.15243 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:30:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.034+0000 7f0517e00640 1 -- 172.21.15.3:0/4151462222 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557fca666520 con 0x557fca776460 2024-03-21T08:30:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.034+0000 7f0517e00640 10 client.15243 handle_command_reply: tid=0 2024-03-21T08:30:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 client.15243 shutdown 2024-03-21T08:30:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 msgr2=0x557fca76a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76a740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f052402c240 tx=0x7f052402c350 comp rx=0 tx=0).stop 2024-03-21T08:30:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 shutdown_connections 2024-03-21T08:30:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca776460 0x557fca77a8e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca7658e0 0x557fca771780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f0517e00640 0 client.15243 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76a740 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 --2- 172.21.15.3:0/4151462222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca76a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 >> 172.21.15.3:0/4151462222 conn(0x557fca75b2f0 msgr2=0x557fca75a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 shutdown_connections 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 1 -- 172.21.15.3:0/4151462222 wait complete. 2024-03-21T08:30:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.035+0000 7f053bdfd740 20 client.15243 trim_cache size 0 max 16384 2024-03-21T08:30:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.053+0000 7f053bdfd740 1 Processor -- start 2024-03-21T08:30:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.053+0000 7f053bdfd740 1 -- start start 2024-03-21T08:30:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca767cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca760770 0x557fca771740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca76a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f053bdfd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca775c80 con 0x557fca7658e0 2024-03-21T08:30:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f053bdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca7762e0 con 0x557fca75fd80 2024-03-21T08:30:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f053bdfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca776940 con 0x557fca760770 2024-03-21T08:30:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f0532800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca76a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f0531e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca767cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f0532800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca76a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34422/0 (socket says 172.21.15.3:34422) 2024-03-21T08:30:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f0532800640 1 -- 172.21.15.3:0/3235302026 learned_addr learned my addr 172.21.15.3:0/3235302026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f0531400640 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca760770 0x557fca771740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.054+0000 7f0532800640 1 -- 172.21.15.3:0/3235302026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca760770 msgr2=0x557fca771740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0532800640 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca760770 0x557fca771740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0532800640 1 -- 172.21.15.3:0/3235302026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 msgr2=0x557fca767cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0532800640 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca767cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0532800640 1 -- 172.21.15.3:0/3235302026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fca74d170 con 0x557fca7658e0 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0531400640 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca760770 0x557fca771740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0532800640 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca76a6a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f052c0049c0 tx=0x7f052c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8b5ff9306835b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0522a00640 1 -- 172.21.15.3:0/3235302026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c012de0 con 0x557fca7658e0 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0522a00640 1 -- 172.21.15.3:0/3235302026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f052c01b900 con 0x557fca7658e0 2024-03-21T08:30:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.055+0000 7f0522a00640 1 -- 172.21.15.3:0/3235302026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c010980 con 0x557fca7658e0 2024-03-21T08:30:46.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 -- 172.21.15.3:0/3235302026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 msgr2=0x557fca76a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca76a6a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f052c0049c0 tx=0x7f052c015000 comp rx=0 tx=0).stop 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 -- 172.21.15.3:0/3235302026 shutdown_connections 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca7658e0 0x557fca76a6a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca760770 0x557fca771740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 --2- 172.21.15.3:0/3235302026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca75fd80 0x557fca767cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 -- 172.21.15.3:0/3235302026 >> 172.21.15.3:0/3235302026 conn(0x557fca75b2f0 msgr2=0x557fca761210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 -- 172.21.15.3:0/3235302026 shutdown_connections 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 -- 172.21.15.3:0/3235302026 wait complete. 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.056+0000 7f053bdfd740 1 -- start start 2024-03-21T08:30:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 1 Processor -- start 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca76d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca7658e0 0x557fca7709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca777600 con 0x557fca760770 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca776fa0 con 0x557fca7658e0 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fca777c60 con 0x557fca75fd80 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f053bdfd740 10 client.0 _ll_register_callbacks cb 0x7f052c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.057+0000 7f0531400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34426/0 (socket says 172.21.15.3:34426) 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 -- 172.21.15.3:0/2524365444 learned_addr learned my addr 172.21.15.3:0/2524365444 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0532800640 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca7658e0 0x557fca7709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531e00640 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca76d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 -- 172.21.15.3:0/2524365444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 msgr2=0x557fca76d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca76d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 -- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca7658e0 msgr2=0x557fca7709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca7658e0 0x557fca7709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 -- 172.21.15.3:0/2524365444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fca771030 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0531400640 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76deb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f052402c6a0 tx=0x7f05240043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d12906d4747e2c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0517e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0517e00640 1 -- 172.21.15.3:0/2524365444 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524044040 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f0517e00640 1 -- 172.21.15.3:0/2524365444 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0524049920 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.058+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f052c01b900 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f0517e00640 1 -- 172.21.15.3:0/2524365444 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0524067050 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 10 client.15249 fetch_fsmap learned FSMap version 22 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557fca7d2020 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f0517e00640 1 -- 172.21.15.3:0/2524365444 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f052c01b900 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f0517e00640 1 -- 172.21.15.3:0/2524365444 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0524040470 con 0x557fca760770 2024-03-21T08:30:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 10 client.15249 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:30:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 10 client.15249 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T08:30:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 20 client.15249 populate_metadata read hostname 'smithi003' 2024-03-21T08:30:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca7d6610 0x557fca7d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 4 client.15249 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:30:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.059+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557fca6d21d0 con 0x557fca7d6610 2024-03-21T08:30:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.060+0000 7f0531e00640 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca7d6610 0x557fca7d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.061+0000 7f0531e00640 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca7d6610 0x557fca7d2250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96f28e79c10803a5 server_cookie=9daf3e12c16e17c8 in_seq=0 out_seq=0 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.061+0000 7f0517e00640 10 client.15249 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.061+0000 7f0517e00640 1 -- 172.21.15.3:0/2524365444 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557fca6d21d0 con 0x557fca7d6610 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.061+0000 7f0517e00640 10 client.15249 handle_command_reply: tid=0 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.061+0000 7f053bdfd740 1 client.15249 shutdown 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 msgr2=0x557fca76deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76deb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f052402c6a0 tx=0x7f05240043d0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 shutdown_connections 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x557fca7d6610 0x557fca7d2250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fca7658e0 0x557fca7709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fca760770 0x557fca76deb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 --2- 172.21.15.3:0/2524365444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fca75fd80 0x557fca76d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f0517e00640 0 client.15249 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 >> 172.21.15.3:0/2524365444 conn(0x557fca75b2f0 msgr2=0x557fca768d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 shutdown_connections 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 1 -- 172.21.15.3:0/2524365444 wait complete. 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f053bdfd740 20 client.15249 trim_cache size 0 max 16384 2024-03-21T08:30:46.067 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 34409, 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 97.746459826999995, 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 96, 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 94, 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 174.02938834700001 2024-03-21T08:30:46.068 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:30:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05180a2fa0 msgr2=0x7f05180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.062+0000 7f0533200640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05180a2fa0 0x7f05180a5460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557fca760fd0 tx=0x7f052c009500 comp rx=0 tx=0).stop 2024-03-21T08:30:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341014f0 msgr2=0x7f0534199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341014f0 0x7f0534199940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e9c0 tx=0x7f051c00ee80 comp rx=0 tx=0).stop 2024-03-21T08:30:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 shutdown_connections 2024-03-21T08:30:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05341030a0 0x7f05341a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f05180a2fa0 0x7f05180a5460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05341026e0 0x7f0534199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 --2- 172.21.15.3:0/3952818784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05341014f0 0x7f0534199940 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 >> 172.21.15.3:0/3952818784 conn(0x7f05340fca60 msgr2=0x7f05340738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 shutdown_connections 2024-03-21T08:30:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.063+0000 7f0533200640 1 -- 172.21.15.3:0/3952818784 wait complete. 2024-03-21T08:30:46.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:46 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3247579652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:46.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:46 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2667841410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:46.083 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-21T08:30:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.295+0000 7f372a600640 1 -- 172.21.15.3:0/2908290246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 msgr2=0x7f3724103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.295+0000 7f372a600640 1 --2- 172.21.15.3:0/2908290246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724103b20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3718009900 tx=0x7f371802bc50 comp rx=0 tx=0).stop 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.295+0000 7f372a600640 1 -- 172.21.15.3:0/2908290246 shutdown_connections 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.295+0000 7f372a600640 1 --2- 172.21.15.3:0/2908290246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37241052b0 0x7f372410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.295+0000 7f372a600640 1 --2- 172.21.15.3:0/2908290246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37241048f0 0x7f3724104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.295+0000 7f372a600640 1 --2- 172.21.15.3:0/2908290246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724103b20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.295+0000 7f372a600640 1 -- 172.21.15.3:0/2908290246 >> 172.21.15.3:0/2908290246 conn(0x7f37240fec70 msgr2=0x7f37241010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.296+0000 7f372a600640 1 -- 172.21.15.3:0/2908290246 shutdown_connections 2024-03-21T08:30:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.296+0000 7f372a600640 1 -- 172.21.15.3:0/2908290246 wait complete. 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 Processor -- start 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 -- start start 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37241048f0 0x7f3724199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37241052b0 0x7f37241a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372410f250 con 0x7f3724103700 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372410f0d0 con 0x7f37241048f0 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.297+0000 7f372a600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372410f3d0 con 0x7f37241052b0 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34448/0 (socket says 172.21.15.3:34448) 2024-03-21T08:30:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 -- 172.21.15.3:0/1690129619 learned_addr learned my addr 172.21.15.3:0/1690129619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3723e00640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37241048f0 0x7f3724199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729c00640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37241052b0 0x7f37241a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 -- 172.21.15.3:0/1690129619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37241052b0 msgr2=0x7f37241a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37241052b0 0x7f37241a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 -- 172.21.15.3:0/1690129619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37241048f0 msgr2=0x7f3724199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37241048f0 0x7f3724199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 -- 172.21.15.3:0/1690129619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37241a1570 con 0x7f3724103700 2024-03-21T08:30:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3723e00640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37241048f0 0x7f3724199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:30:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.298+0000 7f3729200640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724199950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3718005ab0 tx=0x7f37180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.299+0000 7f3721600640 1 -- 172.21.15.3:0/1690129619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718043020 con 0x7f3724103700 2024-03-21T08:30:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.299+0000 7f3721600640 1 -- 172.21.15.3:0/1690129619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f37180054c0 con 0x7f3724103700 2024-03-21T08:30:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.299+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37241a1800 con 0x7f3724103700 2024-03-21T08:30:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.299+0000 7f3721600640 1 -- 172.21.15.3:0/1690129619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718003d10 con 0x7f3724103700 2024-03-21T08:30:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.299+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37241a1d50 con 0x7f3724103700 2024-03-21T08:30:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.303+0000 7f372e987740 1 Processor -- start 2024-03-21T08:30:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f372e987740 1 -- start start 2024-03-21T08:30:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f372e987740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a671a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f3721600640 1 -- 172.21.15.3:0/1690129619 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f3718046020 con 0x7f3724103700 2024-03-21T08:30:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f372e987740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f372e987740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a6ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f372e987740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565479cbc90 con 0x556547a66d80 2024-03-21T08:30:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f372e987740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547974980 con 0x556547a67770 2024-03-21T08:30:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f372e987740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565479c7d30 con 0x556547a6c8e0 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f3729c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f3729c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a671a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34452/0 (socket says 172.21.15.3:34452) 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.304+0000 7f3729c00640 1 -- 172.21.15.3:0/1675772164 learned_addr learned my addr 172.21.15.3:0/1675772164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729200640 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729c00640 1 -- 172.21.15.3:0/1675772164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 msgr2=0x556547a6ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3723e00640 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a6ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729c00640 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a6ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729c00640 1 -- 172.21.15.3:0/1675772164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 msgr2=0x556547a6c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729c00640 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729c00640 1 -- 172.21.15.3:0/1675772164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55654798c510 con 0x556547a66d80 2024-03-21T08:30:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729200640 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:30:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.305+0000 7f3729c00640 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a671a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f371c007dd0 tx=0x7f371c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbfb41bbe96dfcf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.306+0000 7f3712000640 1 -- 172.21.15.3:0/1675772164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c014070 con 0x556547a66d80 2024-03-21T08:30:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.306+0000 7f3721600640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f370c0a2f20 0x7f370c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.306+0000 7f3712000640 1 -- 172.21.15.3:0/1675772164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f371c004d20 con 0x556547a66d80 2024-03-21T08:30:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.306+0000 7f3712000640 1 -- 172.21.15.3:0/1675772164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c0107b0 con 0x556547a66d80 2024-03-21T08:30:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.306+0000 7f3721600640 1 -- 172.21.15.3:0/1690129619 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f37180f1c50 con 0x7f3724103700 2024-03-21T08:30:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.306+0000 7f3729200640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f370c0a2f20 0x7f370c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 -- 172.21.15.3:0/1675772164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 msgr2=0x556547a671a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a671a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f371c007dd0 tx=0x7f371c00e720 comp rx=0 tx=0).stop 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f3729200640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f370c0a2f20 0x7f370c0a53e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f37180bde70 tx=0x7f37180de040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 -- 172.21.15.3:0/1675772164 shutdown_connections 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a6ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 --2- 172.21.15.3:0/1675772164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a671a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 -- 172.21.15.3:0/1675772164 >> 172.21.15.3:0/1675772164 conn(0x556547a622f0 msgr2=0x556547a626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 -- 172.21.15.3:0/1675772164 shutdown_connections 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 -- 172.21.15.3:0/1675772164 wait complete. 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.307+0000 7f372e987740 1 -- start start 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 1 Processor -- start 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a71830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a78870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a6f890 con 0x556547a66d80 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547987dd0 con 0x556547a67770 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a6fa10 con 0x556547a6c8e0 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f372e987740 10 client.0 _ll_register_callbacks cb 0x5565476dfac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f3729c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f3729c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a712f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34462/0 (socket says 172.21.15.3:34462) 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f3729c00640 1 -- 172.21.15.3:0/2909806018 learned_addr learned my addr 172.21.15.3:0/2909806018 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.308+0000 7f3723e00640 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a71830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3729c00640 1 -- 172.21.15.3:0/2909806018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 msgr2=0x556547a78870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3729c00640 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a78870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3729c00640 1 -- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 msgr2=0x556547a71830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3729c00640 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a71830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3729c00640 1 -- 172.21.15.3:0/2909806018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565476e02d0 con 0x556547a66d80 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3723e00640 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a71830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3729c00640 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a712f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f371c007da0 tx=0x7f371c0072e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c55bbfdaffcc725 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3706a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.309+0000 7f3706a00640 1 -- 172.21.15.3:0/2909806018 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c014070 con 0x556547a66d80 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f3706a00640 1 -- 172.21.15.3:0/2909806018 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f371c010530 con 0x556547a66d80 2024-03-21T08:30:46.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f371c004d20 con 0x556547a66d80 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f3706a00640 1 -- 172.21.15.3:0/2909806018 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f371c010730 con 0x556547a66d80 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 10 client.15261 fetch_fsmap learned FSMap version 22 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5565477aa6b0 con 0x556547a66d80 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f3706a00640 1 -- 172.21.15.3:0/2909806018 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f371c004d20 con 0x556547a66d80 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f3706a00640 1 -- 172.21.15.3:0/2909806018 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f371c0259d0 con 0x556547a66d80 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 10 client.15261 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 10 client.15261 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 20 client.15261 populate_metadata read hostname 'smithi003' 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547a7cfd0 0x556547a81a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 4 client.15261 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.310+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55654796d520 con 0x556547a7cfd0 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.311+0000 7f3723e00640 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547a7cfd0 0x556547a81a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.312+0000 7f3723e00640 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547a7cfd0 0x556547a81a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d528b241710fa51 server_cookie=4c527cc4c5b1f32f in_seq=0 out_seq=0 2024-03-21T08:30:46.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.312+0000 7f3706a00640 10 client.15261 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:30:46.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.313+0000 7f3706a00640 1 -- 172.21.15.3:0/2909806018 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55654796d520 con 0x556547a7cfd0 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.313+0000 7f3706a00640 10 client.15261 handle_command_reply: tid=0 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.313+0000 7f372e987740 1 client.15261 shutdown 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 msgr2=0x556547a712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a712f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f371c007da0 tx=0x7f371c0072e0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 shutdown_connections 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547a7cfd0 0x556547a81a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a78870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a71830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f3706a00640 0 client.15261 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 --2- 172.21.15.3:0/2909806018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a712f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 >> 172.21.15.3:0/2909806018 conn(0x556547a622f0 msgr2=0x556547a614f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 shutdown_connections 2024-03-21T08:30:46.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 1 -- 172.21.15.3:0/2909806018 wait complete. 2024-03-21T08:30:46.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.314+0000 7f372e987740 20 client.15261 trim_cache size 0 max 16384 2024-03-21T08:30:46.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.332+0000 7f372e987740 1 Processor -- start 2024-03-21T08:30:46.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f372e987740 1 -- start start 2024-03-21T08:30:46.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f372e987740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a71790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f372e987740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f372e987740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a78800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f372e987740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a7ce50 con 0x556547a66d80 2024-03-21T08:30:46.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f372e987740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a7d490 con 0x556547a67770 2024-03-21T08:30:46.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f372e987740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a7daf0 con 0x556547a6c8e0 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f3729c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a71790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f3729c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a71790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34476/0 (socket says 172.21.15.3:34476) 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f3729c00640 1 -- 172.21.15.3:0/3149712990 learned_addr learned my addr 172.21.15.3:0/3149712990 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.333+0000 7f3723e00640 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.334+0000 7f3729c00640 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a78800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.334+0000 7f3729c00640 1 -- 172.21.15.3:0/3149712990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 msgr2=0x556547a78800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.334+0000 7f3729c00640 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a78800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.334+0000 7f3729c00640 1 -- 172.21.15.3:0/3149712990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 msgr2=0x556547a6ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.334+0000 7f3729c00640 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.334+0000 7f3729c00640 1 -- 172.21.15.3:0/3149712990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556547a54170 con 0x556547a66d80 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.334+0000 7f3729c00640 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a71790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f371c0073b0 tx=0x7f371c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cb6e4f3c40f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f3711600640 1 -- 172.21.15.3:0/3149712990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c039070 con 0x556547a66d80 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f3711600640 1 -- 172.21.15.3:0/3149712990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f371c020070 con 0x556547a66d80 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f3711600640 1 -- 172.21.15.3:0/3149712990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c014070 con 0x556547a66d80 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 -- 172.21.15.3:0/3149712990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 msgr2=0x556547a71790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a71790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f371c0073b0 tx=0x7f371c0056e0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 -- 172.21.15.3:0/3149712990 shutdown_connections 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a6c8e0 0x556547a78800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 --2- 172.21.15.3:0/3149712990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a66d80 0x556547a71790 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 -- 172.21.15.3:0/3149712990 >> 172.21.15.3:0/3149712990 conn(0x556547a622f0 msgr2=0x556547a68210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.335+0000 7f372e987740 1 -- 172.21.15.3:0/3149712990 shutdown_connections 2024-03-21T08:30:46.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.336+0000 7f372e987740 1 -- 172.21.15.3:0/3149712990 wait complete. 2024-03-21T08:30:46.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.336+0000 7f372e987740 1 -- start start 2024-03-21T08:30:46.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.336+0000 7f372e987740 1 Processor -- start 2024-03-21T08:30:46.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f372e987740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a66d80 0x556547a6ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f372e987740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f372e987740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a6c8e0 0x556547ad6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f372e987740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a7e7b0 con 0x556547a6c8e0 2024-03-21T08:30:46.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f372e987740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a7e150 con 0x556547a67770 2024-03-21T08:30:46.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f372e987740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x556547a7ee10 con 0x556547a66d80 2024-03-21T08:30:46.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f372e987740 10 client.0 _ll_register_callbacks cb 0x7f371c007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:30:46.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3729c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a6c8e0 0x556547ad6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56194/0 (socket says 172.21.15.3:56194) 2024-03-21T08:30:46.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 -- 172.21.15.3:0/3309586522 learned_addr learned my addr 172.21.15.3:0/3309586522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3729c00640 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a66d80 0x556547a6ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 -- 172.21.15.3:0/3309586522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a66d80 msgr2=0x556547a6ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a66d80 0x556547a6ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 -- 172.21.15.3:0/3309586522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a6c8e0 msgr2=0x556547ad6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a6c8e0 0x556547ad6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3723e00640 1 -- 172.21.15.3:0/3309586522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556547ad7190 con 0x556547a67770 2024-03-21T08:30:46.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.337+0000 7f3729c00640 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a6c8e0 0x556547ad6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:30:46.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.338+0000 7f3723e00640 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6b340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x556547a63260 tx=0x7f3714005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=848c6e2e2cfbb6cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.338+0000 7f3706a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.338+0000 7f3706a00640 1 -- 172.21.15.3:0/3309586522 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371401d6f0 con 0x556547a67770 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.338+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f371c020070 con 0x556547a67770 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.338+0000 7f3706a00640 1 -- 172.21.15.3:0/3309586522 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f371401d890 con 0x556547a67770 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.339+0000 7f3706a00640 1 -- 172.21.15.3:0/3309586522 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f371401da90 con 0x556547a67770 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.339+0000 7f372e987740 10 client.35048 fetch_fsmap learned FSMap version 22 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.339+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556547ad8590 con 0x556547a67770 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.339+0000 7f3706a00640 1 -- 172.21.15.3:0/3309586522 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f371c020070 con 0x556547a67770 2024-03-21T08:30:46.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.339+0000 7f3706a00640 1 -- 172.21.15.3:0/3309586522 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3714024070 con 0x556547a67770 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.340+0000 7f372e987740 10 client.35048 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.340+0000 7f372e987740 10 client.35048 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.340+0000 7f372e987740 20 client.35048 populate_metadata read hostname 'smithi003' 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.340+0000 7f372e987740 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547adccb0 0x556547ad8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.340+0000 7f372e987740 4 client.35048 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.340+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565479d91d0 con 0x556547adccb0 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.340+0000 7f3729c00640 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547adccb0 0x556547ad8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f3729c00640 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547adccb0 0x556547ad8880 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf42c84a15cf287a server_cookie=395204d50a2f22aa in_seq=0 out_seq=0 2024-03-21T08:30:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f3706a00640 10 client.35048 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:30:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f3706a00640 1 -- 172.21.15.3:0/3309586522 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5565479d91d0 con 0x556547adccb0 2024-03-21T08:30:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f3706a00640 10 client.35048 handle_command_reply: tid=0 2024-03-21T08:30:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 client.35048 shutdown 2024-03-21T08:30:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 msgr2=0x556547a6b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6b340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x556547a63260 tx=0x7f3714005410 comp rx=0 tx=0).stop 2024-03-21T08:30:46.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 shutdown_connections 2024-03-21T08:30:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556547a6c8e0 0x556547ad6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556547a67770 0x556547a6b340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x556547adccb0 0x556547ad8880 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 --2- 172.21.15.3:0/3309586522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556547a66d80 0x556547a6ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 >> 172.21.15.3:0/3309586522 conn(0x556547a622f0 msgr2=0x556547a626d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.341+0000 7f3706a00640 0 client.35048 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:30:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 shutdown_connections 2024-03-21T08:30:46.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372e987740 1 -- 172.21.15.3:0/3309586522 wait complete. 2024-03-21T08:30:46.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372e987740 20 client.35048 trim_cache size 0 max 16384 2024-03-21T08:30:46.351 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:30:46.351 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:30:46.352 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-03-21T08:30:46.352 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 34439, 2024-03-21T08:30:46.352 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-03-21T08:30:46.352 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-03-21T08:30:46.352 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-03-21T08:30:46.352 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 99.044741193999997, 2024-03-21T08:30:46.353 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-03-21T08:30:46.353 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 96, 2024-03-21T08:30:46.353 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 94, 2024-03-21T08:30:46.353 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 162.379284248 2024-03-21T08:30:46.353 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f370c0a2f20 msgr2=0x7f370c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f370c0a2f20 0x7f370c0a53e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f37180bde70 tx=0x7f37180de040 comp rx=0 tx=0).stop 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 msgr2=0x7f3724199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724199950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3718005ab0 tx=0x7f37180042f0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 shutdown_connections 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f370c0a2f20 0x7f370c0a53e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37241052b0 0x7f37241a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37241048f0 0x7f3724199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 --2- 172.21.15.3:0/1690129619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3724103700 0x7f3724199950 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.342+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 >> 172.21.15.3:0/1690129619 conn(0x7f37240fec70 msgr2=0x7f3724107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.343+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 shutdown_connections 2024-03-21T08:30:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.343+0000 7f372a600640 1 -- 172.21.15.3:0/1690129619 wait complete. 2024-03-21T08:30:46.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-21T08:30:46.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:46 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3247579652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:46.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:46 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2667841410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:46.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:46 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3247579652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:30:46.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:46 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2667841410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.580+0000 7f4a15800640 1 -- 172.21.15.3:0/2998748525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10107ce0 msgr2=0x7f4a10108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.580+0000 7f4a15800640 1 --2- 172.21.15.3:0/2998748525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10107ce0 0x7f4a10108100 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0002bc50 comp rx=0 tx=0).stop 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.581+0000 7f4a15800640 1 -- 172.21.15.3:0/2998748525 shutdown_connections 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.581+0000 7f4a15800640 1 --2- 172.21.15.3:0/2998748525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a10109890 0x7f4a10110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.581+0000 7f4a15800640 1 --2- 172.21.15.3:0/2998748525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a10108ed0 0x7f4a10109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.581+0000 7f4a15800640 1 --2- 172.21.15.3:0/2998748525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10107ce0 0x7f4a10108100 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.581+0000 7f4a15800640 1 -- 172.21.15.3:0/2998748525 >> 172.21.15.3:0/2998748525 conn(0x7f4a10076c90 msgr2=0x7f4a100790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.581+0000 7f4a15800640 1 -- 172.21.15.3:0/2998748525 shutdown_connections 2024-03-21T08:30:46.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.581+0000 7f4a15800640 1 -- 172.21.15.3:0/2998748525 wait complete. 2024-03-21T08:30:46.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 Processor -- start 2024-03-21T08:30:46.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 -- start start 2024-03-21T08:30:46.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a10107ce0 0x7f4a1019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10108ed0 0x7f4a1019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a10109890 0x7f4a101a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a101138d0 con 0x7f4a10108ed0 2024-03-21T08:30:46.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10113750 con 0x7f4a10107ce0 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.582+0000 7f4a15800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10113a50 con 0x7f4a10109890 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10108ed0 0x7f4a1019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a10107ce0 0x7f4a1019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10108ed0 0x7f4a1019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34494/0 (socket says 172.21.15.3:34494) 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 -- 172.21.15.3:0/172713141 learned_addr learned my addr 172.21.15.3:0/172713141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0f400640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a10109890 0x7f4a101a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 -- 172.21.15.3:0/172713141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a10109890 msgr2=0x7f4a101a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a10109890 0x7f4a101a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 -- 172.21.15.3:0/172713141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a10107ce0 msgr2=0x7f4a1019e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a10107ce0 0x7f4a1019e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.583+0000 7f4a0e000640 1 -- 172.21.15.3:0/172713141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a101a5c20 con 0x7f4a10108ed0 2024-03-21T08:30:46.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.584+0000 7f4a0e000640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10108ed0 0x7f4a1019e540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f49f800e9f0 tx=0x7f49f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.584+0000 7f4a07400640 1 -- 172.21.15.3:0/172713141 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800cd80 con 0x7f4a10108ed0 2024-03-21T08:30:46.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.584+0000 7f4a07400640 1 -- 172.21.15.3:0/172713141 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f49f80108e0 con 0x7f4a10108ed0 2024-03-21T08:30:46.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.584+0000 7f4a07400640 1 -- 172.21.15.3:0/172713141 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8018920 con 0x7f4a10108ed0 2024-03-21T08:30:46.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.584+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a101a5f10 con 0x7f4a10108ed0 2024-03-21T08:30:46.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.584+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a101a6400 con 0x7f4a10108ed0 2024-03-21T08:30:46.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a07400640 1 -- 172.21.15.3:0/172713141 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f49f8010a90 con 0x7f4a10108ed0 2024-03-21T08:30:46.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 Processor -- start 2024-03-21T08:30:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 -- start start 2024-03-21T08:30:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dd590 0x55b6102e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102e2700 0x55b6102e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b610241c90 con 0x55b6102dcba0 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6101ea980 con 0x55b6102e2700 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.588+0000 7f4a1856e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61023dd30 con 0x55b6102dd590 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102dcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34504/0 (socket says 172.21.15.3:34504) 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 -- 172.21.15.3:0/1227022435 learned_addr learned my addr 172.21.15.3:0/1227022435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0e000640 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102e2700 0x55b6102e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dd590 0x55b6102e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 -- 172.21.15.3:0/1227022435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dd590 msgr2=0x55b6102e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dd590 0x55b6102e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 -- 172.21.15.3:0/1227022435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102e2700 msgr2=0x55b6102e4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102e2700 0x55b6102e4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a0f400640 1 -- 172.21.15.3:0/1227022435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b610202510 con 0x55b6102dcba0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.589+0000 7f4a07400640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49f40a2fa0 0x7f49f40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.590+0000 7f4a07400640 1 -- 172.21.15.3:0/172713141 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f49f80cfb60 con 0x7f4a10108ed0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.590+0000 7f4a0f400640 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102dcfc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009f60 tx=0x7f4a08009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a672e33ce813d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.590+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49f40a2fa0 0x7f49f40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.590+0000 7f49ffe00640 1 -- 172.21.15.3:0/1227022435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0801a070 con 0x55b6102dcba0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.590+0000 7f49ffe00640 1 -- 172.21.15.3:0/1227022435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a080158c0 con 0x55b6102dcba0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.590+0000 7f49ffe00640 1 -- 172.21.15.3:0/1227022435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800a750 con 0x55b6102dcba0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 -- 172.21.15.3:0/1227022435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 msgr2=0x55b6102dcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49f40a2fa0 0x7f49f40a5460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002c260 tx=0x7f4a0002c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102dcfc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009f60 tx=0x7f4a08009f90 comp rx=0 tx=0).stop 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 -- 172.21.15.3:0/1227022435 shutdown_connections 2024-03-21T08:30:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102e2700 0x55b6102e4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dd590 0x55b6102e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1227022435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102dcfc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 -- 172.21.15.3:0/1227022435 >> 172.21.15.3:0/1227022435 conn(0x55b6102d8110 msgr2=0x55b6102d8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 -- 172.21.15.3:0/1227022435 shutdown_connections 2024-03-21T08:30:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.591+0000 7f4a1856e740 1 -- 172.21.15.3:0/1227022435 wait complete. 2024-03-21T08:30:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 -- start start 2024-03-21T08:30:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 Processor -- start 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dcba0 0x55b6102e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dd590 0x55b6102e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 0x55b6102ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102e56b0 con 0x55b6102dd590 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6101fddd0 con 0x55b6102dcba0 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.592+0000 7f4a1856e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102e5830 con 0x55b6102e2700 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a1856e740 10 client.0 _ll_register_callbacks cb 0x55b60ff55ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:30:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dd590 0x55b6102e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dd590 0x55b6102e7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34516/0 (socket says 172.21.15.3:34516) 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 -- 172.21.15.3:0/1252206076 learned_addr learned my addr 172.21.15.3:0/1252206076 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dcba0 0x55b6102e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0f400640 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 0x55b6102ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 -- 172.21.15.3:0/1252206076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 msgr2=0x55b6102ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 0x55b6102ee6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 -- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dcba0 msgr2=0x55b6102e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dcba0 0x55b6102e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 -- 172.21.15.3:0/1252206076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b60ff562d0 con 0x55b6102dd590 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f4a0e000640 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dd590 0x55b6102e7680 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f49f8098250 tx=0x7f49f809bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03a763d9b6b0c5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.593+0000 7f49fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f49fcc00640 1 -- 172.21.15.3:0/1252206076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f809c4f0 con 0x55b6102dd590 2024-03-21T08:30:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f49fcc00640 1 -- 172.21.15.3:0/1252206076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f49f80cd2d0 con 0x55b6102dd590 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a080158c0 con 0x55b6102dd590 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f49fcc00640 1 -- 172.21.15.3:0/1252206076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f809bdf0 con 0x55b6102dd590 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f4a1856e740 10 client.15279 fetch_fsmap learned FSMap version 22 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b6100206b0 con 0x55b6102dd590 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f49fcc00640 1 -- 172.21.15.3:0/1252206076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a080158c0 con 0x55b6102dd590 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.594+0000 7f49fcc00640 1 -- 172.21.15.3:0/1252206076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f49f809b240 con 0x55b6102dd590 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.595+0000 7f4a1856e740 10 client.15279 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.595+0000 7f4a1856e740 10 client.15279 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.595+0000 7f4a1856e740 20 client.15279 populate_metadata read hostname 'smithi003' 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.595+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6102f38f0 0x55b6102f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.595+0000 7f4a1856e740 4 client.15279 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.595+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6101e3520 con 0x55b6102f38f0 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.595+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6102f38f0 0x55b6102f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.596+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6102f38f0 0x55b6102f2a00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db17d96e893f95a server_cookie=1e5b0532d9f8fa88 in_seq=0 out_seq=0 2024-03-21T08:30:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.596+0000 7f49fcc00640 10 client.15279 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f49fcc00640 1 -- 172.21.15.3:0/1252206076 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b6101e3520 con 0x55b6102f38f0 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f49fcc00640 10 client.15279 handle_command_reply: tid=0 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 client.15279 shutdown 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dd590 msgr2=0x55b6102e7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dd590 0x55b6102e7680 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f49f8098250 tx=0x7f49f809bc80 comp rx=0 tx=0).stop 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 shutdown_connections 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6102f38f0 0x55b6102f2a00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 0x55b6102ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dd590 0x55b6102e7680 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1252206076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dcba0 0x55b6102e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 >> 172.21.15.3:0/1252206076 conn(0x55b6102d8110 msgr2=0x55b6102d7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f49fcc00640 0 client.15279 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:30:46.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 shutdown_connections 2024-03-21T08:30:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.597+0000 7f4a1856e740 1 -- 172.21.15.3:0/1252206076 wait complete. 2024-03-21T08:30:46.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.598+0000 7f4a1856e740 20 client.15279 trim_cache size 0 max 16384 2024-03-21T08:30:46.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 Processor -- start 2024-03-21T08:30:46.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 -- start start 2024-03-21T08:30:46.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102ddf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 0x55b6102d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102f3110 con 0x55b6102dcba0 2024-03-21T08:30:46.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102f3770 con 0x55b6102dd590 2024-03-21T08:30:46.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.616+0000 7f4a1856e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102f3dd0 con 0x55b6102e2700 2024-03-21T08:30:46.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102ddf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102ddf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34524/0 (socket says 172.21.15.3:34524) 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 -- 172.21.15.3:0/1621249099 learned_addr learned my addr 172.21.15.3:0/1621249099 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0e000640 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 -- 172.21.15.3:0/1621249099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 msgr2=0x55b6102d84f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 0x55b6102d84f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 -- 172.21.15.3:0/1621249099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 msgr2=0x55b6102ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.617+0000 7f4a0ea00640 1 -- 172.21.15.3:0/1621249099 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6102e79a0 con 0x55b6102dcba0 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.618+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102ddf90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005ab0 tx=0x7f4a00040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b8417538c36f18 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.618+0000 7f49ff400640 1 -- 172.21.15.3:0/1621249099 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000042a0 con 0x55b6102dcba0 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.618+0000 7f49ff400640 1 -- 172.21.15.3:0/1621249099 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a0005b070 con 0x55b6102dcba0 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.618+0000 7f49ff400640 1 -- 172.21.15.3:0/1621249099 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00004ab0 con 0x55b6102dcba0 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 -- 172.21.15.3:0/1621249099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 msgr2=0x55b6102ddf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102ddf90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005ab0 tx=0x7f4a00040000 comp rx=0 tx=0).stop 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 -- 172.21.15.3:0/1621249099 shutdown_connections 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102e2700 0x55b6102d84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 --2- 172.21.15.3:0/1621249099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102dcba0 0x55b6102ddf90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 -- 172.21.15.3:0/1621249099 >> 172.21.15.3:0/1621249099 conn(0x55b6102d8110 msgr2=0x55b6102e5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 -- 172.21.15.3:0/1621249099 shutdown_connections 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 -- 172.21.15.3:0/1621249099 wait complete. 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.619+0000 7f4a1856e740 1 -- start start 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 1 Processor -- start 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dcba0 0x55b6102ea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102e2700 0x55b6102edea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102f4a90 con 0x55b6102e2700 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102f4430 con 0x55b6102dd590 2024-03-21T08:30:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6102f50f0 con 0x55b6102dcba0 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a1856e740 10 client.0 _ll_register_callbacks cb 0x7f49f80d3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a0f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102e2700 0x55b6102edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a0f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102e2700 0x55b6102edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34540/0 (socket says 172.21.15.3:34540) 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a0f400640 1 -- 172.21.15.3:0/2785483169 learned_addr learned my addr 172.21.15.3:0/2785483169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.620+0000 7f4a0e000640 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f4a0f400640 1 -- 172.21.15.3:0/2785483169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dcba0 msgr2=0x55b6102ea940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f4a0f400640 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dcba0 0x55b6102ea940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f4a0f400640 1 -- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 msgr2=0x55b6102eae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:30:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f4a0f400640 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102eae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f4a0f400640 1 -- 172.21.15.3:0/2785483169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6102ee490 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f4a0f400640 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102e2700 0x55b6102edea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a08018040 tx=0x7f4a080096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdeee33c256ec47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f49fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f49fcc00640 1 -- 172.21.15.3:0/2785483169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0801a040 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f49fcc00640 1 -- 172.21.15.3:0/2785483169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f4a08003a90 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.621+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a0005b070 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f49fcc00640 1 -- 172.21.15.3:0/2785483169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a08003c90 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 10 client.15285 fetch_fsmap learned FSMap version 22 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b61034f430 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f49fcc00640 1 -- 172.21.15.3:0/2785483169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a0005b070 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f49fcc00640 1 -- 172.21.15.3:0/2785483169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4a08028030 con 0x55b6102e2700 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 10 client.15285 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:30:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 10 client.15285 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 20 client.15285 populate_metadata read hostname 'smithi003' 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6103537f0 0x55b61034f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 4 client.15285 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.622+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b61024f1d0 con 0x55b6103537f0 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.623+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6103537f0 0x55b61034f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a0ea00640 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6103537f0 0x55b61034f710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96337f35c5398d5 server_cookie=16655ad456c8e18 in_seq=0 out_seq=0 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f49fcc00640 10 client.15285 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f49fcc00640 1 -- 172.21.15.3:0/2785483169 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b61024f1d0 con 0x55b6103537f0 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f49fcc00640 10 client.15285 handle_command_reply: tid=0 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 client.15285 shutdown 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102e2700 msgr2=0x55b6102edea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102e2700 0x55b6102edea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a08018040 tx=0x7f4a080096c0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 shutdown_connections 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6102e2700 0x55b6102edea0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b6102dd590 0x55b6102eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b6103537f0 0x55b61034f710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 --2- 172.21.15.3:0/2785483169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b6102dcba0 0x55b6102ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f49fcc00640 0 client.15285 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.624+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 >> 172.21.15.3:0/2785483169 conn(0x55b6102d8110 msgr2=0x55b6102ddf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.625+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 shutdown_connections 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.625+0000 7f4a1856e740 1 -- 172.21.15.3:0/2785483169 wait complete. 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.625+0000 7f4a1856e740 20 client.15285 trim_cache size 0 max 16384 2024-03-21T08:30:46.633 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 34472, 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 129.91721767000001, 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 96, 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 94, 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 150.75949673100001 2024-03-21T08:30:46.634 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.625+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49f40a2fa0 msgr2=0x7f49f40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.625+0000 7f4a15800640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49f40a2fa0 0x7f49f40a5460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002c260 tx=0x7f4a0002c800 comp rx=0 tx=0).stop 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.625+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10108ed0 msgr2=0x7f4a1019e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.625+0000 7f4a15800640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10108ed0 0x7f4a1019e540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f49f800e9f0 tx=0x7f49f800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 shutdown_connections 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49f40a2fa0 0x7f49f40a5460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a10109890 0x7f4a101a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a10108ed0 0x7f4a1019e540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 --2- 172.21.15.3:0/172713141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a10107ce0 0x7f4a1019e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 >> 172.21.15.3:0/172713141 conn(0x7f4a10076c90 msgr2=0x7f4a1010c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 shutdown_connections 2024-03-21T08:30:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:46.626+0000 7f4a15800640 1 -- 172.21.15.3:0/172713141 wait complete. 2024-03-21T08:30:46.647 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T08:30:46.647 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T08:30:46.660 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:30:46.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-21T08:30:47.129 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:47.276 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:47 smithi003 ceph-mon[31675]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:47.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:47 smithi084 ceph-mon[35477]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:47.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:47 smithi171 ceph-mon[35388]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:49.084 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:49 smithi003 ceph-mon[31675]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:30:49.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 -- 172.21.15.3:0/2432548475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8077d90 msgr2=0x7fd1e8078210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:49.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 --2- 172.21.15.3:0/2432548475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8077d90 0x7fd1e8078210 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009990 tx=0x7fd1d4009ca0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 -- 172.21.15.3:0/2432548475 shutdown_connections 2024-03-21T08:30:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 --2- 172.21.15.3:0/2432548475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1e8078750 0x7fd1e8078fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 --2- 172.21.15.3:0/2432548475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e8076b40 0x7fd1e8076f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 --2- 172.21.15.3:0/2432548475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8077d90 0x7fd1e8078210 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 -- 172.21.15.3:0/2432548475 >> 172.21.15.3:0/2432548475 conn(0x7fd1e80ff950 msgr2=0x7fd1e8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.359+0000 7fd1ece00700 1 -- 172.21.15.3:0/2432548475 shutdown_connections 2024-03-21T08:30:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.360+0000 7fd1ece00700 1 -- 172.21.15.3:0/2432548475 wait complete. 2024-03-21T08:30:49.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 Processor -- start 2024-03-21T08:30:49.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 -- start start 2024-03-21T08:30:49.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8076b40 0x7fd1e819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e8077d90 0x7fd1e819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1e8078750 0x7fd1e81a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e81142b0 con 0x7fd1e8076b40 2024-03-21T08:30:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8114130 con 0x7fd1e8078750 2024-03-21T08:30:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.361+0000 7fd1ece00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8114430 con 0x7fd1e8077d90 2024-03-21T08:30:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8076b40 0x7fd1e819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8076b40 0x7fd1e819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34554/0 (socket says 172.21.15.3:34554) 2024-03-21T08:30:49.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 -- 172.21.15.3:0/78373658 learned_addr learned my addr 172.21.15.3:0/78373658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:49.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e6800700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1e8078750 0x7fd1e81a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:49.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5400700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e8077d90 0x7fd1e819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:49.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 -- 172.21.15.3:0/78373658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e8077d90 msgr2=0x7fd1e819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:49.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e8077d90 0x7fd1e819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 -- 172.21.15.3:0/78373658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1e8078750 msgr2=0x7fd1e81a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:49.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1e8078750 0x7fd1e81a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.362+0000 7fd1e5e00700 1 -- 172.21.15.3:0/78373658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e81a6560 con 0x7fd1e8076b40 2024-03-21T08:30:49.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.363+0000 7fd1e5e00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8076b40 0x7fd1e819e900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007e50 tx=0x7fd1d800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:49.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.363+0000 7fd1dea00700 1 -- 172.21.15.3:0/78373658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8016020 con 0x7fd1e8076b40 2024-03-21T08:30:49.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.363+0000 7fd1dea00700 1 -- 172.21.15.3:0/78373658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd1d800adc0 con 0x7fd1e8076b40 2024-03-21T08:30:49.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.363+0000 7fd1dea00700 1 -- 172.21.15.3:0/78373658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8011700 con 0x7fd1e8076b40 2024-03-21T08:30:49.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.363+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e81a6850 con 0x7fd1e8076b40 2024-03-21T08:30:49.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.363+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e81a6d90 con 0x7fd1e8076b40 2024-03-21T08:30:49.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.364+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e8068e50 con 0x7fd1e8076b40 2024-03-21T08:30:49.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.366+0000 7fd1dea00700 1 -- 172.21.15.3:0/78373658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd1d8019020 con 0x7fd1e8076b40 2024-03-21T08:30:49.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.368+0000 7fd1dea00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd1cc0a2fb0 0x7fd1cc0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:49.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.368+0000 7fd1dea00700 1 -- 172.21.15.3:0/78373658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd1d80cf3a0 con 0x7fd1e8076b40 2024-03-21T08:30:49.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.368+0000 7fd1e5400700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd1cc0a2fb0 0x7fd1cc0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:49.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.369+0000 7fd1e5400700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd1cc0a2fb0 0x7fd1cc0a5470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006010 tx=0x7fd1d4003870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.373+0000 7fd1dea00700 1 -- 172.21.15.3:0/78373658 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1d8020070 con 0x7fd1e8076b40 2024-03-21T08:30:49.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:49 smithi084 ceph-mon[35477]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:30:49.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:49 smithi171 ceph-mon[35388]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:30:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.623+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1e80625d0 con 0x7fd1cc0a2fb0 2024-03-21T08:30:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.625+0000 7fd1dea00700 1 -- 172.21.15.3:0/78373658 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd1e80625d0 con 0x7fd1cc0a2fb0 2024-03-21T08:30:49.626 INFO:teuthology.orchestra.run.smithi003.stdout:Backend: cephadm 2024-03-21T08:30:49.626 INFO:teuthology.orchestra.run.smithi003.stdout:Available: Yes 2024-03-21T08:30:49.627 INFO:teuthology.orchestra.run.smithi003.stdout:Paused: No 2024-03-21T08:30:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd1cc0a2fb0 msgr2=0x7fd1cc0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd1cc0a2fb0 0x7fd1cc0a5470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006010 tx=0x7fd1d4003870 comp rx=0 tx=0).stop 2024-03-21T08:30:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8076b40 msgr2=0x7fd1e819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8076b40 0x7fd1e819e900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007e50 tx=0x7fd1d800a340 comp rx=0 tx=0).stop 2024-03-21T08:30:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 shutdown_connections 2024-03-21T08:30:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd1cc0a2fb0 0x7fd1cc0a5470 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1e8076b40 0x7fd1e819e900 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1e8078750 0x7fd1e81a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 --2- 172.21.15.3:0/78373658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e8077d90 0x7fd1e819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:49.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.628+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 >> 172.21.15.3:0/78373658 conn(0x7fd1e80ff950 msgr2=0x7fd1e8101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:49.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.629+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 shutdown_connections 2024-03-21T08:30:49.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:49.629+0000 7fd1ece00700 1 -- 172.21.15.3:0/78373658 wait complete. 2024-03-21T08:30:50.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-21T08:30:50.898 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:51.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:51 smithi084 ceph-mon[35477]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:51.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:51 smithi084 ceph-mon[35477]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:51.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:51 smithi171 ceph-mon[35388]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:51.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:51 smithi171 ceph-mon[35388]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:51.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:51 smithi003 ceph-mon[31675]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:30:51.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:51 smithi003 ceph-mon[31675]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 -- 172.21.15.3:0/1222793769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4102100 msgr2=0x7fccc4102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 --2- 172.21.15.3:0/1222793769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4102100 0x7fccc4102520 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fccb80099c0 tx=0x7fccb8009cd0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 -- 172.21.15.3:0/1222793769 shutdown_connections 2024-03-21T08:30:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 --2- 172.21.15.3:0/1222793769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4102100 0x7fccc4102520 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 --2- 172.21.15.3:0/1222793769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4103ca0 0x7fccc410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 --2- 172.21.15.3:0/1222793769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc4103300 0x7fccc4103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 -- 172.21.15.3:0/1222793769 >> 172.21.15.3:0/1222793769 conn(0x7fccc40fd6a0 msgr2=0x7fccc40ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 -- 172.21.15.3:0/1222793769 shutdown_connections 2024-03-21T08:30:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.037+0000 7fcccb000700 1 -- 172.21.15.3:0/1222793769 wait complete. 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.038+0000 7fcccb000700 1 Processor -- start 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.038+0000 7fcccb000700 1 -- start start 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fcccb000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc4102100 0x7fccc419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fcccb000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4103300 0x7fccc419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fcccb000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4103ca0 0x7fccc41a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fcccb000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc410dbc0 con 0x7fccc4103ca0 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fcccb000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc410da40 con 0x7fccc4102100 2024-03-21T08:30:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fcccb000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc410dd40 con 0x7fccc4103300 2024-03-21T08:30:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fccca600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4103ca0 0x7fccc41a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fccca600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4103ca0 0x7fccc41a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41336/0 (socket says 172.21.15.3:41336) 2024-03-21T08:30:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fccca600700 1 -- 172.21.15.3:0/2630344326 learned_addr learned my addr 172.21.15.3:0/2630344326 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fccc9c00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc4102100 0x7fccc419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.039+0000 7fccc9200700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4103300 0x7fccc419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.040+0000 7fccca600700 1 -- 172.21.15.3:0/2630344326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4103300 msgr2=0x7fccc419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.040+0000 7fccca600700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4103300 0x7fccc419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.040+0000 7fccca600700 1 -- 172.21.15.3:0/2630344326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc4102100 msgr2=0x7fccc419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.040+0000 7fccca600700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc4102100 0x7fccc419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.040+0000 7fccca600700 1 -- 172.21.15.3:0/2630344326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc41a2060 con 0x7fccc4103ca0 2024-03-21T08:30:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.040+0000 7fccca600700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4103ca0 0x7fccc41a19c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fccbc007da0 tx=0x7fccbc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.043+0000 7fccc2000700 1 -- 172.21.15.3:0/2630344326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc016020 con 0x7fccc4103ca0 2024-03-21T08:30:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.043+0000 7fccc2000700 1 -- 172.21.15.3:0/2630344326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fccbc00ad00 con 0x7fccc4103ca0 2024-03-21T08:30:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.043+0000 7fccc2000700 1 -- 172.21.15.3:0/2630344326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0115e0 con 0x7fccc4103ca0 2024-03-21T08:30:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.043+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc41a2350 con 0x7fccc4103ca0 2024-03-21T08:30:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.044+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc41a2890 con 0x7fccc4103ca0 2024-03-21T08:30:53.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.045+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4068e50 con 0x7fccc4103ca0 2024-03-21T08:30:53.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.047+0000 7fccc2000700 1 -- 172.21.15.3:0/2630344326 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fccbc019020 con 0x7fccc4103ca0 2024-03-21T08:30:53.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.048+0000 7fccc2000700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fccac0a30b0 0x7fccac0a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:53.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.048+0000 7fccc2000700 1 -- 172.21.15.3:0/2630344326 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fccbc0cf830 con 0x7fccc4103ca0 2024-03-21T08:30:53.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.048+0000 7fccc9c00700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fccac0a30b0 0x7fccac0a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:53.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.049+0000 7fccc9c00700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fccac0a30b0 0x7fccac0a5570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fccb800b580 tx=0x7fccb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:53.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.053+0000 7fccc2000700 1 -- 172.21.15.3:0/2630344326 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fccbc020020 con 0x7fccc4103ca0 2024-03-21T08:30:53.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:53 smithi003 ceph-mon[31675]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-21T08:30:53.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.296+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fccc41a2ba0 con 0x7fccac0a30b0 2024-03-21T08:30:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.333+0000 7fccc2000700 1 -- 172.21.15.3:0/2630344326 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7fccc41a2ba0 con 0x7fccac0a30b0 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:mds.a smithi003 running (2m) 29s ago 2m 12.3M - 19.0.0-2231-g2edfee68 237e43c7a65f 04c79969f28d 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:mds.b smithi084 running (2m) 68s ago 2m 12.2M - 19.0.0-2231-g2edfee68 237e43c7a65f 8f5c231aee30 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:mds.c smithi171 running (2m) 69s ago 2m 14.4M - 19.0.0-2231-g2edfee68 237e43c7a65f a817d127d543 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:mds.d smithi003 running (2m) 29s ago 2m 12.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 217b9b8b110b 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:mds.e smithi084 running (2m) 68s ago 2m 15.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 88eb30fefa9e 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:mds.f smithi171 running (2m) 69s ago 2m 14.4M - 19.0.0-2231-g2edfee68 237e43c7a65f c2edf7ea5ad6 2024-03-21T08:30:53.335 INFO:teuthology.orchestra.run.smithi003.stdout:mds.g smithi003 running (2m) 29s ago 2m 12.3M - 19.0.0-2231-g2edfee68 237e43c7a65f 01c0afe2d76d 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mds.h smithi084 running (2m) 68s ago 2m 24.6M - 19.0.0-2231-g2edfee68 237e43c7a65f 83c96345e0c4 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mds.i smithi171 running (2m) 69s ago 2m 15.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 5339140889dd 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mds.j smithi003 running (2m) 29s ago 2m 12.5M - 19.0.0-2231-g2edfee68 237e43c7a65f a0523c19653a 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mds.k smithi084 running (3m) 68s ago 3m 15.6M - 19.0.0-2231-g2edfee68 237e43c7a65f 0498498dd345 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mds.l smithi171 running (3m) 69s ago 3m 12.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 378e40e0af04 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.x smithi003 *:9283,8765 running (12m) 29s ago 12m 515M - 19.0.0-2231-g2edfee68 237e43c7a65f 4a0ec4f047a2 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.y smithi084 *:8443,8765 running (10m) 68s ago 10m 451M - 19.0.0-2231-g2edfee68 237e43c7a65f 8fa6f4e455c9 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.z smithi171 *:8443,8765 running (10m) 69s ago 10m 451M - 19.0.0-2231-g2edfee68 237e43c7a65f 2645ed24aaae 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mon.a smithi003 running (12m) 29s ago 13m 48.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 0b9774172d25 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mon.b smithi084 running (10m) 68s ago 10m 39.7M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 9f1b9919572d 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:mon.c smithi171 running (10m) 69s ago 10m 39.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 5ae9e19f644e 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 smithi003 running (9m) 29s ago 9m 52.4M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f cf4fe1fd78f6 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 smithi084 running (9m) 68s ago 9m 46.3M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f de968d5cfdee 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 smithi171 running (8m) 69s ago 8m 54.0M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 1da8db4d81f7 2024-03-21T08:30:53.336 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 smithi003 running (8m) 29s ago 8m 52.1M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 990ff090437a 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 smithi084 running (7m) 68s ago 7m 42.0M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 396fff485347 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 smithi171 running (7m) 69s ago 7m 43.8M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 6c11021182b6 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 smithi003 running (6m) 29s ago 6m 49.6M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 8f5d2df0d519 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 smithi084 running (5m) 68s ago 5m 53.7M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f db476597cfd8 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 smithi171 running (5m) 69s ago 5m 50.2M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 5fb019aff6ae 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 smithi003 running (4m) 29s ago 4m 49.2M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f cc321b34bc33 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 smithi084 running (4m) 68s ago 4m 42.7M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f b7c2cfd6e7cc 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 smithi171 running (3m) 69s ago 3m 40.6M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 0862fe05f1d8 2024-03-21T08:30:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fccac0a30b0 msgr2=0x7fccac0a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fccac0a30b0 0x7fccac0a5570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fccb800b580 tx=0x7fccb8005fb0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4103ca0 msgr2=0x7fccc41a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4103ca0 0x7fccc41a19c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fccbc007da0 tx=0x7fccbc00a340 comp rx=0 tx=0).stop 2024-03-21T08:30:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 shutdown_connections 2024-03-21T08:30:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fccac0a30b0 0x7fccac0a5570 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccc4103ca0 0x7fccc41a19c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccc4102100 0x7fccc419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 --2- 172.21.15.3:0/2630344326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4103300 0x7fccc419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.337+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 >> 172.21.15.3:0/2630344326 conn(0x7fccc40fd6a0 msgr2=0x7fccc40743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.338+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 shutdown_connections 2024-03-21T08:30:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:53.338+0000 7fcccb000700 1 -- 172.21.15.3:0/2630344326 wait complete. 2024-03-21T08:30:53.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:53 smithi084 ceph-mon[35477]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-21T08:30:53.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:53 smithi171 ceph-mon[35388]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-21T08:30:54.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-21T08:30:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:55.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:55 smithi084 ceph-mon[35477]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:55.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:55 smithi084 ceph-mon[35477]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:55.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:55 smithi171 ceph-mon[35388]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:55.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:55 smithi171 ceph-mon[35388]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:55.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:55 smithi003 ceph-mon[31675]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:55.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:55 smithi003 ceph-mon[31675]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:30:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 -- 172.21.15.3:0/235708962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105fb0 msgr2=0x7f07cc10c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 --2- 172.21.15.3:0/235708962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105fb0 0x7f07cc10c860 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f07c400b0d0 tx=0x7f07c400b3e0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 -- 172.21.15.3:0/235708962 shutdown_connections 2024-03-21T08:30:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 --2- 172.21.15.3:0/235708962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105fb0 0x7f07cc10c860 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 --2- 172.21.15.3:0/235708962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07cc104410 0x7f07cc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 --2- 172.21.15.3:0/235708962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07cc105610 0x7f07cc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 -- 172.21.15.3:0/235708962 >> 172.21.15.3:0/235708962 conn(0x7f07cc0ff9b0 msgr2=0x7f07cc101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 -- 172.21.15.3:0/235708962 shutdown_connections 2024-03-21T08:30:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.698+0000 7f07d4600700 1 -- 172.21.15.3:0/235708962 wait complete. 2024-03-21T08:30:56.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.699+0000 7f07d4600700 1 Processor -- start 2024-03-21T08:30:56.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d4600700 1 -- start start 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d4600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07cc104410 0x7f07cc079f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d4600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105610 0x7f07cc07a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d4600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07cc105fb0 0x7f07cc076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d4600700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc10fe90 con 0x7f07cc105610 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d4600700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc10fd10 con 0x7f07cc105fb0 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d4600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc110010 con 0x7f07cc104410 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.700+0000 7f07d1200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105610 0x7f07cc07a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:56.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d2600700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07cc105fb0 0x7f07cc076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105610 0x7f07cc07a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41358/0 (socket says 172.21.15.3:41358) 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1200700 1 -- 172.21.15.3:0/4052365197 learned_addr learned my addr 172.21.15.3:0/4052365197 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1c00700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07cc104410 0x7f07cc079f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1200700 1 -- 172.21.15.3:0/4052365197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07cc104410 msgr2=0x7f07cc079f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1200700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07cc104410 0x7f07cc079f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1200700 1 -- 172.21.15.3:0/4052365197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07cc105fb0 msgr2=0x7f07cc076ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1200700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07cc105fb0 0x7f07cc076ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d1200700 1 -- 172.21.15.3:0/4052365197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc077390 con 0x7f07cc105610 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.701+0000 7f07d2600700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07cc105fb0 0x7f07cc076ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:30:56.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.702+0000 7f07d1200700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105610 0x7f07cc07a480 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f07b800d950 tx=0x7f07b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.702+0000 7f07ca000700 1 -- 172.21.15.3:0/4052365197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8016070 con 0x7f07cc105610 2024-03-21T08:30:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.702+0000 7f07ca000700 1 -- 172.21.15.3:0/4052365197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f07b800fd40 con 0x7f07cc105610 2024-03-21T08:30:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.702+0000 7f07ca000700 1 -- 172.21.15.3:0/4052365197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800e3b0 con 0x7f07cc105610 2024-03-21T08:30:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.702+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc1a68b0 con 0x7f07cc105610 2024-03-21T08:30:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.702+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc1a6cb0 con 0x7f07cc105610 2024-03-21T08:30:56.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.704+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc068e90 con 0x7f07cc105610 2024-03-21T08:30:56.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.705+0000 7f07ca000700 1 -- 172.21.15.3:0/4052365197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f07b8019070 con 0x7f07cc105610 2024-03-21T08:30:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.707+0000 7f07ca000700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07b40a3030 0x7f07b40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:30:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.707+0000 7f07ca000700 1 -- 172.21.15.3:0/4052365197 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f07b80cf2a0 con 0x7f07cc105610 2024-03-21T08:30:56.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.708+0000 7f07d1c00700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07b40a3030 0x7f07b40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:30:56.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.708+0000 7f07d1c00700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07b40a3030 0x7f07b40a54f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07cc105470 tx=0x7f07c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:30:56.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.712+0000 7f07ca000700 1 -- 172.21.15.3:0/4052365197 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f07b809bc30 con 0x7f07cc105610 2024-03-21T08:30:56.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.954+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f07cc0625d0 con 0x7f07b40a3030 2024-03-21T08:30:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.970+0000 7f07ca000700 1 -- 172.21.15.3:0/4052365197 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f07cc0625d0 con 0x7f07b40a3030 2024-03-21T08:30:56.972 INFO:teuthology.orchestra.run.smithi003.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-21T08:30:56.972 INFO:teuthology.orchestra.run.smithi003.stdout:mds.all 12/12 73s ago 3m smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-21T08:30:56.972 INFO:teuthology.orchestra.run.smithi003.stdout:mgr 3/3 73s ago 10m smithi003=x;smithi084=y;smithi171=z;count:3 2024-03-21T08:30:56.972 INFO:teuthology.orchestra.run.smithi003.stdout:mon 3/3 73s ago 10m smithi003:172.21.15.3=a;smithi084:172.21.15.84=b;smithi171:172.21.15.171=c;count:3 2024-03-21T08:30:56.972 INFO:teuthology.orchestra.run.smithi003.stdout:osd 12 73s ago - 2024-03-21T08:30:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07b40a3030 msgr2=0x7f07b40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07b40a3030 0x7f07b40a54f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07cc105470 tx=0x7f07c0009260 comp rx=0 tx=0).stop 2024-03-21T08:30:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105610 msgr2=0x7f07cc07a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:30:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105610 0x7f07cc07a480 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f07b800d950 tx=0x7f07b800dc60 comp rx=0 tx=0).stop 2024-03-21T08:30:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 shutdown_connections 2024-03-21T08:30:56.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f07b40a3030 0x7f07b40a54f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07cc105fb0 0x7f07cc076ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07cc104410 0x7f07cc079f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 --2- 172.21.15.3:0/4052365197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc105610 0x7f07cc07a480 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:30:56.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 >> 172.21.15.3:0/4052365197 conn(0x7f07cc0ff9b0 msgr2=0x7f07cc108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:30:56.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 shutdown_connections 2024-03-21T08:30:56.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:30:56.973+0000 7f07d4600700 1 -- 172.21.15.3:0/4052365197 wait complete. 2024-03-21T08:30:57.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:57 smithi084 ceph-mon[35477]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:57.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:57 smithi171 ceph-mon[35388]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:57.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:57 smithi003 ceph-mon[31675]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:30:58.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-21T08:30:58.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:58 smithi084 ceph-mon[35477]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:58.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:58 smithi171 ceph-mon[35388]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:58.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:58 smithi003 ceph-mon[31675]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:30:58.515 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:30:59.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:30:59 smithi084 ceph-mon[35477]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:59.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:30:59 smithi171 ceph-mon[35388]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:30:59.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:30:59 smithi003 ceph-mon[31675]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:00.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 -- 172.21.15.3:0/1248181693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b4103d60 msgr2=0x7fa2b410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1248181693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b4103d60 0x7fa2b410a610 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00b0d0 tx=0x7fa2ac00b3e0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 -- 172.21.15.3:0/1248181693 shutdown_connections 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1248181693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b4103d60 0x7fa2b410a610 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1248181693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2b41021c0 0x7fa2b41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1248181693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b41033c0 0x7fa2b4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 -- 172.21.15.3:0/1248181693 >> 172.21.15.3:0/1248181693 conn(0x7fa2b40fd7a0 msgr2=0x7fa2b40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 -- 172.21.15.3:0/1248181693 shutdown_connections 2024-03-21T08:31:00.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.637+0000 7fa2bc200700 1 -- 172.21.15.3:0/1248181693 wait complete. 2024-03-21T08:31:00.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 Processor -- start 2024-03-21T08:31:00.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 -- start start 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2b41021c0 0x7fa2b4075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b41033c0 0x7fa2b4074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b4103d60 0x7fa2b4074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b410dc70 con 0x7fa2b41033c0 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b410daf0 con 0x7fa2b41021c0 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.639+0000 7fa2bc200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b410ddf0 con 0x7fa2b4103d60 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b41033c0 0x7fa2b4074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b9800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2b41021c0 0x7fa2b4075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b41033c0 0x7fa2b4074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43796/0 (socket says 172.21.15.3:43796) 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 -- 172.21.15.3:0/1639765086 learned_addr learned my addr 172.21.15.3:0/1639765086 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b9800700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2b41021c0 0x7fa2b4075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:42066/0 (socket says 172.21.15.3:42066) 2024-03-21T08:31:00.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2ba200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b4103d60 0x7fa2b4074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:00.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 -- 172.21.15.3:0/1639765086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b4103d60 msgr2=0x7fa2b4074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:00.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b4103d60 0x7fa2b4074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 -- 172.21.15.3:0/1639765086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2b41021c0 msgr2=0x7fa2b4075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2b41021c0 0x7fa2b4075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2b8e00700 1 -- 172.21.15.3:0/1639765086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4075370 con 0x7fa2b41033c0 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.640+0000 7fa2ba200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b4103d60 0x7fa2b4074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.641+0000 7fa2b8e00700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b41033c0 0x7fa2b4074580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000d980 tx=0x7fa2a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.641+0000 7fa2b2000700 1 -- 172.21.15.3:0/1639765086 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0017070 con 0x7fa2b41033c0 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.641+0000 7fa2b2000700 1 -- 172.21.15.3:0/1639765086 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2a0005020 con 0x7fa2b41033c0 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.641+0000 7fa2b2000700 1 -- 172.21.15.3:0/1639765086 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0009b50 con 0x7fa2b41033c0 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.641+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b41a2440 con 0x7fa2b41033c0 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.641+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b41a2900 con 0x7fa2b41033c0 2024-03-21T08:31:00.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.643+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b4068e50 con 0x7fa2b41033c0 2024-03-21T08:31:00.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.645+0000 7fa2b2000700 1 -- 172.21.15.3:0/1639765086 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fa2a0009cf0 con 0x7fa2b41033c0 2024-03-21T08:31:00.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.646+0000 7fa2b2000700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa29c0a3030 0x7fa29c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:00.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.646+0000 7fa2b2000700 1 -- 172.21.15.3:0/1639765086 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa2a00cf670 con 0x7fa2b41033c0 2024-03-21T08:31:00.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.646+0000 7fa2b9800700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa29c0a3030 0x7fa29c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:00.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.647+0000 7fa2b9800700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa29c0a3030 0x7fa29c0a54f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4103220 tx=0x7fa2a8007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:00.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.651+0000 7fa2b2000700 1 -- 172.21.15.3:0/1639765086 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2a0020020 con 0x7fa2b41033c0 2024-03-21T08:31:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.889+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2b40625d0 con 0x7fa29c0a3030 2024-03-21T08:31:00.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.891+0000 7fa2b2000700 1 -- 172.21.15.3:0/1639765086 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fa2b40625d0 con 0x7fa29c0a3030 2024-03-21T08:31:00.892 INFO:teuthology.orchestra.run.smithi003.stdout:HOST ADDR LABELS STATUS 2024-03-21T08:31:00.893 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 172.21.15.3 2024-03-21T08:31:00.893 INFO:teuthology.orchestra.run.smithi003.stdout:smithi084 172.21.15.84 2024-03-21T08:31:00.893 INFO:teuthology.orchestra.run.smithi003.stdout:smithi171 172.21.15.171 2024-03-21T08:31:00.893 INFO:teuthology.orchestra.run.smithi003.stdout:3 hosts in cluster 2024-03-21T08:31:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa29c0a3030 msgr2=0x7fa29c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa29c0a3030 0x7fa29c0a54f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4103220 tx=0x7fa2a8007400 comp rx=0 tx=0).stop 2024-03-21T08:31:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b41033c0 msgr2=0x7fa2b4074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b41033c0 0x7fa2b4074580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000d980 tx=0x7fa2a000dc90 comp rx=0 tx=0).stop 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 shutdown_connections 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa29c0a3030 0x7fa29c0a54f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2b4103d60 0x7fa2b4074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2b41021c0 0x7fa2b4075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 --2- 172.21.15.3:0/1639765086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b41033c0 0x7fa2b4074580 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 >> 172.21.15.3:0/1639765086 conn(0x7fa2b40fd7a0 msgr2=0x7fa2b4106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.894+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 shutdown_connections 2024-03-21T08:31:00.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:00.895+0000 7fa2bc200700 1 -- 172.21.15.3:0/1639765086 wait complete. 2024-03-21T08:31:01.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:01 smithi003 ceph-mon[31675]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:01.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:01 smithi084 ceph-mon[35477]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:01.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:01 smithi171 ceph-mon[35388]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:01.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-21T08:31:02.298 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:31:02.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:02 smithi084 ceph-mon[35477]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:02.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:02 smithi171 ceph-mon[35388]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:02.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:02 smithi003 ceph-mon[31675]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:03.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:03 smithi084 ceph-mon[35477]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:03.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:03 smithi171 ceph-mon[35388]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:03.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:03 smithi003 ceph-mon[31675]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:04.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.630+0000 7f6a47e00700 1 -- 172.21.15.3:0/2993042066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 msgr2=0x7f6a48078fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:04.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.630+0000 7f6a47e00700 1 --2- 172.21.15.3:0/2993042066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a48078fd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009990 tx=0x7f6a40009ca0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.631+0000 7f6a47e00700 1 -- 172.21.15.3:0/2993042066 shutdown_connections 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.631+0000 7f6a47e00700 1 --2- 172.21.15.3:0/2993042066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a48078fd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.631+0000 7f6a47e00700 1 --2- 172.21.15.3:0/2993042066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a48076b40 0x7f6a48076f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.631+0000 7f6a47e00700 1 --2- 172.21.15.3:0/2993042066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a48077d90 0x7f6a48078210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.631+0000 7f6a47e00700 1 -- 172.21.15.3:0/2993042066 >> 172.21.15.3:0/2993042066 conn(0x7f6a480ff880 msgr2=0x7f6a48101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.631+0000 7f6a47e00700 1 -- 172.21.15.3:0/2993042066 shutdown_connections 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.631+0000 7f6a47e00700 1 -- 172.21.15.3:0/2993042066 wait complete. 2024-03-21T08:31:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.632+0000 7f6a47e00700 1 Processor -- start 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.632+0000 7f6a47e00700 1 -- start start 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47e00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a48076b40 0x7f6a4819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a48077d90 0x7f6a4819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47e00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a481a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47e00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481141d0 con 0x7f6a48078750 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47e00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48114050 con 0x7f6a48076b40 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48114350 con 0x7f6a48077d90 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a481a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:04.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a46a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a48076b40 0x7f6a4819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a481a5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43820/0 (socket says 172.21.15.3:43820) 2024-03-21T08:31:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a47400700 1 -- 172.21.15.3:0/1145752863 learned_addr learned my addr 172.21.15.3:0/1145752863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:04.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.633+0000 7f6a3e000700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a48077d90 0x7f6a4819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.634+0000 7f6a47400700 1 -- 172.21.15.3:0/1145752863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a48077d90 msgr2=0x7f6a4819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.634+0000 7f6a47400700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a48077d90 0x7f6a4819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.634+0000 7f6a47400700 1 -- 172.21.15.3:0/1145752863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a48076b40 msgr2=0x7f6a4819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.634+0000 7f6a47400700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a48076b40 0x7f6a4819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.634+0000 7f6a47400700 1 -- 172.21.15.3:0/1145752863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a481a63d0 con 0x7f6a48078750 2024-03-21T08:31:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.634+0000 7f6a47400700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a481a5d30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005910 tx=0x7f6a400057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:04.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.635+0000 7f6a3fe00700 1 -- 172.21.15.3:0/1145752863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40027070 con 0x7f6a48078750 2024-03-21T08:31:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.635+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a481a6660 con 0x7f6a48078750 2024-03-21T08:31:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.635+0000 7f6a3fe00700 1 -- 172.21.15.3:0/1145752863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a40004150 con 0x7f6a48078750 2024-03-21T08:31:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.635+0000 7f6a3fe00700 1 -- 172.21.15.3:0/1145752863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000f460 con 0x7f6a48078750 2024-03-21T08:31:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.635+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a481a6b10 con 0x7f6a48078750 2024-03-21T08:31:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.636+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a48068e50 con 0x7f6a48078750 2024-03-21T08:31:04.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.638+0000 7f6a3fe00700 1 -- 172.21.15.3:0/1145752863 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f6a4002a070 con 0x7f6a48078750 2024-03-21T08:31:04.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.639+0000 7f6a3fe00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f6a2c0a2fb0 0x7f6a2c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:04.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.640+0000 7f6a3fe00700 1 -- 172.21.15.3:0/1145752863 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f6a400d07c0 con 0x7f6a48078750 2024-03-21T08:31:04.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.640+0000 7f6a46a00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f6a2c0a2fb0 0x7f6a2c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:04.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.641+0000 7f6a46a00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f6a2c0a2fb0 0x7f6a2c0a5470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6a38003f90 tx=0x7f6a38009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:04.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.644+0000 7f6a3fe00700 1 -- 172.21.15.3:0/1145752863 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a4005e740 con 0x7f6a48078750 2024-03-21T08:31:04.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.885+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a480625d0 con 0x7f6a2c0a2fb0 2024-03-21T08:31:04.893 INFO:teuthology.orchestra.run.smithi003.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-21T08:31:04.893 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-21T08:31:04.893 INFO:teuthology.orchestra.run.smithi003.stdout:smithi084 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WF400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-21T08:31:04.893 INFO:teuthology.orchestra.run.smithi003.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-21T08:31:04.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.891+0000 7f6a3fe00700 1 -- 172.21.15.3:0/1145752863 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f6a480625d0 con 0x7f6a2c0a2fb0 2024-03-21T08:31:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f6a2c0a2fb0 msgr2=0x7f6a2c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f6a2c0a2fb0 0x7f6a2c0a5470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6a38003f90 tx=0x7f6a38009180 comp rx=0 tx=0).stop 2024-03-21T08:31:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 msgr2=0x7f6a481a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a481a5d30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005910 tx=0x7f6a400057c0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 shutdown_connections 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f6a2c0a2fb0 0x7f6a2c0a5470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a48078750 0x7f6a481a5d30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a48076b40 0x7f6a4819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 --2- 172.21.15.3:0/1145752863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a48077d90 0x7f6a4819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 >> 172.21.15.3:0/1145752863 conn(0x7f6a480ff880 msgr2=0x7f6a48101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 shutdown_connections 2024-03-21T08:31:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:04.895+0000 7f6a47e00700 1 -- 172.21.15.3:0/1145752863 wait complete. 2024-03-21T08:31:05.181 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:05 smithi003 ceph-mon[31675]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:05.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:05 smithi084 ceph-mon[35477]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:05.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:05 smithi171 ceph-mon[35388]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:05.827 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T08:31:05.855 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:31:05.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T08:31:06.345 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:31:06.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:06 smithi084 ceph-mon[35477]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:06.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:06 smithi171 ceph-mon[35388]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:06.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:06 smithi003 ceph-mon[31675]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:07.361 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:07 smithi003 ceph-mon[31675]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:07.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:07 smithi084 ceph-mon[35477]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:07.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:07 smithi171 ceph-mon[35388]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.675+0000 7f52f4a00700 1 -- 172.21.15.3:0/239295546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f01059f0 msgr2=0x7f52f0105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.675+0000 7f52f4a00700 1 --2- 172.21.15.3:0/239295546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f01059f0 0x7f52f0105e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009990 tx=0x7f52e0009ca0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.675+0000 7f52f4a00700 1 -- 172.21.15.3:0/239295546 shutdown_connections 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.675+0000 7f52f4a00700 1 --2- 172.21.15.3:0/239295546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f01059f0 0x7f52f0105e10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.675+0000 7f52f4a00700 1 --2- 172.21.15.3:0/239295546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f01048d0 0x7f52f010c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.675+0000 7f52f4a00700 1 --2- 172.21.15.3:0/239295546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f0103f30 0x7f52f0104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.675+0000 7f52f4a00700 1 -- 172.21.15.3:0/239295546 >> 172.21.15.3:0/239295546 conn(0x7f52f00ff8b0 msgr2=0x7f52f0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.676+0000 7f52f4a00700 1 -- 172.21.15.3:0/239295546 shutdown_connections 2024-03-21T08:31:08.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.676+0000 7f52f4a00700 1 -- 172.21.15.3:0/239295546 wait complete. 2024-03-21T08:31:08.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.677+0000 7f52f4a00700 1 Processor -- start 2024-03-21T08:31:08.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.677+0000 7f52f4a00700 1 -- start start 2024-03-21T08:31:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52f4a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f0103f30 0x7f52f019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52f4a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f01048d0 0x7f52f019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52f4a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f01059f0 0x7f52f01a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52ee800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f0103f30 0x7f52f019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52f4a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f010ff00 con 0x7f52f0103f30 2024-03-21T08:31:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52f4a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f010fd80 con 0x7f52f01059f0 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52f4a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f0110080 con 0x7f52f01048d0 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52ee800700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f0103f30 0x7f52f019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43834/0 (socket says 172.21.15.3:43834) 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52ee800700 1 -- 172.21.15.3:0/2818136798 learned_addr learned my addr 172.21.15.3:0/2818136798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.678+0000 7f52ef200700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f01059f0 0x7f52f01a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.679+0000 7f52ee800700 1 -- 172.21.15.3:0/2818136798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f01048d0 msgr2=0x7f52f019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.679+0000 7f52ee800700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f01048d0 0x7f52f019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.679+0000 7f52ee800700 1 -- 172.21.15.3:0/2818136798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f01059f0 msgr2=0x7f52f01a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.679+0000 7f52ee800700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f01059f0 0x7f52f01a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.679+0000 7f52ee800700 1 -- 172.21.15.3:0/2818136798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f01a2180 con 0x7f52f0103f30 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.679+0000 7f52ef200700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f01059f0 0x7f52f01a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.679+0000 7f52ee800700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f0103f30 0x7f52f019a570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f52e00037f0 tx=0x7f52e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.680+0000 7f52e7400700 1 -- 172.21.15.3:0/2818136798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0003c90 con 0x7f52f0103f30 2024-03-21T08:31:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.680+0000 7f52e7400700 1 -- 172.21.15.3:0/2818136798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f52e0004830 con 0x7f52f0103f30 2024-03-21T08:31:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.680+0000 7f52e7400700 1 -- 172.21.15.3:0/2818136798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00054e0 con 0x7f52f0103f30 2024-03-21T08:31:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.680+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f01a2410 con 0x7f52f0103f30 2024-03-21T08:31:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.680+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f01a27b0 con 0x7f52f0103f30 2024-03-21T08:31:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.681+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f0068e50 con 0x7f52f0103f30 2024-03-21T08:31:08.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.685+0000 7f52e7400700 1 -- 172.21.15.3:0/2818136798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f52e00049e0 con 0x7f52f0103f30 2024-03-21T08:31:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.686+0000 7f52e7400700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f52d40a3030 0x7f52d40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.687+0000 7f52e7400700 1 -- 172.21.15.3:0/2818136798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f52e00d0d30 con 0x7f52f0103f30 2024-03-21T08:31:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.687+0000 7f52ede00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f52d40a3030 0x7f52d40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.687+0000 7f52ede00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f52d40a3030 0x7f52d40a54f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f52dc005950 tx=0x7f52dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:08.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.690+0000 7f52e7400700 1 -- 172.21.15.3:0/2818136798 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52e0013070 con 0x7f52f0103f30 2024-03-21T08:31:08.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.969+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f52f002de60 con 0x7f52f0103f30 2024-03-21T08:31:08.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.970+0000 7f52e7400700 1 -- 172.21.15.3:0/2818136798 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2990 (secure 0 0 0) 0x7f52e009d750 con 0x7f52f0103f30 2024-03-21T08:31:08.972 INFO:teuthology.orchestra.run.smithi003.stdout:e22 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:legacy client fscid: 1 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:Filesystem 'cephfs' (1) 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:fs_name cephfs 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 22 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-03-21T08:28:36.113879+0000 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-03-21T08:29:09.323523+0000 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:tableserver 0 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:root 0 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:session_timeout 300 2024-03-21T08:31:08.973 INFO:teuthology.orchestra.run.smithi003.stdout:session_autoclose 300 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:max_file_size 1099511627776 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:max_xattr_size 65536 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:required_client_features {} 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure 0 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure_osd_epoch 0 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:max_mds 3 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:in 0,1,2 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:up {0=34472,1=34439,2=34409} 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:failed 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:damaged 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:stopped 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:data_pools [3,4] 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:metadata_pool 2 2024-03-21T08:31:08.974 INFO:teuthology.orchestra.run.smithi003.stdout:inline_data disabled 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:balancer 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:bal_rank_mask -1 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:standby_count_wanted 0 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:qdb_cluster leader: 34472 members: 34439,34409,34472 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.h{0:34472} state up:active seq 7 addr [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.i{0:24487} state up:standby-replay seq 1 addr [v2:172.21.15.171:6838/1010033316,v1:172.21.15.171:6839/1010033316] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.e{1:34439} state up:active seq 17 addr [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c{1:24469} state up:standby-replay seq 1 addr [v2:172.21.15.171:6836/2662792128,v1:172.21.15.171:6837/2662792128] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.k{2:34409} state up:active seq 21 addr [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.f{2:24454} state up:standby-replay seq 1 addr [v2:172.21.15.171:6834/1859190507,v1:172.21.15.171:6835/1859190507] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:Standby daemons: 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:08.975 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.3:6834/3089491924,v1:172.21.15.3:6835/3089491924] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.976 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.3:6836/4111852519,v1:172.21.15.3:6837/4111852519] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.976 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.g{-1:14640} state up:standby seq 1 addr [v2:172.21.15.3:6838/3720466130,v1:172.21.15.3:6839/3720466130] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.976 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a{-1:14655} state up:standby seq 1 addr [v2:172.21.15.3:6840/2348469157,v1:172.21.15.3:6841/2348469157] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.976 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b{-1:14673} state up:standby seq 1 addr [v2:172.21.15.84:6838/1820502310,v1:172.21.15.84:6839/1820502310] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.976 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.l{-1:24415} state up:standby seq 1 addr [v2:172.21.15.171:6832/3477321197,v1:172.21.15.171:6833/3477321197] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f52d40a3030 msgr2=0x7f52d40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f52d40a3030 0x7f52d40a54f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f52dc005950 tx=0x7f52dc0058e0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f0103f30 msgr2=0x7f52f019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f0103f30 0x7f52f019a570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f52e00037f0 tx=0x7f52e00047b0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 shutdown_connections 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f52d40a3030 0x7f52d40a54f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f0103f30 0x7f52f019a570 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f01059f0 0x7f52f01a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 --2- 172.21.15.3:0/2818136798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f01048d0 0x7f52f019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.975+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 >> 172.21.15.3:0/2818136798 conn(0x7f52f00ff8b0 msgr2=0x7f52f01088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.976+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 shutdown_connections 2024-03-21T08:31:08.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:08.976+0000 7f52f4a00700 1 -- 172.21.15.3:0/2818136798 wait complete. 2024-03-21T08:31:08.978 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-03-21T08:31:09.363 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:09 smithi003 ceph-mon[31675]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:09.363 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:09 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2818136798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T08:31:09.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:09 smithi084 ceph-mon[35477]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:09.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:09 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2818136798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T08:31:09.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:09 smithi171 ceph-mon[35388]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:09.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:09 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2818136798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T08:31:09.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-21T08:31:10.340 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:31:11.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:11 smithi084 ceph-mon[35477]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:11.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:11 smithi003 ceph-mon[31675]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:11.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:11 smithi171 ceph-mon[35388]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:12.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 -- 172.21.15.3:0/664438462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 msgr2=0x7f908c1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 --2- 172.21.15.3:0/664438462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c1025f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f907c0099c0 tx=0x7f907c009cd0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 -- 172.21.15.3:0/664438462 shutdown_connections 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 --2- 172.21.15.3:0/664438462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c1025f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 --2- 172.21.15.3:0/664438462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c103d70 0x7f908c10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 --2- 172.21.15.3:0/664438462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f908c1033d0 0x7f908c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 -- 172.21.15.3:0/664438462 >> 172.21.15.3:0/664438462 conn(0x7f908c0fd730 msgr2=0x7f908c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 -- 172.21.15.3:0/664438462 shutdown_connections 2024-03-21T08:31:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.559+0000 7f9091a00700 1 -- 172.21.15.3:0/664438462 wait complete. 2024-03-21T08:31:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.561+0000 7f9091a00700 1 Processor -- start 2024-03-21T08:31:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.561+0000 7f9091a00700 1 -- start start 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.561+0000 7f9091a00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.561+0000 7f9091a00700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c1033d0 0x7f908c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f9091a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f908c103d70 0x7f908c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f9091a00700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c10dcb0 con 0x7f908c1021d0 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f9091a00700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c10db30 con 0x7f908c1033d0 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f9091a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c10de30 con 0x7f908c103d70 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52310/0 (socket says 172.21.15.3:52310) 2024-03-21T08:31:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 -- 172.21.15.3:0/2867777912 learned_addr learned my addr 172.21.15.3:0/2867777912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908a000700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c1033d0 0x7f908c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908b400700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f908c103d70 0x7f908c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 -- 172.21.15.3:0/2867777912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f908c103d70 msgr2=0x7f908c074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f908c103d70 0x7f908c074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 -- 172.21.15.3:0/2867777912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c1033d0 msgr2=0x7f908c074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c1033d0 0x7f908c074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.562+0000 7f908aa00700 1 -- 172.21.15.3:0/2867777912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c075370 con 0x7f908c1021d0 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.563+0000 7f908b400700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f908c103d70 0x7f908c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.563+0000 7f908aa00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c075f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f907c0057c0 tx=0x7f907c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.563+0000 7f9083400700 1 -- 172.21.15.3:0/2867777912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c026070 con 0x7f908c1021d0 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.563+0000 7f9083400700 1 -- 172.21.15.3:0/2867777912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f907c004110 con 0x7f908c1021d0 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.563+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c1a24e0 con 0x7f908c1021d0 2024-03-21T08:31:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.563+0000 7f9083400700 1 -- 172.21.15.3:0/2867777912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00fda0 con 0x7f908c1021d0 2024-03-21T08:31:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.564+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c1a2930 con 0x7f908c1021d0 2024-03-21T08:31:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.565+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c068e50 con 0x7f908c1021d0 2024-03-21T08:31:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.567+0000 7f9083400700 1 -- 172.21.15.3:0/2867777912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f907c029070 con 0x7f908c1021d0 2024-03-21T08:31:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.568+0000 7f9083400700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f90700a2fb0 0x7f90700a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:12.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.568+0000 7f9083400700 1 -- 172.21.15.3:0/2867777912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f907c0d0430 con 0x7f908c1021d0 2024-03-21T08:31:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.569+0000 7f908a000700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f90700a2fb0 0x7f90700a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:12.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.569+0000 7f908a000700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f90700a2fb0 0x7f90700a5470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9078005950 tx=0x7f90780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:12.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.573+0000 7f9083400700 1 -- 172.21.15.3:0/2867777912 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f907c05d740 con 0x7f908c1021d0 2024-03-21T08:31:12.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.806+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f908c062da0 con 0x7f908c1021d0 2024-03-21T08:31:12.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.808+0000 7f9083400700 1 -- 172.21.15.3:0/2867777912 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v96) v1 ==== 56+0+5771 (secure 0 0 0) 0x7f907c013070 con 0x7f908c1021d0 2024-03-21T08:31:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 96 2024-03-21T08:31:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:31:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-03-21T08:17:50.596819+0000 2024-03-21T08:31:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-03-21T08:30:06.635311+0000 2024-03-21T08:31:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-21T08:31:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:crush_version 27 2024-03-21T08:31:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:full_ratio 0.95 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:backfillfull_ratio 0.9 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:nearfull_ratio 0.85 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:require_min_compat_client luminous 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:min_compat_client luminous 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:require_osd_release squid 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:stretch_mode_enabled false 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:max_osd 12 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6802/3803233195,v1:172.21.15.3:6803/3803233195] [v2:172.21.15.3:6804/3803233195,v1:172.21.15.3:6805/3803233195] exists,up 94762ab2-9404-4097-911f-0f6a15f4788a 2024-03-21T08:31:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 up in weight 1 up_from 13 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6800/2209911029,v1:172.21.15.84:6801/2209911029] [v2:172.21.15.84:6802/2209911029,v1:172.21.15.84:6803/2209911029] exists,up 4ae4ad7a-eb28-42c8-a31a-26241b629dae 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/2357635272,v1:172.21.15.171:6801/2357635272] [v2:172.21.15.171:6802/2357635272,v1:172.21.15.171:6803/2357635272] exists,up 72b1933d-fa5d-423e-9711-162d319d2c10 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 up in weight 1 up_from 26 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6810/674164001,v1:172.21.15.3:6811/674164001] [v2:172.21.15.3:6812/674164001,v1:172.21.15.3:6813/674164001] exists,up 097d03f7-da4c-4c3e-b98d-1da6de58611f 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 up in weight 1 up_from 34 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6808/2287981020,v1:172.21.15.84:6809/2287981020] [v2:172.21.15.84:6810/2287981020,v1:172.21.15.84:6811/2287981020] exists,up 73fe75be-a893-4d19-b3ec-9d75c169f300 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 up in weight 1 up_from 39 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/246158153,v1:172.21.15.171:6809/246158153] [v2:172.21.15.171:6810/246158153,v1:172.21.15.171:6811/246158153] exists,up 51c9f7b7-be7e-46db-b99c-ea76625620dc 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 up in weight 1 up_from 47 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6818/3180742022,v1:172.21.15.3:6819/3180742022] [v2:172.21.15.3:6820/3180742022,v1:172.21.15.3:6821/3180742022] exists,up 68131baf-10d4-4e02-a3cc-b7ec95fe3708 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 up in weight 1 up_from 53 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6816/3012309737,v1:172.21.15.84:6817/3012309737] [v2:172.21.15.84:6818/3012309737,v1:172.21.15.84:6819/3012309737] exists,up 06b9b8c1-ca04-4499-96d1-79e0e0f316d7 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 up in weight 1 up_from 62 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/1162829385,v1:172.21.15.171:6817/1162829385] [v2:172.21.15.171:6818/1162829385,v1:172.21.15.171:6819/1162829385] exists,up e75bbebb-99e2-47c0-8e5a-0842a58c1092 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 up in weight 1 up_from 70 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6826/1803156787,v1:172.21.15.3:6827/1803156787] [v2:172.21.15.3:6828/1803156787,v1:172.21.15.3:6829/1803156787] exists,up 5bad569e-33cd-4ec5-a11f-66e7b60cc9ae 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 up in weight 1 up_from 75 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6824/3984184621,v1:172.21.15.84:6825/3984184621] [v2:172.21.15.84:6826/3984184621,v1:172.21.15.84:6827/3984184621] exists,up 960b5226-4fe7-42b0-bba6-193c2425e0de 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 up in weight 1 up_from 83 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/2511529180,v1:172.21.15.171:6825/2511529180] [v2:172.21.15.171:6826/2511529180,v1:172.21.15.171:6827/2511529180] exists,up 553edf28-7b5a-48cf-8ab4-c003e219e1b0 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.0 [7,11] 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.8 [2,11] 2024-03-21T08:31:12.812 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.e [2,4] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.f [5,6] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.1a [8,1] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.1d [2,4] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.1f [8,1] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.14 [4,6] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.1a [5,7] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.24 [4,9] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.2a [3,8] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.33 [5,9] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.36 [4,9] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.3a [4,8] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.3 [5,8] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.b [9,8] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.1c [9,8] 2024-03-21T08:31:12.813 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.1f [9,10] 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2117697938 expires 2024-03-22T08:19:05.157913+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2293101955 expires 2024-03-22T08:19:05.157913+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/2168061035 expires 2024-03-22T08:19:05.157913+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/2168061035 expires 2024-03-22T08:19:05.157913+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/1378601347 expires 2024-03-22T08:18:30.424775+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/1378601347 expires 2024-03-22T08:18:30.424775+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/404577225 expires 2024-03-22T08:18:30.424775+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1597279144 expires 2024-03-22T08:18:30.424775+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/4112229601 expires 2024-03-22T08:18:30.424775+0000 2024-03-21T08:31:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2051354447 expires 2024-03-22T08:19:05.157913+0000 2024-03-21T08:31:12.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f90700a2fb0 msgr2=0x7f90700a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:12.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f90700a2fb0 0x7f90700a5470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9078005950 tx=0x7f90780058e0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 msgr2=0x7f908c075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:12.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c075f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f907c0057c0 tx=0x7f907c009580 comp rx=0 tx=0).stop 2024-03-21T08:31:12.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 shutdown_connections 2024-03-21T08:31:12.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f90700a2fb0 0x7f90700a5470 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f908c1021d0 0x7f908c075f30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f908c103d70 0x7f908c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 --2- 172.21.15.3:0/2867777912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908c1033d0 0x7f908c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:12.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 >> 172.21.15.3:0/2867777912 conn(0x7f908c0fd730 msgr2=0x7f908c1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:12.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 shutdown_connections 2024-03-21T08:31:12.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:12.811+0000 7f9091a00700 1 -- 172.21.15.3:0/2867777912 wait complete. 2024-03-21T08:31:13.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:13 smithi171 ceph-mon[35388]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:13 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2867777912' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T08:31:13.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:13 smithi003 ceph-mon[31675]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:13.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:13 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2867777912' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T08:31:13.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:13 smithi084 ceph-mon[35477]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:13.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:13 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2867777912' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T08:31:13.899 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-21T08:31:13.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:31:14.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 -- 172.21.15.3:0/1011084379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1049b0 msgr2=0x7fb5fc104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 --2- 172.21.15.3:0/1011084379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1049b0 0x7fb5fc104e30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009900 tx=0x7fb5e802bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:14.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 -- 172.21.15.3:0/1011084379 shutdown_connections 2024-03-21T08:31:14.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 --2- 172.21.15.3:0/1011084379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5fc105370 0x7fb5fc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 --2- 172.21.15.3:0/1011084379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1049b0 0x7fb5fc104e30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 --2- 172.21.15.3:0/1011084379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5fc1037c0 0x7fb5fc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 -- 172.21.15.3:0/1011084379 >> 172.21.15.3:0/1011084379 conn(0x7fb5fc0fed50 msgr2=0x7fb5fc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:14.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 -- 172.21.15.3:0/1011084379 shutdown_connections 2024-03-21T08:31:14.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.146+0000 7fb602e00640 1 -- 172.21.15.3:0/1011084379 wait complete. 2024-03-21T08:31:14.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 Processor -- start 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 -- start start 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1037c0 0x7fb5fc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5fc1049b0 0x7fb5fc199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5fc105370 0x7fb5fc1a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc10f2f0 con 0x7fb5fc1037c0 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc10f170 con 0x7fb5fc1049b0 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.148+0000 7fb602e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc10f470 con 0x7fb5fc105370 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1037c0 0x7fb5fc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:14.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1037c0 0x7fb5fc199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52336/0 (socket says 172.21.15.3:52336) 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 -- 172.21.15.3:0/295903713 learned_addr learned my addr 172.21.15.3:0/295903713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601000640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5fc1049b0 0x7fb5fc199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb602400640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5fc105370 0x7fb5fc1a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 -- 172.21.15.3:0/295903713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5fc105370 msgr2=0x7fb5fc1a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5fc105370 0x7fb5fc1a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 -- 172.21.15.3:0/295903713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5fc1049b0 msgr2=0x7fb5fc199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5fc1049b0 0x7fb5fc199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 -- 172.21.15.3:0/295903713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc1a15b0 con 0x7fb5fc1037c0 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb602400640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5fc105370 0x7fb5fc1a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.149+0000 7fb601a00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1037c0 0x7fb5fc199990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000d770 tx=0x7fb5f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:14.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.150+0000 7fb5fa000640 1 -- 172.21.15.3:0/295903713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0017070 con 0x7fb5fc1037c0 2024-03-21T08:31:14.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.150+0000 7fb5fa000640 1 -- 172.21.15.3:0/295903713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb5f0009c30 con 0x7fb5fc1037c0 2024-03-21T08:31:14.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.150+0000 7fb5fa000640 1 -- 172.21.15.3:0/295903713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0003d00 con 0x7fb5fc1037c0 2024-03-21T08:31:14.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.150+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc1a18a0 con 0x7fb5fc1037c0 2024-03-21T08:31:14.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.150+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc1a1e40 con 0x7fb5fc1037c0 2024-03-21T08:31:14.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.151+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc10abb0 con 0x7fb5fc1037c0 2024-03-21T08:31:14.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.153+0000 7fb5fa000640 1 -- 172.21.15.3:0/295903713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fb5f0003ea0 con 0x7fb5fc1037c0 2024-03-21T08:31:14.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.155+0000 7fb5fa000640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb5e40a2f90 0x7fb5e40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.155+0000 7fb5fa000640 1 -- 172.21.15.3:0/295903713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fb5f00cf5e0 con 0x7fb5fc1037c0 2024-03-21T08:31:14.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.155+0000 7fb601000640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb5e40a2f90 0x7fb5e40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:14.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.156+0000 7fb601000640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb5e40a2f90 0x7fb5e40a5450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc19aea0 tx=0x7fb5e8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:14.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.159+0000 7fb5fa000640 1 -- 172.21.15.3:0/295903713 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb5f009bf70 con 0x7fb5fc1037c0 2024-03-21T08:31:14.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.366+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5fc02d320 con 0x7fb5fc1037c0 2024-03-21T08:31:14.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.369+0000 7fb5fa000640 1 -- 172.21.15.3:0/295903713 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7fb5f0009de0 con 0x7fb5fc1037c0 2024-03-21T08:31:14.371 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:14.371 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:31:14.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb5e40a2f90 msgr2=0x7fb5e40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb5e40a2f90 0x7fb5e40a5450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc19aea0 tx=0x7fb5e8005be0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1037c0 msgr2=0x7fb5fc199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1037c0 0x7fb5fc199990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000d770 tx=0x7fb5f000dc30 comp rx=0 tx=0).stop 2024-03-21T08:31:14.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 shutdown_connections 2024-03-21T08:31:14.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb5e40a2f90 0x7fb5e40a5450 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5fc105370 0x7fb5fc1a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5fc1049b0 0x7fb5fc199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 --2- 172.21.15.3:0/295903713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5fc1037c0 0x7fb5fc199990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.372+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 >> 172.21.15.3:0/295903713 conn(0x7fb5fc0fed50 msgr2=0x7fb5fc107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:14.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.373+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 shutdown_connections 2024-03-21T08:31:14.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.373+0000 7fb602e00640 1 -- 172.21.15.3:0/295903713 wait complete. 2024-03-21T08:31:14.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:31:14.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:31:14.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 -- 172.21.15.3:0/1048150475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3594074080 msgr2=0x7f35940744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 --2- 172.21.15.3:0/1048150475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3594074080 0x7f35940744e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3580009860 tx=0x7f358002bc00 comp rx=0 tx=0).stop 2024-03-21T08:31:14.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 -- 172.21.15.3:0/1048150475 shutdown_connections 2024-03-21T08:31:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 --2- 172.21.15.3:0/1048150475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35941093c0 0x7f359410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 --2- 172.21.15.3:0/1048150475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3594074080 0x7f35940744e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 --2- 172.21.15.3:0/1048150475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 0x7f3594073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 -- 172.21.15.3:0/1048150475 >> 172.21.15.3:0/1048150475 conn(0x7f35940fca80 msgr2=0x7f35940feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 -- 172.21.15.3:0/1048150475 shutdown_connections 2024-03-21T08:31:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.624+0000 7f3599a00640 1 -- 172.21.15.3:0/1048150475 wait complete. 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.625+0000 7f3599a00640 1 Processor -- start 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.625+0000 7f3599a00640 1 -- start start 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3599a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3594074080 0x7f35941998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3599a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 0x7f3594199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3599a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35941093c0 0x7f35941a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3599a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359410f200 con 0x7f3594075660 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3599a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359410f080 con 0x7f35941093c0 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3599a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359410f380 con 0x7f3594074080 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 0x7f3594199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:14.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 0x7f3594199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52346/0 (socket says 172.21.15.3:52346) 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 -- 172.21.15.3:0/3421425734 learned_addr learned my addr 172.21.15.3:0/3421425734 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3599000640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35941093c0 0x7f35941a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 -- 172.21.15.3:0/3421425734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3594074080 msgr2=0x7f35941998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3594074080 0x7f35941998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 -- 172.21.15.3:0/3421425734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35941093c0 msgr2=0x7f35941a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35941093c0 0x7f35941a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.626+0000 7f3593400640 1 -- 172.21.15.3:0/3421425734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35941a1510 con 0x7f3594075660 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.627+0000 7f3593400640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 0x7f3594199e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3580009980 tx=0x7f35800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.627+0000 7f3590c00640 1 -- 172.21.15.3:0/3421425734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580039050 con 0x7f3594075660 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.627+0000 7f3590c00640 1 -- 172.21.15.3:0/3421425734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3580045070 con 0x7f3594075660 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.627+0000 7f3590c00640 1 -- 172.21.15.3:0/3421425734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35800406b0 con 0x7f3594075660 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.627+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35941a17a0 con 0x7f3594075660 2024-03-21T08:31:14.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.627+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35941a1c90 con 0x7f3594075660 2024-03-21T08:31:14.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.628+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594108180 con 0x7f3594075660 2024-03-21T08:31:14.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.630+0000 7f3590c00640 1 -- 172.21.15.3:0/3421425734 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f35800035f0 con 0x7f3594075660 2024-03-21T08:31:14.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.632+0000 7f3590c00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35780a2f20 0x7f35780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:14.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.632+0000 7f3590c00640 1 -- 172.21.15.3:0/3421425734 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f35800f1770 con 0x7f3594075660 2024-03-21T08:31:14.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.632+0000 7f3593e00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35780a2f20 0x7f35780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:14.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.633+0000 7f3593e00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35780a2f20 0x7f35780a53e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35840058c0 tx=0x7f3584005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.636+0000 7f3590c00640 1 -- 172.21.15.3:0/3421425734 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35800be100 con 0x7f3594075660 2024-03-21T08:31:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.838+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3594062340 con 0x7f3594075660 2024-03-21T08:31:14.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.839+0000 7f3590c00640 1 -- 172.21.15.3:0/3421425734 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35800bda00 con 0x7f3594075660 2024-03-21T08:31:14.841 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35780a2f20 msgr2=0x7f35780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35780a2f20 0x7f35780a53e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35840058c0 tx=0x7f3584005850 comp rx=0 tx=0).stop 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 msgr2=0x7f3594199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 0x7f3594199e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3580009980 tx=0x7f35800042f0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 shutdown_connections 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35941093c0 0x7f35941a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3594075660 0x7f3594199e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f35780a2f20 0x7f35780a53e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 --2- 172.21.15.3:0/3421425734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3594074080 0x7f35941998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 >> 172.21.15.3:0/3421425734 conn(0x7f35940fca80 msgr2=0x7f3594105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 shutdown_connections 2024-03-21T08:31:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:14.842+0000 7f3599a00640 1 -- 172.21.15.3:0/3421425734 wait complete. 2024-03-21T08:31:14.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:31:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 -- 172.21.15.3:0/4260294130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364102130 msgr2=0x7fd364104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4260294130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364102130 0x7fd364104520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd354009ca0 tx=0x7fd35402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 -- 172.21.15.3:0/4260294130 shutdown_connections 2024-03-21T08:31:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4260294130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364107390 0x7fd3641097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4260294130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd364104a60 0x7fd364106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4260294130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364102130 0x7fd364104520 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 -- 172.21.15.3:0/4260294130 >> 172.21.15.3:0/4260294130 conn(0x7fd3640fbb60 msgr2=0x7fd3640fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 -- 172.21.15.3:0/4260294130 shutdown_connections 2024-03-21T08:31:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.093+0000 7fd36bc00640 1 -- 172.21.15.3:0/4260294130 wait complete. 2024-03-21T08:31:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.094+0000 7fd36bc00640 1 Processor -- start 2024-03-21T08:31:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.094+0000 7fd36bc00640 1 -- start start 2024-03-21T08:31:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd36bc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd364102130 0x7fd364101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd36bc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364104a60 0x7fd3641000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd36bc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364107390 0x7fd364100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd36bc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36410d200 con 0x7fd364107390 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd36bc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36410d080 con 0x7fd364104a60 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd36bc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36410d380 con 0x7fd364102130 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd369c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364107390 0x7fd364100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd35be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364104a60 0x7fd3641000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd369c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364107390 0x7fd364100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52362/0 (socket says 172.21.15.3:52362) 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd369c00640 1 -- 172.21.15.3:0/4246360619 learned_addr learned my addr 172.21.15.3:0/4246360619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.095+0000 7fd369200640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd364102130 0x7fd364101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd369c00640 1 -- 172.21.15.3:0/4246360619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd364102130 msgr2=0x7fd364101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd369c00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd364102130 0x7fd364101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd369c00640 1 -- 172.21.15.3:0/4246360619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364104a60 msgr2=0x7fd3641000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd369c00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364104a60 0x7fd3641000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd369c00640 1 -- 172.21.15.3:0/4246360619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364100b60 con 0x7fd364107390 2024-03-21T08:31:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd369200640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd364102130 0x7fd364101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd369c00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364107390 0x7fd364100620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e970 tx=0x7fd35c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd362000640 1 -- 172.21.15.3:0/4246360619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00ccc0 con 0x7fd364107390 2024-03-21T08:31:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd362000640 1 -- 172.21.15.3:0/4246360619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd35c00ce60 con 0x7fd364107390 2024-03-21T08:31:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd364100e50 con 0x7fd364107390 2024-03-21T08:31:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd362000640 1 -- 172.21.15.3:0/4246360619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c010430 con 0x7fd364107390 2024-03-21T08:31:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.096+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641a63d0 con 0x7fd364107390 2024-03-21T08:31:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.098+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3640fd650 con 0x7fd364107390 2024-03-21T08:31:15.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.100+0000 7fd362000640 1 -- 172.21.15.3:0/4246360619 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd35c004ff0 con 0x7fd364107390 2024-03-21T08:31:15.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.101+0000 7fd362000640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3480a2f20 0x7fd3480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.101+0000 7fd362000640 1 -- 172.21.15.3:0/4246360619 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd35c0cfcb0 con 0x7fd364107390 2024-03-21T08:31:15.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.101+0000 7fd369200640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3480a2f20 0x7fd3480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.102+0000 7fd369200640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3480a2f20 0x7fd3480a53e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd354037040 tx=0x7fd354000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:15.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.105+0000 7fd362000640 1 -- 172.21.15.3:0/4246360619 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd35c09c6a0 con 0x7fd364107390 2024-03-21T08:31:15.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.310+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3640622a0 con 0x7fd364107390 2024-03-21T08:31:15.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.311+0000 7fd362000640 1 -- 172.21.15.3:0/4246360619 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd35c09bfa0 con 0x7fd364107390 2024-03-21T08:31:15.312 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:15.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3480a2f20 msgr2=0x7fd3480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3480a2f20 0x7fd3480a53e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd354037040 tx=0x7fd354000f40 comp rx=0 tx=0).stop 2024-03-21T08:31:15.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364107390 msgr2=0x7fd364100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364107390 0x7fd364100620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e970 tx=0x7fd35c00ed70 comp rx=0 tx=0).stop 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 shutdown_connections 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364107390 0x7fd364100620 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd364104a60 0x7fd3641000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3480a2f20 0x7fd3480a53e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 --2- 172.21.15.3:0/4246360619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd364102130 0x7fd364101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 >> 172.21.15.3:0/4246360619 conn(0x7fd3640fbb60 msgr2=0x7fd364102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 shutdown_connections 2024-03-21T08:31:15.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.314+0000 7fd36bc00640 1 -- 172.21.15.3:0/4246360619 wait complete. 2024-03-21T08:31:15.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:31:15.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:15 smithi171 ceph-mon[35388]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:15.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:15 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/295903713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:15.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:15 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3421425734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:15.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:15 smithi003 ceph-mon[31675]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:15.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:15 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/295903713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:15.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:15 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3421425734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:15.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 -- 172.21.15.3:0/3360038284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 msgr2=0x7ffa80107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 --2- 172.21.15.3:0/3360038284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa80107f20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009900 tx=0x7ffa7002bc50 comp rx=0 tx=0).stop 2024-03-21T08:31:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 -- 172.21.15.3:0/3360038284 shutdown_connections 2024-03-21T08:31:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 --2- 172.21.15.3:0/3360038284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa801096b0 0x7ffa8010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 --2- 172.21.15.3:0/3360038284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa80108cf0 0x7ffa80109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 --2- 172.21.15.3:0/3360038284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa80107f20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 -- 172.21.15.3:0/3360038284 >> 172.21.15.3:0/3360038284 conn(0x7ffa80076a90 msgr2=0x7ffa80078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 -- 172.21.15.3:0/3360038284 shutdown_connections 2024-03-21T08:31:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.558+0000 7ffa7f200640 1 -- 172.21.15.3:0/3360038284 wait complete. 2024-03-21T08:31:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.559+0000 7ffa7f200640 1 Processor -- start 2024-03-21T08:31:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.559+0000 7ffa7f200640 1 -- start start 2024-03-21T08:31:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.559+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa8019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.559+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa80108cf0 0x7ffa8019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa801096b0 0x7ffa801a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80113600 con 0x7ffa80107b00 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80113480 con 0x7ffa801096b0 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80113780 con 0x7ffa80108cf0 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa8019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa8019dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52392/0 (socket says 172.21.15.3:52392) 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 -- 172.21.15.3:0/1377674916 learned_addr learned my addr 172.21.15.3:0/1377674916 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7e800640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa801096b0 0x7ffa801a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7d400640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa80108cf0 0x7ffa8019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 -- 172.21.15.3:0/1377674916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa80108cf0 msgr2=0x7ffa8019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa80108cf0 0x7ffa8019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 -- 172.21.15.3:0/1377674916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa801096b0 msgr2=0x7ffa801a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa801096b0 0x7ffa801a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.560+0000 7ffa7de00640 1 -- 172.21.15.3:0/1377674916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa801a5930 con 0x7ffa80107b00 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.561+0000 7ffa7de00640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa8019dd20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffa70000c00 tx=0x7ffa70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.561+0000 7ffa76a00640 1 -- 172.21.15.3:0/1377674916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70043020 con 0x7ffa80107b00 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.561+0000 7ffa76a00640 1 -- 172.21.15.3:0/1377674916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa70004aa0 con 0x7ffa80107b00 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.561+0000 7ffa76a00640 1 -- 172.21.15.3:0/1377674916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70003d00 con 0x7ffa80107b00 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.561+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa801a5bc0 con 0x7ffa80107b00 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.561+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa801a60c0 con 0x7ffa80107b00 2024-03-21T08:31:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.562+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8010eea0 con 0x7ffa80107b00 2024-03-21T08:31:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.564+0000 7ffa76a00640 1 -- 172.21.15.3:0/1377674916 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7ffa70046020 con 0x7ffa80107b00 2024-03-21T08:31:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.565+0000 7ffa76a00640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ffa640a2fa0 0x7ffa640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.566+0000 7ffa76a00640 1 -- 172.21.15.3:0/1377674916 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ffa700f1eb0 con 0x7ffa80107b00 2024-03-21T08:31:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.566+0000 7ffa7d400640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ffa640a2fa0 0x7ffa640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.567+0000 7ffa7d400640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ffa640a2fa0 0x7ffa640a5460 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c005fd0 tx=0x7ffa6c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.570+0000 7ffa76a00640 1 -- 172.21.15.3:0/1377674916 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa7004d020 con 0x7ffa80107b00 2024-03-21T08:31:15.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:15 smithi084 ceph-mon[35477]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:15.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:15 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/295903713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:15.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:15 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3421425734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.778+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa80062340 con 0x7ffa80107b00 2024-03-21T08:31:15.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.779+0000 7ffa76a00640 1 -- 172.21.15.3:0/1377674916 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa700be840 con 0x7ffa80107b00 2024-03-21T08:31:15.780 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:31:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ffa640a2fa0 msgr2=0x7ffa640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ffa640a2fa0 0x7ffa640a5460 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c005fd0 tx=0x7ffa6c005850 comp rx=0 tx=0).stop 2024-03-21T08:31:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 msgr2=0x7ffa8019dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:15.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa8019dd20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffa70000c00 tx=0x7ffa70004270 comp rx=0 tx=0).stop 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 shutdown_connections 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa801096b0 0x7ffa801a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ffa640a2fa0 0x7ffa640a5460 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa80108cf0 0x7ffa8019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 --2- 172.21.15.3:0/1377674916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa80107b00 0x7ffa8019dd20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 >> 172.21.15.3:0/1377674916 conn(0x7ffa80076a90 msgr2=0x7ffa8010c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 shutdown_connections 2024-03-21T08:31:15.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:15.782+0000 7ffa7f200640 1 -- 172.21.15.3:0/1377674916 wait complete. 2024-03-21T08:31:15.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.030+0000 7f9967e00640 1 -- 172.21.15.3:0/2121537508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996810dac0 msgr2=0x7f996810feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.030+0000 7f9967e00640 1 --2- 172.21.15.3:0/2121537508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996810dac0 0x7f996810feb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9960009cd0 tx=0x7f996002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.031+0000 7f9967e00640 1 -- 172.21.15.3:0/2121537508 shutdown_connections 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.031+0000 7f9967e00640 1 --2- 172.21.15.3:0/2121537508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996810dac0 0x7f996810feb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.031+0000 7f9967e00640 1 --2- 172.21.15.3:0/2121537508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968074100 0x7f9968074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.031+0000 7f9967e00640 1 --2- 172.21.15.3:0/2121537508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99680756e0 0x7f9968073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.031+0000 7f9967e00640 1 -- 172.21.15.3:0/2121537508 >> 172.21.15.3:0/2121537508 conn(0x7f99680fca60 msgr2=0x7f99680feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.031+0000 7f9967e00640 1 -- 172.21.15.3:0/2121537508 shutdown_connections 2024-03-21T08:31:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.031+0000 7f9967e00640 1 -- 172.21.15.3:0/2121537508 wait complete. 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 Processor -- start 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 -- start start 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9968074100 0x7f996819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99680756e0 0x7f996819e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996810dac0 0x7f99681a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681135f0 con 0x7f9968074100 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9968113470 con 0x7f99680756e0 2024-03-21T08:31:16.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.032+0000 7f9967e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9968113770 con 0x7f996810dac0 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9968074100 0x7f996819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9968074100 0x7f996819dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52406/0 (socket says 172.21.15.3:52406) 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 -- 172.21.15.3:0/2082048414 learned_addr learned my addr 172.21.15.3:0/2082048414 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966000640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99680756e0 0x7f996819e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9967400640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996810dac0 0x7f99681a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 -- 172.21.15.3:0/2082048414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996810dac0 msgr2=0x7f99681a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996810dac0 0x7f99681a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 -- 172.21.15.3:0/2082048414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99680756e0 msgr2=0x7f996819e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99680756e0 0x7f996819e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 -- 172.21.15.3:0/2082048414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99681a58e0 con 0x7f9968074100 2024-03-21T08:31:16.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.033+0000 7f9966a00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9968074100 0x7f996819dca0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f995800e9f0 tx=0x7f995800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:16.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.034+0000 7f996ca00640 1 -- 172.21.15.3:0/2082048414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995800cd80 con 0x7f9968074100 2024-03-21T08:31:16.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.034+0000 7f996ca00640 1 -- 172.21.15.3:0/2082048414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99580108e0 con 0x7f9968074100 2024-03-21T08:31:16.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.034+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99681a5bd0 con 0x7f9968074100 2024-03-21T08:31:16.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.034+0000 7f996ca00640 1 -- 172.21.15.3:0/2082048414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958018920 con 0x7f9968074100 2024-03-21T08:31:16.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.034+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99681a6120 con 0x7f9968074100 2024-03-21T08:31:16.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.035+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9968103c90 con 0x7f9968074100 2024-03-21T08:31:16.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.037+0000 7f996ca00640 1 -- 172.21.15.3:0/2082048414 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f9958010a90 con 0x7f9968074100 2024-03-21T08:31:16.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.039+0000 7f996ca00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f994c0a2f20 0x7f994c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.039+0000 7f996ca00640 1 -- 172.21.15.3:0/2082048414 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f99580cfff0 con 0x7f9968074100 2024-03-21T08:31:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.039+0000 7f9966000640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f994c0a2f20 0x7f994c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.040+0000 7f9966000640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f994c0a2f20 0x7f994c0a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f996819f1a0 tx=0x7f995400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:16.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.042+0000 7f996ca00640 1 -- 172.21.15.3:0/2082048414 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f995809c370 con 0x7f9968074100 2024-03-21T08:31:16.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.244+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f996802d270 con 0x7f9968074100 2024-03-21T08:31:16.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.245+0000 7f996ca00640 1 -- 172.21.15.3:0/2082048414 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f99580a0410 con 0x7f9968074100 2024-03-21T08:31:16.246 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f994c0a2f20 msgr2=0x7f994c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f994c0a2f20 0x7f994c0a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f996819f1a0 tx=0x7f995400b390 comp rx=0 tx=0).stop 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9968074100 msgr2=0x7f996819dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9968074100 0x7f996819dca0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f995800e9f0 tx=0x7f995800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 shutdown_connections 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f994c0a2f20 0x7f994c0a53e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996810dac0 0x7f99681a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99680756e0 0x7f996819e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 --2- 172.21.15.3:0/2082048414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9968074100 0x7f996819dca0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 >> 172.21.15.3:0/2082048414 conn(0x7f99680fca60 msgr2=0x7f9968100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 shutdown_connections 2024-03-21T08:31:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.248+0000 7f9967e00640 1 -- 172.21.15.3:0/2082048414 wait complete. 2024-03-21T08:31:16.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:31:16.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:16 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4246360619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:16 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1377674916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 -- 172.21.15.3:0/640085801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081049b0 msgr2=0x7f9708104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 --2- 172.21.15.3:0/640085801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081049b0 0x7f9708104e30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009900 tx=0x7f96f002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 -- 172.21.15.3:0/640085801 shutdown_connections 2024-03-21T08:31:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 --2- 172.21.15.3:0/640085801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9708105370 0x7f970810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 --2- 172.21.15.3:0/640085801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081049b0 0x7f9708104e30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 --2- 172.21.15.3:0/640085801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97081037c0 0x7f9708103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 -- 172.21.15.3:0/640085801 >> 172.21.15.3:0/640085801 conn(0x7f97080fed50 msgr2=0x7f9708101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 -- 172.21.15.3:0/640085801 shutdown_connections 2024-03-21T08:31:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.494+0000 7f970da00640 1 -- 172.21.15.3:0/640085801 wait complete. 2024-03-21T08:31:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.495+0000 7f970da00640 1 Processor -- start 2024-03-21T08:31:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.495+0000 7f970da00640 1 -- start start 2024-03-21T08:31:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f970da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081037c0 0x7f9708199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f970da00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97081049b0 0x7f9708199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f970da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9708105370 0x7f97081a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f970da00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970810f2f0 con 0x7f97081037c0 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f970da00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970810f170 con 0x7f9708105370 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f970da00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970810f470 con 0x7f97081049b0 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f9707e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081037c0 0x7f9708199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f9707e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081037c0 0x7f9708199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52420/0 (socket says 172.21.15.3:52420) 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f9707e00640 1 -- 172.21.15.3:0/159347010 learned_addr learned my addr 172.21.15.3:0/159347010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f970d000640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9708105370 0x7f97081a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.496+0000 7f96ffe00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97081049b0 0x7f9708199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9707e00640 1 -- 172.21.15.3:0/159347010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97081049b0 msgr2=0x7f9708199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9707e00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97081049b0 0x7f9708199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9707e00640 1 -- 172.21.15.3:0/159347010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9708105370 msgr2=0x7f97081a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9707e00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9708105370 0x7f97081a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9707e00640 1 -- 172.21.15.3:0/159347010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97081a15a0 con 0x7f97081037c0 2024-03-21T08:31:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9707e00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081037c0 0x7f9708199a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f96f800e9c0 tx=0x7f96f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9705600640 1 -- 172.21.15.3:0/159347010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800cd80 con 0x7f97081037c0 2024-03-21T08:31:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9705600640 1 -- 172.21.15.3:0/159347010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96f80035f0 con 0x7f97081037c0 2024-03-21T08:31:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f9705600640 1 -- 172.21.15.3:0/159347010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8010690 con 0x7f97081037c0 2024-03-21T08:31:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97081a1890 con 0x7f97081037c0 2024-03-21T08:31:16.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.497+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97081a1de0 con 0x7f97081037c0 2024-03-21T08:31:16.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.499+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970810aae0 con 0x7f97081037c0 2024-03-21T08:31:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.501+0000 7f9705600640 1 -- 172.21.15.3:0/159347010 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f96f80037a0 con 0x7f97081037c0 2024-03-21T08:31:16.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.502+0000 7f9705600640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96ec0a2f10 0x7f96ec0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:16.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.502+0000 7f9705600640 1 -- 172.21.15.3:0/159347010 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f96f80cfcb0 con 0x7f97081037c0 2024-03-21T08:31:16.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.502+0000 7f96ffe00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96ec0a2f10 0x7f96ec0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:16.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.503+0000 7f96ffe00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96ec0a2f10 0x7f96ec0a53d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f970819ae90 tx=0x7f96f0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.506+0000 7f9705600640 1 -- 172.21.15.3:0/159347010 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f96f809c640 con 0x7f97081037c0 2024-03-21T08:31:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:16 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4246360619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:16 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1377674916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:16 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4246360619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:16 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1377674916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:16.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.759+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f970802d070 con 0x7f97081037c0 2024-03-21T08:31:16.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.760+0000 7f9705600640 1 -- 172.21.15.3:0/159347010 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f96f809bf40 con 0x7f97081037c0 2024-03-21T08:31:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:16.762 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:29:09.323523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34409,34472]},"id":1}]} 2024-03-21T08:31:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-03-21T08:31:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96ec0a2f10 msgr2=0x7f96ec0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96ec0a2f10 0x7f96ec0a53d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f970819ae90 tx=0x7f96f0005be0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081037c0 msgr2=0x7f9708199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081037c0 0x7f9708199a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f96f800e9c0 tx=0x7f96f800ee80 comp rx=0 tx=0).stop 2024-03-21T08:31:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 shutdown_connections 2024-03-21T08:31:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9708105370 0x7f97081a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f96ec0a2f10 0x7f96ec0a53d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97081049b0 0x7f9708199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 --2- 172.21.15.3:0/159347010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97081037c0 0x7f9708199a20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 >> 172.21.15.3:0/159347010 conn(0x7f97080fed50 msgr2=0x7f9708107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 shutdown_connections 2024-03-21T08:31:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:16.763+0000 7f970da00640 1 -- 172.21.15.3:0/159347010 wait complete. 2024-03-21T08:31:16.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:31:17.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 -- 172.21.15.3:0/1339174773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb000772a0 msgr2=0x7feb00077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 --2- 172.21.15.3:0/1339174773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb000772a0 0x7feb00077700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feae8009900 tx=0x7feae802bc50 comp rx=0 tx=0).stop 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 -- 172.21.15.3:0/1339174773 shutdown_connections 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 --2- 172.21.15.3:0/1339174773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb00077c40 0x7feb000784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 --2- 172.21.15.3:0/1339174773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb000772a0 0x7feb00077700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 --2- 172.21.15.3:0/1339174773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb00076060 0x7feb00076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 -- 172.21.15.3:0/1339174773 >> 172.21.15.3:0/1339174773 conn(0x7feb000feeb0 msgr2=0x7feb00101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.005+0000 7feb07200640 1 -- 172.21.15.3:0/1339174773 shutdown_connections 2024-03-21T08:31:17.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.006+0000 7feb07200640 1 -- 172.21.15.3:0/1339174773 wait complete. 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.006+0000 7feb07200640 1 Processor -- start 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb07200640 1 -- start start 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb07200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb00076060 0x7feb0019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb07200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb000772a0 0x7feb0019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb07200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb00077c40 0x7feb001a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb07200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00113870 con 0x7feb00077c40 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb07200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb001136f0 con 0x7feb000772a0 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb07200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb001139f0 con 0x7feb00076060 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb05200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb00077c40 0x7feb001a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb05200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb00077c40 0x7feb001a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52440/0 (socket says 172.21.15.3:52440) 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feb05200640 1 -- 172.21.15.3:0/3304737316 learned_addr learned my addr 172.21.15.3:0/3304737316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.007+0000 7feaf7400640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb000772a0 0x7feb0019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feaffe00640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb00076060 0x7feb0019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feb05200640 1 -- 172.21.15.3:0/3304737316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb00076060 msgr2=0x7feb0019df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feb05200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb00076060 0x7feb0019df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feb05200640 1 -- 172.21.15.3:0/3304737316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb000772a0 msgr2=0x7feb0019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feb05200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb000772a0 0x7feb0019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feb05200640 1 -- 172.21.15.3:0/3304737316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb001a5b20 con 0x7feb00077c40 2024-03-21T08:31:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feb05200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb00077c40 0x7feb001a54b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feaf800d750 tx=0x7feaf800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feafd600640 1 -- 172.21.15.3:0/3304737316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8017070 con 0x7feb00077c40 2024-03-21T08:31:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feafd600640 1 -- 172.21.15.3:0/3304737316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feaf80054d0 con 0x7feb00077c40 2024-03-21T08:31:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.008+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb001a5e10 con 0x7feb00077c40 2024-03-21T08:31:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.009+0000 7feafd600640 1 -- 172.21.15.3:0/3304737316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8004c60 con 0x7feb00077c40 2024-03-21T08:31:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.009+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb001a62d0 con 0x7feb00077c40 2024-03-21T08:31:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.010+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb001104b0 con 0x7feb00077c40 2024-03-21T08:31:17.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.012+0000 7feafd600640 1 -- 172.21.15.3:0/3304737316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7feaf8004e00 con 0x7feb00077c40 2024-03-21T08:31:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.013+0000 7feafd600640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7feae40a2f20 0x7feae40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.014+0000 7feafd600640 1 -- 172.21.15.3:0/3304737316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7feaf80cf640 con 0x7feb00077c40 2024-03-21T08:31:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.014+0000 7feaffe00640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7feae40a2f20 0x7feae40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.014+0000 7feaffe00640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7feae40a2f20 0x7feae40a53e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005fd0 tx=0x7feaf0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:17.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.016+0000 7feafd600640 1 -- 172.21.15.3:0/3304737316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feaf809bfd0 con 0x7feb00077c40 2024-03-21T08:31:17.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.222+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb0002d270 con 0x7feb00077c40 2024-03-21T08:31:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.226+0000 7feafd600640 1 -- 172.21.15.3:0/3304737316 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7feaf809b8d0 con 0x7feb00077c40 2024-03-21T08:31:17.228 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:17.228 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:31:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7feae40a2f20 msgr2=0x7feae40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7feae40a2f20 0x7feae40a53e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005fd0 tx=0x7feaf0009500 comp rx=0 tx=0).stop 2024-03-21T08:31:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb00077c40 msgr2=0x7feb001a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb00077c40 0x7feb001a54b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feaf800d750 tx=0x7feaf800dc10 comp rx=0 tx=0).stop 2024-03-21T08:31:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 shutdown_connections 2024-03-21T08:31:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb00077c40 0x7feb001a54b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb000772a0 0x7feb0019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7feae40a2f20 0x7feae40a53e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 --2- 172.21.15.3:0/3304737316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb00076060 0x7feb0019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 >> 172.21.15.3:0/3304737316 conn(0x7feb000feeb0 msgr2=0x7feb001004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.229+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 shutdown_connections 2024-03-21T08:31:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.230+0000 7feb07200640 1 -- 172.21.15.3:0/3304737316 wait complete. 2024-03-21T08:31:17.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:31:17.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:31:17.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:17 smithi003 ceph-mon[31675]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:17 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2082048414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:17.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:17 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/159347010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:17.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 -- 172.21.15.3:0/3658984134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0107ce0 msgr2=0x7f85f0108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 --2- 172.21.15.3:0/3658984134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0107ce0 0x7f85f0108100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009900 tx=0x7f85e002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 -- 172.21.15.3:0/3658984134 shutdown_connections 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 --2- 172.21.15.3:0/3658984134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85f0109890 0x7f85f0110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 --2- 172.21.15.3:0/3658984134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85f0108ed0 0x7f85f0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 --2- 172.21.15.3:0/3658984134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0107ce0 0x7f85f0108100 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 -- 172.21.15.3:0/3658984134 >> 172.21.15.3:0/3658984134 conn(0x7f85f0076c90 msgr2=0x7f85f00790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 -- 172.21.15.3:0/3658984134 shutdown_connections 2024-03-21T08:31:17.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.486+0000 7f85f6800640 1 -- 172.21.15.3:0/3658984134 wait complete. 2024-03-21T08:31:17.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.487+0000 7f85f6800640 1 Processor -- start 2024-03-21T08:31:17.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85f6800640 1 -- start start 2024-03-21T08:31:17.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85f6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85f0107ce0 0x7f85f019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85f6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85f0108ed0 0x7f85f019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85f6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0109890 0x7f85f01a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85f6800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0113810 con 0x7f85f0109890 2024-03-21T08:31:17.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85f6800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0113690 con 0x7f85f0108ed0 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85f6800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0113990 con 0x7f85f0107ce0 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85efe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0109890 0x7f85f01a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85efe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0109890 0x7f85f01a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52466/0 (socket says 172.21.15.3:52466) 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85efe00640 1 -- 172.21.15.3:0/164857460 learned_addr learned my addr 172.21.15.3:0/164857460 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.488+0000 7f85eea00640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85f0108ed0 0x7f85f019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85ef400640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85f0107ce0 0x7f85f019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85efe00640 1 -- 172.21.15.3:0/164857460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85f0107ce0 msgr2=0x7f85f019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85efe00640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85f0107ce0 0x7f85f019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85efe00640 1 -- 172.21.15.3:0/164857460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85f0108ed0 msgr2=0x7f85f019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85efe00640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85f0108ed0 0x7f85f019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85efe00640 1 -- 172.21.15.3:0/164857460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f01a5b70 con 0x7f85f0109890 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85eea00640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85f0108ed0 0x7f85f019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:17.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85efe00640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0109890 0x7f85f01a5500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f85e800eaa0 tx=0x7f85e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85e7e00640 1 -- 172.21.15.3:0/164857460 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800cdb0 con 0x7f85f0109890 2024-03-21T08:31:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85e7e00640 1 -- 172.21.15.3:0/164857460 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f85e8005020 con 0x7f85f0109890 2024-03-21T08:31:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.489+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f01a5e60 con 0x7f85f0109890 2024-03-21T08:31:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.490+0000 7f85e7e00640 1 -- 172.21.15.3:0/164857460 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80116d0 con 0x7f85f0109890 2024-03-21T08:31:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.490+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f01a6320 con 0x7f85f0109890 2024-03-21T08:31:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.491+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f010f110 con 0x7f85f0109890 2024-03-21T08:31:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.493+0000 7f85e7e00640 1 -- 172.21.15.3:0/164857460 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f85e80051d0 con 0x7f85f0109890 2024-03-21T08:31:17.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.495+0000 7f85e7e00640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f85d40a2f20 0x7f85d40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.495+0000 7f85e7e00640 1 -- 172.21.15.3:0/164857460 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f85e80cfaa0 con 0x7f85f0109890 2024-03-21T08:31:17.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.495+0000 7f85ef400640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f85d40a2f20 0x7f85d40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.496+0000 7f85ef400640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f85d40a2f20 0x7f85d40a53e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85e0006010 tx=0x7f85e0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:17.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.500+0000 7f85e7e00640 1 -- 172.21.15.3:0/164857460 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f85e8016030 con 0x7f85f0109890 2024-03-21T08:31:17.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:17 smithi084 ceph-mon[35477]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:17.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:17 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2082048414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:17.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:17 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/159347010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:17.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:17 smithi171 ceph-mon[35388]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:17.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:17 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2082048414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:17.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:17 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/159347010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.702+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85f0062340 con 0x7f85f0109890 2024-03-21T08:31:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.703+0000 7f85e7e00640 1 -- 172.21.15.3:0/164857460 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85e809c430 con 0x7f85f0109890 2024-03-21T08:31:17.704 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:31:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f85d40a2f20 msgr2=0x7f85d40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f85d40a2f20 0x7f85d40a53e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85e0006010 tx=0x7f85e0005be0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0109890 msgr2=0x7f85f01a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0109890 0x7f85f01a5500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f85e800eaa0 tx=0x7f85e800ef60 comp rx=0 tx=0).stop 2024-03-21T08:31:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 shutdown_connections 2024-03-21T08:31:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85f0109890 0x7f85f01a5500 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85f0108ed0 0x7f85f019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f85d40a2f20 0x7f85d40a53e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 --2- 172.21.15.3:0/164857460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85f0107ce0 0x7f85f019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.706+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 >> 172.21.15.3:0/164857460 conn(0x7f85f0076c90 msgr2=0x7f85f010c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.707+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 shutdown_connections 2024-03-21T08:31:17.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.707+0000 7f85f6800640 1 -- 172.21.15.3:0/164857460 wait complete. 2024-03-21T08:31:17.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:31:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 -- 172.21.15.3:0/3108900705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 msgr2=0x7f2208104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 --2- 172.21.15.3:0/3108900705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f2208104360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009900 tx=0x7f21f002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 -- 172.21.15.3:0/3108900705 shutdown_connections 2024-03-21T08:31:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 --2- 172.21.15.3:0/3108900705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22081071d0 0x7f2208109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 --2- 172.21.15.3:0/3108900705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22081048a0 0x7f2208106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 --2- 172.21.15.3:0/3108900705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f2208104360 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 -- 172.21.15.3:0/3108900705 >> 172.21.15.3:0/3108900705 conn(0x7f22080fb980 msgr2=0x7f22080fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:17.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 -- 172.21.15.3:0/3108900705 shutdown_connections 2024-03-21T08:31:17.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.958+0000 7f220d000640 1 -- 172.21.15.3:0/3108900705 wait complete. 2024-03-21T08:31:17.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.959+0000 7f220d000640 1 Processor -- start 2024-03-21T08:31:17.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.959+0000 7f220d000640 1 -- start start 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f220d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f22081998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f220d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22081048a0 0x7f2208199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f220d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22081071d0 0x7f22081a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f220d000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220810d080 con 0x7f2208101f70 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f220d000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220810cf00 con 0x7f22081071d0 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f220d000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220810d200 con 0x7f22081048a0 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f22081998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f22081998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52480/0 (socket says 172.21.15.3:52480) 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 -- 172.21.15.3:0/1566497779 learned_addr learned my addr 172.21.15.3:0/1566497779 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207e00640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22081071d0 0x7f22081a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2206a00640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22081048a0 0x7f2208199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 -- 172.21.15.3:0/1566497779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22081048a0 msgr2=0x7f2208199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22081048a0 0x7f2208199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 -- 172.21.15.3:0/1566497779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22081071d0 msgr2=0x7f22081a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22081071d0 0x7f22081a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2207400640 1 -- 172.21.15.3:0/1566497779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22081a1510 con 0x7f2208101f70 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.960+0000 7f2206a00640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22081048a0 0x7f2208199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.961+0000 7f2207400640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f22081998f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000c00 tx=0x7f21f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.961+0000 7f21ffe00640 1 -- 172.21.15.3:0/1566497779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0043020 con 0x7f2208101f70 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.961+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22081a17a0 con 0x7f2208101f70 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.961+0000 7f21ffe00640 1 -- 172.21.15.3:0/1566497779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f21f00054c0 con 0x7f2208101f70 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.961+0000 7f21ffe00640 1 -- 172.21.15.3:0/1566497779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0003d70 con 0x7f2208101f70 2024-03-21T08:31:17.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.961+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22081a1cf0 con 0x7f2208101f70 2024-03-21T08:31:17.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.962+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2208108d20 con 0x7f2208101f70 2024-03-21T08:31:17.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.964+0000 7f21ffe00640 1 -- 172.21.15.3:0/1566497779 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f21f0046020 con 0x7f2208101f70 2024-03-21T08:31:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.966+0000 7f21ffe00640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21ec0a2f20 0x7f21ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.966+0000 7f21ffe00640 1 -- 172.21.15.3:0/1566497779 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f21f00f21c0 con 0x7f2208101f70 2024-03-21T08:31:17.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.966+0000 7f2206a00640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21ec0a2f20 0x7f21ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:17.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.967+0000 7f2206a00640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21ec0a2f20 0x7f21ec0a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f220819ae00 tx=0x7f21f80093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:17.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:17.968+0000 7f21ffe00640 1 -- 172.21.15.3:0/1566497779 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f21f004d020 con 0x7f2208101f70 2024-03-21T08:31:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.177+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22080622a0 con 0x7f2208101f70 2024-03-21T08:31:18.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.178+0000 7f21ffe00640 1 -- 172.21.15.3:0/1566497779 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21f00bec70 con 0x7f2208101f70 2024-03-21T08:31:18.179 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21ec0a2f20 msgr2=0x7f21ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21ec0a2f20 0x7f21ec0a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f220819ae00 tx=0x7f21f80093e0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 msgr2=0x7f22081998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f22081998f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000c00 tx=0x7f21f00042f0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 shutdown_connections 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22081071d0 0x7f22081a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f21ec0a2f20 0x7f21ec0a53e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22081048a0 0x7f2208199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 --2- 172.21.15.3:0/1566497779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208101f70 0x7f22081998f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 >> 172.21.15.3:0/1566497779 conn(0x7f22080fb980 msgr2=0x7f2208101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:18.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 shutdown_connections 2024-03-21T08:31:18.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.181+0000 7f220d000640 1 -- 172.21.15.3:0/1566497779 wait complete. 2024-03-21T08:31:18.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:31:18.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3304737316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:18.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/164857460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 -- 172.21.15.3:0/14383387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec1038e0 msgr2=0x7fa0ec103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 --2- 172.21.15.3:0/14383387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec1038e0 0x7fa0ec103d00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009ca0 tx=0x7fa0dc02c100 comp rx=0 tx=0).stop 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 -- 172.21.15.3:0/14383387 shutdown_connections 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 --2- 172.21.15.3:0/14383387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0ec105490 0x7fa0ec10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 --2- 172.21.15.3:0/14383387 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0ec104ad0 0x7fa0ec104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 --2- 172.21.15.3:0/14383387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec1038e0 0x7fa0ec103d00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 -- 172.21.15.3:0/14383387 >> 172.21.15.3:0/14383387 conn(0x7fa0ec0fee50 msgr2=0x7fa0ec101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 -- 172.21.15.3:0/14383387 shutdown_connections 2024-03-21T08:31:18.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.438+0000 7fa0f4600640 1 -- 172.21.15.3:0/14383387 wait complete. 2024-03-21T08:31:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.439+0000 7fa0f4600640 1 Processor -- start 2024-03-21T08:31:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.439+0000 7fa0f4600640 1 -- start start 2024-03-21T08:31:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0ec1038e0 0x7fa0ec199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec104ad0 0x7fa0ec19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0ec105490 0x7fa0ec1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f4600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec10f3e0 con 0x7fa0ec104ad0 2024-03-21T08:31:18.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f4600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec10f260 con 0x7fa0ec1038e0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f4600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec10f560 con 0x7fa0ec105490 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec104ad0 0x7fa0ec19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec104ad0 0x7fa0ec19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52488/0 (socket says 172.21.15.3:52488) 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 -- 172.21.15.3:0/4226125321 learned_addr learned my addr 172.21.15.3:0/4226125321 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1c00640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0ec1038e0 0x7fa0ec199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 -- 172.21.15.3:0/4226125321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0ec105490 msgr2=0x7fa0ec1a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0ec105490 0x7fa0ec1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 -- 172.21.15.3:0/4226125321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0ec1038e0 msgr2=0x7fa0ec199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0ec1038e0 0x7fa0ec199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.440+0000 7fa0f1200640 1 -- 172.21.15.3:0/4226125321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec1a1780 con 0x7fa0ec104ad0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.441+0000 7fa0f1200640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec104ad0 0x7fa0ec19a0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800c890 tx=0x7fa0d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.441+0000 7fa0ea000640 1 -- 172.21.15.3:0/4226125321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8004220 con 0x7fa0ec104ad0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.441+0000 7fa0ea000640 1 -- 172.21.15.3:0/4226125321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0d80043c0 con 0x7fa0ec104ad0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.441+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec1a1a70 con 0x7fa0ec104ad0 2024-03-21T08:31:18.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.441+0000 7fa0ea000640 1 -- 172.21.15.3:0/4226125321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8004a00 con 0x7fa0ec104ad0 2024-03-21T08:31:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.441+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec1a1fc0 con 0x7fa0ec104ad0 2024-03-21T08:31:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.443+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec10ad20 con 0x7fa0ec104ad0 2024-03-21T08:31:18.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.445+0000 7fa0ea000640 1 -- 172.21.15.3:0/4226125321 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fa0d8004ba0 con 0x7fa0ec104ad0 2024-03-21T08:31:18.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.447+0000 7fa0ea000640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a2f20 0x7fa0d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.447+0000 7fa0ea000640 1 -- 172.21.15.3:0/4226125321 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa0d80cf540 con 0x7fa0ec104ad0 2024-03-21T08:31:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.447+0000 7fa0f1c00640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a2f20 0x7fa0d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:18.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.448+0000 7fa0f1c00640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a2f20 0x7fa0d00a53e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc02c710 tx=0x7fa0dc032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:18.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.451+0000 7fa0ea000640 1 -- 172.21.15.3:0/4226125321 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0d8014030 con 0x7fa0ec104ad0 2024-03-21T08:31:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:18 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3304737316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:18 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/164857460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:18.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:18 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3304737316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:18.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:18 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/164857460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.655+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0ec0622a0 con 0x7fa0ec104ad0 2024-03-21T08:31:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.656+0000 7fa0ea000640 1 -- 172.21.15.3:0/4226125321 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0d809bed0 con 0x7fa0ec104ad0 2024-03-21T08:31:18.657 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:31:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.659+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a2f20 msgr2=0x7fa0d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.659+0000 7fa0f4600640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a2f20 0x7fa0d00a53e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc02c710 tx=0x7fa0dc032340 comp rx=0 tx=0).stop 2024-03-21T08:31:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.659+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec104ad0 msgr2=0x7fa0ec19a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.659+0000 7fa0f4600640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec104ad0 0x7fa0ec19a0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800c890 tx=0x7fa0d800cd50 comp rx=0 tx=0).stop 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 shutdown_connections 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa0d00a2f20 0x7fa0d00a53e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0ec105490 0x7fa0ec1a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ec104ad0 0x7fa0ec19a0a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 --2- 172.21.15.3:0/4226125321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0ec1038e0 0x7fa0ec199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 >> 172.21.15.3:0/4226125321 conn(0x7fa0ec0fee50 msgr2=0x7fa0ec107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 shutdown_connections 2024-03-21T08:31:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.660+0000 7fa0f4600640 1 -- 172.21.15.3:0/4226125321 wait complete. 2024-03-21T08:31:18.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.910+0000 7f9aaac00640 1 -- 172.21.15.3:0/3544692229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa41071d0 msgr2=0x7f9aa4109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.910+0000 7f9aaac00640 1 --2- 172.21.15.3:0/3544692229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa41071d0 0x7f9aa4109620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009ca0 tx=0x7f9a9c02c100 comp rx=0 tx=0).stop 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.911+0000 7f9aaac00640 1 -- 172.21.15.3:0/3544692229 shutdown_connections 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.911+0000 7f9aaac00640 1 --2- 172.21.15.3:0/3544692229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa41071d0 0x7f9aa4109620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.911+0000 7f9aaac00640 1 --2- 172.21.15.3:0/3544692229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9aa41048a0 0x7f9aa4106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.911+0000 7f9aaac00640 1 --2- 172.21.15.3:0/3544692229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa4101f70 0x7f9aa4104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.911+0000 7f9aaac00640 1 -- 172.21.15.3:0/3544692229 >> 172.21.15.3:0/3544692229 conn(0x7f9aa40fb980 msgr2=0x7f9aa40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.911+0000 7f9aaac00640 1 -- 172.21.15.3:0/3544692229 shutdown_connections 2024-03-21T08:31:18.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.911+0000 7f9aaac00640 1 -- 172.21.15.3:0/3544692229 wait complete. 2024-03-21T08:31:18.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.912+0000 7f9aaac00640 1 Processor -- start 2024-03-21T08:31:18.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.912+0000 7f9aaac00640 1 -- start start 2024-03-21T08:31:18.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aaac00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa4101f70 0x7f9aa41976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aaac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa41048a0 0x7f9aa4197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aaac00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9aa41071d0 0x7f9aa419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aaac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa410d030 con 0x7f9aa41048a0 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aaac00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa410ceb0 con 0x7f9aa4101f70 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aaac00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa410d1b0 con 0x7f9aa41071d0 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aa8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa41048a0 0x7f9aa4197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aa8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa41048a0 0x7f9aa4197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52508/0 (socket says 172.21.15.3:52508) 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aa8e00640 1 -- 172.21.15.3:0/1269900816 learned_addr learned my addr 172.21.15.3:0/1269900816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.913+0000 7f9aa9800640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa4101f70 0x7f9aa41976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aaa200640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9aa41071d0 0x7f9aa419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa8e00640 1 -- 172.21.15.3:0/1269900816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9aa41071d0 msgr2=0x7f9aa419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa8e00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9aa41071d0 0x7f9aa419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa8e00640 1 -- 172.21.15.3:0/1269900816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa4101f70 msgr2=0x7f9aa41976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa8e00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa4101f70 0x7f9aa41976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa8e00640 1 -- 172.21.15.3:0/1269900816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa419f2e0 con 0x7f9aa41048a0 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aaa200640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9aa41071d0 0x7f9aa419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa8e00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa41048a0 0x7f9aa4197c00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000d770 tx=0x7f9a9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa2000640 1 -- 172.21.15.3:0/1269900816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90017070 con 0x7f9aa41048a0 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa2000640 1 -- 172.21.15.3:0/1269900816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a90009c30 con 0x7f9aa41048a0 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa419f5d0 con 0x7f9aa41048a0 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.914+0000 7f9aa2000640 1 -- 172.21.15.3:0/1269900816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000f620 con 0x7f9aa41048a0 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.915+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa419fb70 con 0x7f9aa41048a0 2024-03-21T08:31:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.916+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa40fd470 con 0x7f9aa41048a0 2024-03-21T08:31:18.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.918+0000 7f9aa2000640 1 -- 172.21.15.3:0/1269900816 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f9a9001a070 con 0x7f9aa41048a0 2024-03-21T08:31:18.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.919+0000 7f9aa2000640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a880a2fa0 0x7f9a880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:18.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.919+0000 7f9aa2000640 1 -- 172.21.15.3:0/1269900816 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f9a900cf2f0 con 0x7f9aa41048a0 2024-03-21T08:31:18.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.920+0000 7f9aa9800640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a880a2fa0 0x7f9a880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:18.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.920+0000 7f9aa9800640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a880a2fa0 0x7f9a880a5460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009bd0 tx=0x7f9a94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:18.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:18.924+0000 7f9aa2000640 1 -- 172.21.15.3:0/1269900816 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9a90009de0 con 0x7f9aa41048a0 2024-03-21T08:31:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.126+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9aa402d2e0 con 0x7f9aa41048a0 2024-03-21T08:31:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.127+0000 7f9aa2000640 1 -- 172.21.15.3:0/1269900816 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9a9009bd00 con 0x7f9aa41048a0 2024-03-21T08:31:19.128 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a880a2fa0 msgr2=0x7f9a880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a880a2fa0 0x7f9a880a5460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009bd0 tx=0x7f9a94005f90 comp rx=0 tx=0).stop 2024-03-21T08:31:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa41048a0 msgr2=0x7f9aa4197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa41048a0 0x7f9aa4197c00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000d770 tx=0x7f9a9000dc30 comp rx=0 tx=0).stop 2024-03-21T08:31:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 shutdown_connections 2024-03-21T08:31:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f9a880a2fa0 0x7f9a880a5460 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9aa41071d0 0x7f9aa419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9aa41048a0 0x7f9aa4197c00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 --2- 172.21.15.3:0/1269900816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9aa4101f70 0x7f9aa41976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 >> 172.21.15.3:0/1269900816 conn(0x7f9aa40fb980 msgr2=0x7f9aa41028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 shutdown_connections 2024-03-21T08:31:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.130+0000 7f9aaac00640 1 -- 172.21.15.3:0/1269900816 wait complete. 2024-03-21T08:31:19.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:31:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 -- 172.21.15.3:0/2386141240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 msgr2=0x7f2238103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 --2- 172.21.15.3:0/2386141240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238103cf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2228009900 tx=0x7f222802bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 -- 172.21.15.3:0/2386141240 shutdown_connections 2024-03-21T08:31:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 --2- 172.21.15.3:0/2386141240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2238105480 0x7f223810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 --2- 172.21.15.3:0/2386141240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2238104ac0 0x7f2238104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 --2- 172.21.15.3:0/2386141240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238103cf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 -- 172.21.15.3:0/2386141240 >> 172.21.15.3:0/2386141240 conn(0x7f22380fee60 msgr2=0x7f2238101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:19.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 -- 172.21.15.3:0/2386141240 shutdown_connections 2024-03-21T08:31:19.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.379+0000 7f223ca00640 1 -- 172.21.15.3:0/2386141240 wait complete. 2024-03-21T08:31:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.380+0000 7f223ca00640 1 Processor -- start 2024-03-21T08:31:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.380+0000 7f223ca00640 1 -- start start 2024-03-21T08:31:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.380+0000 7f223ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.380+0000 7f223ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2238104ac0 0x7f223819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.380+0000 7f223ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2238105480 0x7f22381a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f223ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223810f3d0 con 0x7f22381038d0 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f223ca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223810f250 con 0x7f2238104ac0 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f223ca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223810f550 con 0x7f2238105480 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52528/0 (socket says 172.21.15.3:52528) 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 -- 172.21.15.3:0/1926299036 learned_addr learned my addr 172.21.15.3:0/1926299036 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235400640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2238104ac0 0x7f223819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2236800640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2238105480 0x7f22381a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 -- 172.21.15.3:0/1926299036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2238105480 msgr2=0x7f22381a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2238105480 0x7f22381a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 -- 172.21.15.3:0/1926299036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2238104ac0 msgr2=0x7f223819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2238104ac0 0x7f223819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.381+0000 7f2235e00640 1 -- 172.21.15.3:0/1926299036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22381a1730 con 0x7f22381038d0 2024-03-21T08:31:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.382+0000 7f2235e00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238199b10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2228000c00 tx=0x7f2228004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:19.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.382+0000 7f222ea00640 1 -- 172.21.15.3:0/1926299036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228043020 con 0x7f22381038d0 2024-03-21T08:31:19.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.382+0000 7f222ea00640 1 -- 172.21.15.3:0/1926299036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2228004440 con 0x7f22381038d0 2024-03-21T08:31:19.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.382+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22381a19c0 con 0x7f22381038d0 2024-03-21T08:31:19.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.382+0000 7f222ea00640 1 -- 172.21.15.3:0/1926299036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228003c90 con 0x7f22381038d0 2024-03-21T08:31:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.382+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22381a1f10 con 0x7f22381038d0 2024-03-21T08:31:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.383+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223810ad10 con 0x7f22381038d0 2024-03-21T08:31:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.385+0000 7f222ea00640 1 -- 172.21.15.3:0/1926299036 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f2228046020 con 0x7f22381038d0 2024-03-21T08:31:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.388+0000 7f222ea00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f221c0a2f20 0x7f221c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.388+0000 7f222ea00640 1 -- 172.21.15.3:0/1926299036 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f22280f1b10 con 0x7f22381038d0 2024-03-21T08:31:19.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.388+0000 7f2235400640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f221c0a2f20 0x7f221c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.389+0000 7f2235400640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f221c0a2f20 0x7f221c0a53e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f223819b020 tx=0x7f2224005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:19.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.392+0000 7f222ea00640 1 -- 172.21.15.3:0/1926299036 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f222804d020 con 0x7f22381038d0 2024-03-21T08:31:19.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:19 smithi003 ceph-mon[31675]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:19.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:19 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1566497779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:19 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4226125321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:19 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1269900816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:19 smithi084 ceph-mon[35477]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:19.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1566497779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4226125321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:19 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1269900816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:19 smithi171 ceph-mon[35388]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:19.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:19 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1566497779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:19 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4226125321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:19 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1269900816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.638+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22380622a0 con 0x7f22381038d0 2024-03-21T08:31:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.640+0000 7f222ea00640 1 -- 172.21.15.3:0/1926299036 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f22280be4a0 con 0x7f22381038d0 2024-03-21T08:31:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:19.641 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:29:09.323523+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34409,34472]},"id":1}]} 2024-03-21T08:31:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.642+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f221c0a2f20 msgr2=0x7f221c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.642+0000 7f223ca00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f221c0a2f20 0x7f221c0a53e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f223819b020 tx=0x7f2224005f90 comp rx=0 tx=0).stop 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.642+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 msgr2=0x7f2238199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.642+0000 7f223ca00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238199b10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2228000c00 tx=0x7f2228004040 comp rx=0 tx=0).stop 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 shutdown_connections 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f221c0a2f20 0x7f221c0a53e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2238105480 0x7f22381a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2238104ac0 0x7f223819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 --2- 172.21.15.3:0/1926299036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22381038d0 0x7f2238199b10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 >> 172.21.15.3:0/1926299036 conn(0x7f22380fee60 msgr2=0x7f2238107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 shutdown_connections 2024-03-21T08:31:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.643+0000 7f223ca00640 1 -- 172.21.15.3:0/1926299036 wait complete. 2024-03-21T08:31:19.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.889+0000 7f281e400640 1 -- 172.21.15.3:0/673032439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 msgr2=0x7f2818102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.889+0000 7f281e400640 1 --2- 172.21.15.3:0/673032439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 0x7f2818102c30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2800009900 tx=0x7f280002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.890+0000 7f281e400640 1 -- 172.21.15.3:0/673032439 shutdown_connections 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.890+0000 7f281e400640 1 --2- 172.21.15.3:0/673032439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2818103170 0x7f2818109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.890+0000 7f281e400640 1 --2- 172.21.15.3:0/673032439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 0x7f2818102c30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.890+0000 7f281e400640 1 --2- 172.21.15.3:0/673032439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 0x7f28181019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.890+0000 7f281e400640 1 -- 172.21.15.3:0/673032439 >> 172.21.15.3:0/673032439 conn(0x7f28180fcb30 msgr2=0x7f28180fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.890+0000 7f281e400640 1 -- 172.21.15.3:0/673032439 shutdown_connections 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.890+0000 7f281e400640 1 -- 172.21.15.3:0/673032439 wait complete. 2024-03-21T08:31:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 Processor -- start 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 -- start start 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 0x7f2818075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 0x7f2818073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2818103170 0x7f2818073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281810d130 con 0x7f28181015c0 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281810cfb0 con 0x7f28181027b0 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.891+0000 7f281e400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281810d2b0 con 0x7f2818103170 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 0x7f2818075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f2817e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 0x7f2818073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 0x7f2818075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44874/0 (socket says 172.21.15.3:44874) 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 -- 172.21.15.3:0/2435310705 learned_addr learned my addr 172.21.15.3:0/2435310705 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f2817e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 0x7f2818073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:44248/0 (socket says 172.21.15.3:44248) 2024-03-21T08:31:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281da00640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2818103170 0x7f2818073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 -- 172.21.15.3:0/2435310705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2818103170 msgr2=0x7f2818073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2818103170 0x7f2818073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 -- 172.21.15.3:0/2435310705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 msgr2=0x7f2818073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 0x7f2818073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 -- 172.21.15.3:0/2435310705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2818074860 con 0x7f28181015c0 2024-03-21T08:31:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.892+0000 7f281d000640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 0x7f2818075450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2808009d20 tx=0x7f280800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.893+0000 7f2815600640 1 -- 172.21.15.3:0/2435310705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808010630 con 0x7f28181015c0 2024-03-21T08:31:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.893+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28181a18d0 con 0x7f28181015c0 2024-03-21T08:31:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.893+0000 7f2815600640 1 -- 172.21.15.3:0/2435310705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2808010c80 con 0x7f28181015c0 2024-03-21T08:31:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.893+0000 7f2815600640 1 -- 172.21.15.3:0/2435310705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808018d10 con 0x7f28181015c0 2024-03-21T08:31:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.893+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28181a1df0 con 0x7f28181015c0 2024-03-21T08:31:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.894+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28181088e0 con 0x7f28181015c0 2024-03-21T08:31:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.896+0000 7f2815600640 1 -- 172.21.15.3:0/2435310705 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f28080107d0 con 0x7f28181015c0 2024-03-21T08:31:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.897+0000 7f2815600640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f27fc0a2f10 0x7f27fc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.897+0000 7f2815600640 1 -- 172.21.15.3:0/2435310705 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f28080d00b0 con 0x7f28181015c0 2024-03-21T08:31:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.898+0000 7f2817e00640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f27fc0a2f10 0x7f27fc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:19.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.898+0000 7f2817e00640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f27fc0a2f10 0x7f27fc0a53d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f280002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:19.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:19.901+0000 7f2815600640 1 -- 172.21.15.3:0/2435310705 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2808053740 con 0x7f28181015c0 2024-03-21T08:31:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.103+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28180622a0 con 0x7f28181015c0 2024-03-21T08:31:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.107+0000 7f2815600640 1 -- 172.21.15.3:0/2435310705 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7f2808082140 con 0x7f28181015c0 2024-03-21T08:31:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:20.109 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:31:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f27fc0a2f10 msgr2=0x7f27fc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f27fc0a2f10 0x7f27fc0a53d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f280002c7e0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 msgr2=0x7f2818075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 0x7f2818075450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2808009d20 tx=0x7f280800c6a0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 shutdown_connections 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f27fc0a2f10 0x7f27fc0a53d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2818103170 0x7f2818073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28181027b0 0x7f2818073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 --2- 172.21.15.3:0/2435310705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28181015c0 0x7f2818075450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.110+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 >> 172.21.15.3:0/2435310705 conn(0x7f28180fcb30 msgr2=0x7f2818105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.111+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 shutdown_connections 2024-03-21T08:31:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.111+0000 7f281e400640 1 -- 172.21.15.3:0/2435310705 wait complete. 2024-03-21T08:31:20.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T08:28:36.113879+0000', 'modified': '2024-03-21T08:29:09.323523+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34472, 'mds_1': 34439, 'mds_2': 34409}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6835/1859190507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 1859190507}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 1859190507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/2662792128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 2662792128}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 2662792128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6839/1010033316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1010033316}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1010033316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34409': {'gid': 34409, 'name': 'k', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.84:6833/4091779255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 4091779255}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 4091779255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34439': {'gid': 34439, 'name': 'e', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.84:6835/2576104068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 2576104068}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 2576104068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.84:6837/3332280865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 3332280865}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 3332280865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34472, 'qdb_cluster': [34439, 34409, 34472]} 2024-03-21T08:31:20.181 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-21T08:31:20.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-21T08:31:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.413+0000 7f60cac00640 1 -- 172.21.15.3:0/3239065286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c4104ac0 msgr2=0x7f60c4104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.413+0000 7f60cac00640 1 --2- 172.21.15.3:0/3239065286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c4104ac0 0x7f60c4104f40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009900 tx=0x7f60ac02bc50 comp rx=0 tx=0).stop 2024-03-21T08:31:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.413+0000 7f60cac00640 1 -- 172.21.15.3:0/3239065286 shutdown_connections 2024-03-21T08:31:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.413+0000 7f60cac00640 1 --2- 172.21.15.3:0/3239065286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60c4105480 0x7f60c410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.413+0000 7f60cac00640 1 --2- 172.21.15.3:0/3239065286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c4104ac0 0x7f60c4104f40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.413+0000 7f60cac00640 1 --2- 172.21.15.3:0/3239065286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60c41038d0 0x7f60c4103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.413+0000 7f60cac00640 1 -- 172.21.15.3:0/3239065286 >> 172.21.15.3:0/3239065286 conn(0x7f60c40fee60 msgr2=0x7f60c4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.414+0000 7f60cac00640 1 -- 172.21.15.3:0/3239065286 shutdown_connections 2024-03-21T08:31:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.414+0000 7f60cac00640 1 -- 172.21.15.3:0/3239065286 wait complete. 2024-03-21T08:31:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 Processor -- start 2024-03-21T08:31:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 -- start start 2024-03-21T08:31:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c41038d0 0x7f60c4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60c4104ac0 0x7f60c419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60c4105480 0x7f60c41a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410f400 con 0x7f60c41038d0 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410f280 con 0x7f60c4104ac0 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.415+0000 7f60cac00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410f580 con 0x7f60c4105480 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c41038d0 0x7f60c4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c41038d0 0x7f60c4199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44904/0 (socket says 172.21.15.3:44904) 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 -- 172.21.15.3:0/1609625312 learned_addr learned my addr 172.21.15.3:0/1609625312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3400640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60c4104ac0 0x7f60c419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 -- 172.21.15.3:0/1609625312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60c4105480 msgr2=0x7f60c41a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c8c00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60c4105480 0x7f60c41a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60c4105480 0x7f60c41a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 -- 172.21.15.3:0/1609625312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60c4104ac0 msgr2=0x7f60c419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60c4104ac0 0x7f60c419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3e00640 1 -- 172.21.15.3:0/1609625312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c41a1770 con 0x7f60c41038d0 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c3400640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60c4104ac0 0x7f60c419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.416+0000 7f60c8c00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60c4105480 0x7f60c41a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.417+0000 7f60c3e00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c41038d0 0x7f60c4199b50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e9c0 tx=0x7f60b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.417+0000 7f60c0c00640 1 -- 172.21.15.3:0/1609625312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b400cd80 con 0x7f60c41038d0 2024-03-21T08:31:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.417+0000 7f60c0c00640 1 -- 172.21.15.3:0/1609625312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60b40035f0 con 0x7f60c41038d0 2024-03-21T08:31:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.417+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c41a1a60 con 0x7f60c41038d0 2024-03-21T08:31:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.417+0000 7f60c0c00640 1 -- 172.21.15.3:0/1609625312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4010690 con 0x7f60c41038d0 2024-03-21T08:31:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.417+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c41a1fb0 con 0x7f60c41038d0 2024-03-21T08:31:20.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.420+0000 7f60cd9f4740 1 Processor -- start 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.420+0000 7f60cd9f4740 1 -- start start 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.420+0000 7f60c0c00640 1 -- 172.21.15.3:0/1609625312 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f60b40037a0 con 0x7f60c41038d0 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680f9cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680fa2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fa4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680f01c90 con 0x558680fa2700 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680eaa980 con 0x558680f9d590 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680efdd30 con 0x558680f9cba0 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60c3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fa4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60c3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fa4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44910/0 (socket says 172.21.15.3:44910) 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60c3400640 1 -- 172.21.15.3:0/2839732035 learned_addr learned my addr 172.21.15.3:0/2839732035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60c3e00640 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680fa2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60c8c00640 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680f9cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60c3400640 1 -- 172.21.15.3:0/2839732035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 msgr2=0x558680f9cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.421+0000 7f60c3400640 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680f9cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60c3400640 1 -- 172.21.15.3:0/2839732035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 msgr2=0x558680fa2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60c3400640 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680fa2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60c3400640 1 -- 172.21.15.3:0/2839732035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558680ec2510 con 0x558680fa2700 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60c3400640 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fa4b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b79998baae1658 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60c0c00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f60a80a2f20 0x7f60a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60b9200640 1 -- 172.21.15.3:0/2839732035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac044020 con 0x558680fa2700 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60c0c00640 1 -- 172.21.15.3:0/1609625312 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f60b40cfd70 con 0x7f60c41038d0 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60b9200640 1 -- 172.21.15.3:0/2839732035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60ac02cc40 con 0x558680fa2700 2024-03-21T08:31:20.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60b9200640 1 -- 172.21.15.3:0/2839732035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac004920 con 0x558680fa2700 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.422+0000 7f60c8c00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f60a80a2f20 0x7f60a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 -- 172.21.15.3:0/2839732035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 msgr2=0x558680fa4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fa4b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac005d50 comp rx=0 tx=0).stop 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60c8c00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f60a80a2f20 0x7f60a80a53e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x558680f9ddf0 tx=0x7f60bc009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 -- 172.21.15.3:0/2839732035 shutdown_connections 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fa4b90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680fa2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/2839732035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680f9cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 -- 172.21.15.3:0/2839732035 >> 172.21.15.3:0/2839732035 conn(0x558680f98110 msgr2=0x558680f98510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 -- 172.21.15.3:0/2839732035 shutdown_connections 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.423+0000 7f60cd9f4740 1 -- 172.21.15.3:0/2839732035 wait complete. 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.424+0000 7f60cd9f4740 1 -- start start 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.424+0000 7f60cd9f4740 1 Processor -- start 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.424+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9cba0 0x558680fa7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.424+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9d590 0x558680fa7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fa5650 con 0x558680f9d590 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680ebddd0 con 0x558680f9cba0 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fa57d0 con 0x558680fa2700 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60cd9f4740 10 client.0 _ll_register_callbacks cb 0x558680c15ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9d590 0x558680fa7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9d590 0x558680fa7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44926/0 (socket says 172.21.15.3:44926) 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 -- 172.21.15.3:0/3597588879 learned_addr learned my addr 172.21.15.3:0/3597588879 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3e00640 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9cba0 0x558680fa7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:20 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1926299036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:20.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:20 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2435310705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:20.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c8c00640 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 -- 172.21.15.3:0/3597588879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 msgr2=0x558680fae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 -- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9cba0 msgr2=0x558680fa7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9cba0 0x558680fa7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.425+0000 7f60c3400640 1 -- 172.21.15.3:0/3597588879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558680c162d0 con 0x558680f9d590 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60c3400640 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9d590 0x558680fa7580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0096c0 tx=0x7f60ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b2755d15a63785 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60b1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60b1600640 1 -- 172.21.15.3:0/3597588879 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac04d070 con 0x558680f9d590 2024-03-21T08:31:20.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60b1600640 1 -- 172.21.15.3:0/3597588879 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f60ac003cd0 con 0x558680f9d590 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60ac02cc40 con 0x558680f9d590 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60b1600640 1 -- 172.21.15.3:0/3597588879 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60ac059050 con 0x558680f9d590 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60cd9f4740 10 client.15417 fetch_fsmap learned FSMap version 22 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.426+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558680ce06b0 con 0x558680f9d590 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60b1600640 1 -- 172.21.15.3:0/3597588879 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60ac02cc40 con 0x558680f9d590 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60b1600640 1 -- 172.21.15.3:0/3597588879 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f60ac034440 con 0x558680f9d590 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60cd9f4740 10 client.15417 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60cd9f4740 10 client.15417 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60cd9f4740 20 client.15417 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558680fb2720 0x558680fb7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60cd9f4740 4 client.15417 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.427+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558680ea3520 con 0x558680fb2720 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.428+0000 7f60c3e00640 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558680fb2720 0x558680fb7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.428+0000 7f60c3e00640 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558680fb2720 0x558680fb7790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d01985a3b6c6b1b5 server_cookie=307df31c77d090c7 in_seq=0 out_seq=0 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.428+0000 7f60b1600640 10 client.15417 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60b1600640 1 -- 172.21.15.3:0/3597588879 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558680ea3520 con 0x558680fb2720 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60b1600640 10 client.15417 handle_command_reply: tid=0 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 client.15417 shutdown 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9d590 msgr2=0x558680fa7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9d590 0x558680fa7580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0096c0 tx=0x7f60ac0038a0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 shutdown_connections 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558680fb2720 0x558680fb7790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9d590 0x558680fa7580 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60b1600640 0 client.15417 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3597588879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9cba0 0x558680fa7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.430+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 >> 172.21.15.3:0/3597588879 conn(0x558680f98110 msgr2=0x558680f97310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.431+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 shutdown_connections 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.431+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3597588879 wait complete. 2024-03-21T08:31:20.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.431+0000 7f60cd9f4740 20 client.15417 trim_cache size 0 max 16384 2024-03-21T08:31:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.449+0000 7f60cd9f4740 1 Processor -- start 2024-03-21T08:31:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.449+0000 7f60cd9f4740 1 -- start start 2024-03-21T08:31:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9cba0 0x558680f9ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680f9efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fb2b30 con 0x558680f9cba0 2024-03-21T08:31:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fb3190 con 0x558680f9d590 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fb37f0 con 0x558680fa2700 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9cba0 0x558680f9ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680f9efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9cba0 0x558680f9ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44936/0 (socket says 172.21.15.3:44936) 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c3e00640 1 -- 172.21.15.3:0/1520383154 learned_addr learned my addr 172.21.15.3:0/1520383154 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c8c00640 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c3e00640 1 -- 172.21.15.3:0/1520383154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 msgr2=0x558680fae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c3e00640 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.450+0000 7f60c3e00640 1 -- 172.21.15.3:0/1520383154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 msgr2=0x558680f9efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.451+0000 7f60c3e00640 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680f9efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.451+0000 7f60c3e00640 1 -- 172.21.15.3:0/1520383154 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558680f98e80 con 0x558680f9cba0 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.451+0000 7f60c3e00640 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9cba0 0x558680f9ea90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f60b40989d0 tx=0x7f60b4003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd45a2bd786b6653 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.451+0000 7f60b3e00640 1 -- 172.21.15.3:0/1520383154 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4003b30 con 0x558680f9cba0 2024-03-21T08:31:20.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.451+0000 7f60b3e00640 1 -- 172.21.15.3:0/1520383154 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60b409c700 con 0x558680f9cba0 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.451+0000 7f60b3e00640 1 -- 172.21.15.3:0/1520383154 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4098670 con 0x558680f9cba0 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 -- 172.21.15.3:0/1520383154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9cba0 msgr2=0x558680f9ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9cba0 0x558680f9ea90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f60b40989d0 tx=0x7f60b4003e30 comp rx=0 tx=0).stop 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 -- 172.21.15.3:0/1520383154 shutdown_connections 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680fa2700 0x558680fae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680f9efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/1520383154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680f9cba0 0x558680f9ea90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 -- 172.21.15.3:0/1520383154 >> 172.21.15.3:0/1520383154 conn(0x558680f98110 msgr2=0x558680f984f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 -- 172.21.15.3:0/1520383154 shutdown_connections 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.452+0000 7f60cd9f4740 1 -- 172.21.15.3:0/1520383154 wait complete. 2024-03-21T08:31:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 -- start start 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 Processor -- start 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680faa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680faacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fadc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fb44b0 con 0x558680fa2700 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fb3e50 con 0x558680f9d590 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680fb4b10 con 0x558680f9cba0 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60cd9f4740 10 client.0 _ll_register_callbacks cb 0x7f60ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.453+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fadc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fadc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44940/0 (socket says 172.21.15.3:44940) 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 -- 172.21.15.3:0/3016009549 learned_addr learned my addr 172.21.15.3:0/3016009549 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c3400640 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680faacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c3e00640 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680faa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 -- 172.21.15.3:0/3016009549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 msgr2=0x558680faa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680faa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 -- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 msgr2=0x558680faacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680faacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 -- 172.21.15.3:0/3016009549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558680fae210 con 0x558680fa2700 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60c8c00640 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fadc20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0129b0 tx=0x7f60bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e3b3b830cc82f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.454+0000 7f60b1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60b1600640 1 -- 172.21.15.3:0/3016009549 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc021dd0 con 0x558680fa2700 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60b409c700 con 0x558680fa2700 2024-03-21T08:31:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60b1600640 1 -- 172.21.15.3:0/3016009549 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f60bc01f450 con 0x558680fa2700 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60b1600640 1 -- 172.21.15.3:0/3016009549 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60bc01f650 con 0x558680fa2700 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 10 client.15423 fetch_fsmap learned FSMap version 22 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55868100f390 con 0x558680fa2700 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60b1600640 1 -- 172.21.15.3:0/3016009549 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60b409c700 con 0x558680fa2700 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60b1600640 1 -- 172.21.15.3:0/3016009549 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f60bc028070 con 0x558680fa2700 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 10 client.15423 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 10 client.15423 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 20 client.15423 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558681013ac0 0x55868100f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 4 client.15423 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.455+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558680f0f1d0 con 0x558681013ac0 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.456+0000 7f60c3e00640 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558681013ac0 0x55868100f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60c3e00640 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558681013ac0 0x55868100f680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d85edaeb158b887 server_cookie=38d571e326fb13dc in_seq=0 out_seq=0 2024-03-21T08:31:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60b1600640 10 client.15423 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60b1600640 1 -- 172.21.15.3:0/3016009549 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x558680f0f1d0 con 0x558681013ac0 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60b1600640 10 client.15423 handle_command_reply: tid=0 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 client.15423 shutdown 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 msgr2=0x558680fadc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fadc20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0129b0 tx=0x7f60bc004a40 comp rx=0 tx=0).stop 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 shutdown_connections 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x558681013ac0 0x55868100f680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558680fa2700 0x558680fadc20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558680f9d590 0x558680faacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 --2- 172.21.15.3:0/3016009549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558680f9cba0 0x558680faa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60b1600640 0 client.15423 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 >> 172.21.15.3:0/3016009549 conn(0x558680f98110 msgr2=0x558680f9ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.457+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 shutdown_connections 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cd9f4740 1 -- 172.21.15.3:0/3016009549 wait complete. 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cd9f4740 20 client.15423 trim_cache size 0 max 16384 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 34409, 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-03-21T08:31:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 132.142365794, 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 96, 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 94, 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 208.4252931 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:31:20.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f60a80a2f20 msgr2=0x7f60a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f60a80a2f20 0x7f60a80a53e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x558680f9ddf0 tx=0x7f60bc009590 comp rx=0 tx=0).stop 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c41038d0 msgr2=0x7f60c4199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c41038d0 0x7f60c4199b50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e9c0 tx=0x7f60b400ee80 comp rx=0 tx=0).stop 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 shutdown_connections 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f60a80a2f20 0x7f60a80a53e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60c4105480 0x7f60c41a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60c4104ac0 0x7f60c419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 --2- 172.21.15.3:0/1609625312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c41038d0 0x7f60c4199b50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.458+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 >> 172.21.15.3:0/1609625312 conn(0x7f60c40fee60 msgr2=0x7f60c4107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.459+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 shutdown_connections 2024-03-21T08:31:20.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.459+0000 7f60cac00640 1 -- 172.21.15.3:0/1609625312 wait complete. 2024-03-21T08:31:20.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-21T08:31:20.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:20 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1926299036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:20.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:20 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2435310705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:20.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:20 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1926299036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:20.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:20 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2435310705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:20.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 -- 172.21.15.3:0/2323444989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 msgr2=0x7f99e4103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 --2- 172.21.15.3:0/2323444989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4103b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009900 tx=0x7f99d402bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:20.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 -- 172.21.15.3:0/2323444989 shutdown_connections 2024-03-21T08:31:20.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 --2- 172.21.15.3:0/2323444989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e41052b0 0x7f99e410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 --2- 172.21.15.3:0/2323444989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e41048f0 0x7f99e4104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 --2- 172.21.15.3:0/2323444989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4103b20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 -- 172.21.15.3:0/2323444989 >> 172.21.15.3:0/2323444989 conn(0x7f99e40fec70 msgr2=0x7f99e41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 -- 172.21.15.3:0/2323444989 shutdown_connections 2024-03-21T08:31:20.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.707+0000 7f99e3200640 1 -- 172.21.15.3:0/2323444989 wait complete. 2024-03-21T08:31:20.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.708+0000 7f99e3200640 1 Processor -- start 2024-03-21T08:31:20.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.708+0000 7f99e3200640 1 -- start start 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e41048f0 0x7f99e4199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e41052b0 0x7f99e41a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e410f280 con 0x7f99e4103700 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e410f100 con 0x7f99e41052b0 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e410f400 con 0x7f99e41048f0 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44958/0 (socket says 172.21.15.3:44958) 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e1e00640 1 -- 172.21.15.3:0/438348431 learned_addr learned my addr 172.21.15.3:0/438348431 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e2800640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e41052b0 0x7f99e41a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99dbe00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e41048f0 0x7f99e4199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.709+0000 7f99e1e00640 1 -- 172.21.15.3:0/438348431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e41048f0 msgr2=0x7f99e4199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99e1e00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e41048f0 0x7f99e4199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99e1e00640 1 -- 172.21.15.3:0/438348431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e41052b0 msgr2=0x7f99e41a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99e1e00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e41052b0 0x7f99e41a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99e1e00640 1 -- 172.21.15.3:0/438348431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e41a1590 con 0x7f99e4103700 2024-03-21T08:31:20.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99dbe00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e41048f0 0x7f99e4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:20.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99e1e00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4199970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f99d4000c00 tx=0x7f99d40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99daa00640 1 -- 172.21.15.3:0/438348431 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4043020 con 0x7f99e4103700 2024-03-21T08:31:20.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e41a1820 con 0x7f99e4103700 2024-03-21T08:31:20.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99daa00640 1 -- 172.21.15.3:0/438348431 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99d40044c0 con 0x7f99e4103700 2024-03-21T08:31:20.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99daa00640 1 -- 172.21.15.3:0/438348431 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4003c90 con 0x7f99e4103700 2024-03-21T08:31:20.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.710+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e41a1d70 con 0x7f99e4103700 2024-03-21T08:31:20.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99daa00640 1 -- 172.21.15.3:0/438348431 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f99d4046020 con 0x7f99e4103700 2024-03-21T08:31:20.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 Processor -- start 2024-03-21T08:31:20.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 -- start start 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3cd80 0x5653bbe3d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3d770 0x5653bbe42270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 0x5653bbe44d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbda1c90 con 0x5653bbe3d770 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbd4a980 con 0x5653bbe3cd80 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99eb99f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbd9dd30 con 0x5653bbe428e0 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3d770 0x5653bbe42270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99e2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3cd80 0x5653bbe3d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99e2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3cd80 0x5653bbe3d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:44336/0 (socket says 172.21.15.3:44336) 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.714+0000 7f99e2800640 1 -- 172.21.15.3:0/2690211476 learned_addr learned my addr 172.21.15.3:0/2690211476 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99dbe00640 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 0x5653bbe44d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99e2800640 1 -- 172.21.15.3:0/2690211476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 msgr2=0x5653bbe44d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99e2800640 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 0x5653bbe44d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99e2800640 1 -- 172.21.15.3:0/2690211476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3d770 msgr2=0x5653bbe42270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99e2800640 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3d770 0x5653bbe42270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99e2800640 1 -- 172.21.15.3:0/2690211476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653bbd62510 con 0x5653bbe3cd80 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99daa00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f99c80a2f20 0x7f99c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99e2800640 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3cd80 0x5653bbe3d1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99dc010370 tx=0x7f99dc010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4bc5f7503503395 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.715+0000 7f99daa00640 1 -- 172.21.15.3:0/438348431 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f99d40f1b20 con 0x7f99e4103700 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99e1e00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f99c80a2f20 0x7f99c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99d3000640 1 -- 172.21.15.3:0/2690211476 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc016070 con 0x5653bbe3cd80 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99d3000640 1 -- 172.21.15.3:0/2690211476 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99dc004d20 con 0x5653bbe3cd80 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99d3000640 1 -- 172.21.15.3:0/2690211476 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc005370 con 0x5653bbe3cd80 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99eb99f740 1 -- 172.21.15.3:0/2690211476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3cd80 msgr2=0x5653bbe3d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99e1e00640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f99c80a2f20 0x7f99c80a53e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99d40ef180 tx=0x7f99d40de040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99eb99f740 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3cd80 0x5653bbe3d1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99dc010370 tx=0x7f99dc010830 comp rx=0 tx=0).stop 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.716+0000 7f99eb99f740 1 -- 172.21.15.3:0/2690211476 shutdown_connections 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 0x5653bbe44d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3d770 0x5653bbe42270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 --2- 172.21.15.3:0/2690211476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3cd80 0x5653bbe3d1a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 -- 172.21.15.3:0/2690211476 >> 172.21.15.3:0/2690211476 conn(0x5653bbe382f0 msgr2=0x5653bbe386f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 -- 172.21.15.3:0/2690211476 shutdown_connections 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 -- 172.21.15.3:0/2690211476 wait complete. 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 -- start start 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.717+0000 7f99eb99f740 1 Processor -- start 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3d770 0x5653bbe477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 0x5653bbe4e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99eb99f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe45890 con 0x5653bbe3cd80 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99eb99f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbd5ddd0 con 0x5653bbe3d770 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99eb99f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe45a10 con 0x5653bbe428e0 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99eb99f740 10 client.0 _ll_register_callbacks cb 0x5653bbab5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44982/0 (socket says 172.21.15.3:44982) 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 -- 172.21.15.3:0/157484876 learned_addr learned my addr 172.21.15.3:0/157484876 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99dbe00640 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3d770 0x5653bbe477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 -- 172.21.15.3:0/157484876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 msgr2=0x5653bbe4e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 0x5653bbe4e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 -- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3d770 msgr2=0x5653bbe477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3d770 0x5653bbe477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.718+0000 7f99e2800640 1 -- 172.21.15.3:0/157484876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653bbab62d0 con 0x5653bbe3cd80 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99e2800640 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009bf0 tx=0x7f99dc004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16d5d5c4066156d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99c7400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99c7400640 1 -- 172.21.15.3:0/157484876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc016070 con 0x5653bbe3cd80 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99c7400640 1 -- 172.21.15.3:0/157484876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f99dc012710 con 0x5653bbe3cd80 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99dc004d20 con 0x5653bbe3cd80 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99c7400640 1 -- 172.21.15.3:0/157484876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99dc012910 con 0x5653bbe3cd80 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99eb99f740 10 client.15435 fetch_fsmap learned FSMap version 22 2024-03-21T08:31:20.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.719+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5653bbb806b0 con 0x5653bbe3cd80 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99c7400640 1 -- 172.21.15.3:0/157484876 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99dc004d20 con 0x5653bbe3cd80 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99c7400640 1 -- 172.21.15.3:0/157484876 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f99dc00f040 con 0x5653bbe3cd80 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99eb99f740 10 client.15435 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99eb99f740 10 client.15435 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99eb99f740 20 client.15435 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99eb99f740 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbe53570 0x5653bbe579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99eb99f740 4 client.15435 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:31:20.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.720+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653bbd43520 con 0x5653bbe53570 2024-03-21T08:31:20.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.721+0000 7f99dbe00640 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbe53570 0x5653bbe579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.721+0000 7f99dbe00640 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbe53570 0x5653bbe579f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10155ad693a857a7 server_cookie=209b44d06118f8de in_seq=0 out_seq=0 2024-03-21T08:31:20.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.721+0000 7f99c7400640 10 client.15435 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:31:20.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99c7400640 1 -- 172.21.15.3:0/157484876 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5653bbd43520 con 0x5653bbe53570 2024-03-21T08:31:20.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99c7400640 10 client.15435 handle_command_reply: tid=0 2024-03-21T08:31:20.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 client.15435 shutdown 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 msgr2=0x5653bbe47290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009bf0 tx=0x7f99dc004b80 comp rx=0 tx=0).stop 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 shutdown_connections 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbe53570 0x5653bbe579f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe428e0 0x5653bbe4e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe3d770 0x5653bbe477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 --2- 172.21.15.3:0/157484876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99c7400640 0 client.15435 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 >> 172.21.15.3:0/157484876 conn(0x5653bbe382f0 msgr2=0x5653bbe374f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 shutdown_connections 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 1 -- 172.21.15.3:0/157484876 wait complete. 2024-03-21T08:31:20.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.723+0000 7f99eb99f740 20 client.15435 trim_cache size 0 max 16384 2024-03-21T08:31:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 Processor -- start 2024-03-21T08:31:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 -- start start 2024-03-21T08:31:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 0x5653bbe44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe52d90 con 0x5653bbe3cd80 2024-03-21T08:31:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe533f0 con 0x5653bbe428e0 2024-03-21T08:31:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.742+0000 7f99eb99f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe53a50 con 0x5653bbe3d770 2024-03-21T08:31:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44984/0 (socket says 172.21.15.3:44984) 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 -- 172.21.15.3:0/4208239481 learned_addr learned my addr 172.21.15.3:0/4208239481 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99dbe00640 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 -- 172.21.15.3:0/4208239481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 msgr2=0x5653bbe4e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 -- 172.21.15.3:0/4208239481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 msgr2=0x5653bbe44cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 0x5653bbe44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 -- 172.21.15.3:0/4208239481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653bbe2a170 con 0x5653bbe3cd80 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.743+0000 7f99e2800640 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47730 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f99dc006010 tx=0x7f99dc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26839eebfc5f5f06 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99d2600640 1 -- 172.21.15.3:0/4208239481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc016040 con 0x5653bbe3cd80 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99d2600640 1 -- 172.21.15.3:0/4208239481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99dc022070 con 0x5653bbe3cd80 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99d2600640 1 -- 172.21.15.3:0/4208239481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc005630 con 0x5653bbe3cd80 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99eb99f740 1 -- 172.21.15.3:0/4208239481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 msgr2=0x5653bbe47730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47730 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f99dc006010 tx=0x7f99dc009ca0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99eb99f740 1 -- 172.21.15.3:0/4208239481 shutdown_connections 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 0x5653bbe44cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4208239481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe47730 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.744+0000 7f99eb99f740 1 -- 172.21.15.3:0/4208239481 >> 172.21.15.3:0/4208239481 conn(0x5653bbe382f0 msgr2=0x5653bbe3e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.745+0000 7f99eb99f740 1 -- 172.21.15.3:0/4208239481 shutdown_connections 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.745+0000 7f99eb99f740 1 -- 172.21.15.3:0/4208239481 wait complete. 2024-03-21T08:31:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.745+0000 7f99eb99f740 1 -- start start 2024-03-21T08:31:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.745+0000 7f99eb99f740 1 Processor -- start 2024-03-21T08:31:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe49f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99eb99f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 0x5653bbe4d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99eb99f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe54710 con 0x5653bbe3cd80 2024-03-21T08:31:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99eb99f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe540b0 con 0x5653bbe428e0 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99eb99f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653bbe54d70 con 0x5653bbe3d770 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99eb99f740 10 client.0 _ll_register_callbacks cb 0x7f99d40f5430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe49f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe49f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44998/0 (socket says 172.21.15.3:44998) 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 -- 172.21.15.3:0/4015841305 learned_addr learned my addr 172.21.15.3:0/4015841305 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 -- 172.21.15.3:0/4015841305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 msgr2=0x5653bbe4a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99dbe00640 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 -- 172.21.15.3:0/4015841305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 msgr2=0x5653bbe4d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 0x5653bbe4d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.746+0000 7f99e2800640 1 -- 172.21.15.3:0/4015841305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653bbe4dab0 con 0x5653bbe3cd80 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99e2800640 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe49f60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f99dc004ae0 tx=0x7f99dc009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a26d5e8eeb1736c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99c7400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99c7400640 1 -- 172.21.15.3:0/4015841305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc016040 con 0x5653bbe3cd80 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99dc022070 con 0x5653bbe3cd80 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99c7400640 1 -- 172.21.15.3:0/4015841305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f99dc005630 con 0x5653bbe3cd80 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99c7400640 1 -- 172.21.15.3:0/4015841305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99dc005830 con 0x5653bbe3cd80 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99eb99f740 10 client.15441 fetch_fsmap learned FSMap version 22 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5653bbeaebd0 con 0x5653bbe3cd80 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.747+0000 7f99c7400640 1 -- 172.21.15.3:0/4015841305 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99dc022070 con 0x5653bbe3cd80 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99c7400640 1 -- 172.21.15.3:0/4015841305 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f99dc032070 con 0x5653bbe3cd80 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99eb99f740 10 client.15441 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99eb99f740 10 client.15441 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99eb99f740 20 client.15441 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbeb3020 0x5653bbeaedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99eb99f740 4 client.15441 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5653bbdaf1d0 con 0x5653bbeb3020 2024-03-21T08:31:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.748+0000 7f99dbe00640 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbeb3020 0x5653bbeaedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.749+0000 7f99dbe00640 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbeb3020 0x5653bbeaedf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96fa8c9b5a629ddc server_cookie=ce6a7fda2a38e73c in_seq=0 out_seq=0 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.749+0000 7f99c7400640 10 client.15441 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.749+0000 7f99c7400640 1 -- 172.21.15.3:0/4015841305 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5653bbdaf1d0 con 0x5653bbeb3020 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99c7400640 10 client.15441 handle_command_reply: tid=0 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 client.15441 shutdown 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 msgr2=0x5653bbe49f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe49f60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f99dc004ae0 tx=0x7f99dc009a60 comp rx=0 tx=0).stop 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 shutdown_connections 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653bbe428e0 0x5653bbe4d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x5653bbeb3020 0x5653bbeaedf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653bbe3d770 0x5653bbe4a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 --2- 172.21.15.3:0/4015841305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5653bbe3cd80 0x5653bbe49f60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99c7400640 0 client.15441 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 >> 172.21.15.3:0/4015841305 conn(0x5653bbe382f0 msgr2=0x5653bbe45de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 shutdown_connections 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 1 -- 172.21.15.3:0/4015841305 wait complete. 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.750+0000 7f99eb99f740 20 client.15441 trim_cache size 0 max 16384 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 34439, 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-03-21T08:31:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 133.45307678500001, 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 96, 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 94, 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 196.78762681200001 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f99c80a2f20 msgr2=0x7f99c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f99c80a2f20 0x7f99c80a53e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99d40ef180 tx=0x7f99d40de040 comp rx=0 tx=0).stop 2024-03-21T08:31:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 msgr2=0x7f99e4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4199970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f99d4000c00 tx=0x7f99d40040c0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 shutdown_connections 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e41052b0 0x7f99e41a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f99c80a2f20 0x7f99c80a53e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99e41048f0 0x7f99e4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 --2- 172.21.15.3:0/438348431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99e4103700 0x7f99e4199970 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 >> 172.21.15.3:0/438348431 conn(0x7f99e40fec70 msgr2=0x7f99e4107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 shutdown_connections 2024-03-21T08:31:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.751+0000 7f99e3200640 1 -- 172.21.15.3:0/438348431 wait complete. 2024-03-21T08:31:20.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.996+0000 7f8c89800640 1 -- 172.21.15.3:0/2882586425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84107ce0 msgr2=0x7f8c84108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.996+0000 7f8c89800640 1 --2- 172.21.15.3:0/2882586425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84107ce0 0x7f8c84108100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009900 tx=0x7f8c7402bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.997+0000 7f8c89800640 1 -- 172.21.15.3:0/2882586425 shutdown_connections 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.997+0000 7f8c89800640 1 --2- 172.21.15.3:0/2882586425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c84109890 0x7f8c84110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.997+0000 7f8c89800640 1 --2- 172.21.15.3:0/2882586425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c84108ed0 0x7f8c84109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.997+0000 7f8c89800640 1 --2- 172.21.15.3:0/2882586425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84107ce0 0x7f8c84108100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.997+0000 7f8c89800640 1 -- 172.21.15.3:0/2882586425 >> 172.21.15.3:0/2882586425 conn(0x7f8c84076c90 msgr2=0x7f8c840790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.997+0000 7f8c89800640 1 -- 172.21.15.3:0/2882586425 shutdown_connections 2024-03-21T08:31:20.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.997+0000 7f8c89800640 1 -- 172.21.15.3:0/2882586425 wait complete. 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 Processor -- start 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 -- start start 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c84107ce0 0x7f8c8419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c84108ed0 0x7f8c8419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84109890 0x7f8c841a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c84113830 con 0x7f8c84109890 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841136b0 con 0x7f8c84107ce0 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c89800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841139b0 con 0x7f8c84108ed0 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c83400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84109890 0x7f8c841a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c83400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84109890 0x7f8c841a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45016/0 (socket says 172.21.15.3:45016) 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.998+0000 7f8c83400640 1 -- 172.21.15.3:0/703233963 learned_addr learned my addr 172.21.15.3:0/703233963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c82a00640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c84107ce0 0x7f8c8419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c7a000640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c84108ed0 0x7f8c8419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c83400640 1 -- 172.21.15.3:0/703233963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c84108ed0 msgr2=0x7f8c8419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c83400640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c84108ed0 0x7f8c8419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c83400640 1 -- 172.21.15.3:0/703233963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c84107ce0 msgr2=0x7f8c8419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c83400640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c84107ce0 0x7f8c8419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c83400640 1 -- 172.21.15.3:0/703233963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c841a5b50 con 0x7f8c84109890 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c82a00640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c84107ce0 0x7f8c8419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c83400640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84109890 0x7f8c841a54e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00eaa0 tx=0x7f8c7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c7be00640 1 -- 172.21.15.3:0/703233963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00cdb0 con 0x7f8c84109890 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:20.999+0000 7f8c7be00640 1 -- 172.21.15.3:0/703233963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c7c005020 con 0x7f8c84109890 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.000+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c841a5e40 con 0x7f8c84109890 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.000+0000 7f8c7be00640 1 -- 172.21.15.3:0/703233963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0116d0 con 0x7f8c84109890 2024-03-21T08:31:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.000+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c841a6300 con 0x7f8c84109890 2024-03-21T08:31:21.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.004+0000 7f8c8c5c6740 1 Processor -- start 2024-03-21T08:31:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.004+0000 7f8c7be00640 1 -- 172.21.15.3:0/703233963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8c7c0051d0 con 0x7f8c84109890 2024-03-21T08:31:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.004+0000 7f8c8c5c6740 1 -- start start 2024-03-21T08:31:21.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce2ba0 0x55a722ce2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722ce8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 0x55a722ceab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722c47c90 con 0x55a722ce3590 2024-03-21T08:31:21.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722bf0980 con 0x55a722ce2ba0 2024-03-21T08:31:21.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722c43d30 con 0x55a722ce8700 2024-03-21T08:31:21.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722ce8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722ce8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45030/0 (socket says 172.21.15.3:45030) 2024-03-21T08:31:21.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c82a00640 1 -- 172.21.15.3:0/2799419643 learned_addr learned my addr 172.21.15.3:0/2799419643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:21.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c7a000640 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce2ba0 0x55a722ce2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.005+0000 7f8c7a000640 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 0x55a722ceab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c82a00640 1 -- 172.21.15.3:0/2799419643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 msgr2=0x55a722ceab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c7be00640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c680a2fa0 0x7f8c680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c82a00640 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 0x55a722ceab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c82a00640 1 -- 172.21.15.3:0/2799419643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce2ba0 msgr2=0x55a722ce2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c82a00640 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce2ba0 0x55a722ce2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c82a00640 1 -- 172.21.15.3:0/2799419643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a722c08510 con 0x55a722ce3590 2024-03-21T08:31:21.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c7a000640 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce2ba0 0x55a722ce2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c7be00640 1 -- 172.21.15.3:0/703233963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8c7c0cfaa0 con 0x7f8c84109890 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c83400640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c680a2fa0 0x7f8c680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c82a00640 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722ce8090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005ab0 tx=0x7f8c74005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3f07628b0247d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.006+0000 7f8c73e00640 1 -- 172.21.15.3:0/2799419643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74044020 con 0x55a722ce3590 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c73e00640 1 -- 172.21.15.3:0/2799419643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c7402cbe0 con 0x55a722ce3590 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c73e00640 1 -- 172.21.15.3:0/2799419643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c740048c0 con 0x55a722ce3590 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/2799419643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 msgr2=0x55a722ce8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722ce8090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005ab0 tx=0x7f8c74005df0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c83400640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c680a2fa0 0x7f8c680a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c098190 tx=0x7f8c7c00cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/2799419643 shutdown_connections 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 0x55a722ceab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722ce8090 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/2799419643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce2ba0 0x55a722ce2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/2799419643 >> 172.21.15.3:0/2799419643 conn(0x55a722cde110 msgr2=0x55a722cde510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.007+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/2799419643 shutdown_connections 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.008+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/2799419643 wait complete. 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.008+0000 7f8c8c5c6740 1 -- start start 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.008+0000 7f8c8c5c6740 1 Processor -- start 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce2ba0 0x55a722ced080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722ced5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 0x55a722cf4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722ceb650 con 0x55a722ce2ba0 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722c03dd0 con 0x55a722ce3590 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722ceb7d0 con 0x55a722ce8700 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c8c5c6740 10 client.0 _ll_register_callbacks cb 0x55a72295bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce2ba0 0x55a722ced080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce2ba0 0x55a722ced080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45046/0 (socket says 172.21.15.3:45046) 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 -- 172.21.15.3:0/1711349591 learned_addr learned my addr 172.21.15.3:0/1711349591 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c82a00640 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722ced5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 -- 172.21.15.3:0/1711349591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 msgr2=0x55a722cf4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 0x55a722cf4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 -- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 msgr2=0x55a722ced5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722ced5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.009+0000 7f8c7a000640 1 -- 172.21.15.3:0/1711349591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a72295c2d0 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c7a000640 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce2ba0 0x55a722ced080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00cfa0 tx=0x7f8c6c00cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361ff8e081998d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c70c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c70c00640 1 -- 172.21.15.3:0/1711349591 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c011dd0 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c70c00640 1 -- 172.21.15.3:0/1711349591 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8c6c004020 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c7402cbe0 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c70c00640 1 -- 172.21.15.3:0/1711349591 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6c024050 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c8c5c6740 10 client.15453 fetch_fsmap learned FSMap version 22 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.010+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a722a266b0 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c70c00640 1 -- 172.21.15.3:0/1711349591 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c7402cbe0 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c70c00640 1 -- 172.21.15.3:0/1711349591 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8c6c00f480 con 0x55a722ce2ba0 2024-03-21T08:31:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c8c5c6740 10 client.15453 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c8c5c6740 10 client.15453 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c8c5c6740 20 client.15453 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722cf98a0 0x55a722cf8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c8c5c6740 4 client.15453 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.011+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a722be9520 con 0x55a722cf98a0 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.012+0000 7f8c82a00640 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722cf98a0 0x55a722cf8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.013+0000 7f8c82a00640 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722cf98a0 0x55a722cf8950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d2142119091ac9 server_cookie=da2af14709e483cc in_seq=0 out_seq=0 2024-03-21T08:31:21.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.013+0000 7f8c70c00640 10 client.15453 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:21.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c70c00640 1 -- 172.21.15.3:0/1711349591 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a722be9520 con 0x55a722cf98a0 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c70c00640 10 client.15453 handle_command_reply: tid=0 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 client.15453 shutdown 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce2ba0 msgr2=0x55a722ced080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce2ba0 0x55a722ced080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00cfa0 tx=0x7f8c6c00cfd0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 shutdown_connections 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722cf98a0 0x55a722cf8950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce8700 0x55a722cf4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722ced5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c70c00640 0 client.15453 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1711349591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce2ba0 0x55a722ced080 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 >> 172.21.15.3:0/1711349591 conn(0x55a722cde110 msgr2=0x55a722cdd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 shutdown_connections 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1711349591 wait complete. 2024-03-21T08:31:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.015+0000 7f8c8c5c6740 20 client.15453 trim_cache size 0 max 16384 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 Processor -- start 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 -- start start 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722cf4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce8700 0x55a722ced1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722cf90c0 con 0x55a722ce8700 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722cf9720 con 0x55a722ce3590 2024-03-21T08:31:21.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.034+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722cf9d80 con 0x55a722ce2ba0 2024-03-21T08:31:21.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722cf4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c7a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce8700 0x55a722ced1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722cf4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:44384/0 (socket says 172.21.15.3:44384) 2024-03-21T08:31:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 -- 172.21.15.3:0/1684131395 learned_addr learned my addr 172.21.15.3:0/1684131395 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c7a000640 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 -- 172.21.15.3:0/1684131395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 msgr2=0x55a722cecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 -- 172.21.15.3:0/1684131395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce8700 msgr2=0x55a722ced1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce8700 0x55a722ced1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c82a00640 1 -- 172.21.15.3:0/1684131395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a722cdee80 con 0x55a722ce3590 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.035+0000 7f8c7a000640 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce8700 0x55a722ced1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.036+0000 7f8c82a00640 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722cf4590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c74005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9127d989d9d8e054 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.036+0000 7f8c73400640 1 -- 172.21.15.3:0/1684131395 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74057070 con 0x55a722ce3590 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.036+0000 7f8c73400640 1 -- 172.21.15.3:0/1684131395 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c740035f0 con 0x55a722ce3590 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c73400640 1 -- 172.21.15.3:0/1684131395 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74044040 con 0x55a722ce3590 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1684131395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 msgr2=0x55a722cf4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722cf4590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c74005fd0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1684131395 shutdown_connections 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce8700 0x55a722ced1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce3590 0x55a722cf4590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/1684131395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1684131395 >> 172.21.15.3:0/1684131395 conn(0x55a722cde110 msgr2=0x55a722cd8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1684131395 shutdown_connections 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.037+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/1684131395 wait complete. 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 -- start start 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 Processor -- start 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cf4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722cf0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce8700 0x55a722cf12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722cfaa40 con 0x55a722ce3590 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722cfa3e0 con 0x55a722ce8700 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a722cfb0a0 con 0x55a722ce2ba0 2024-03-21T08:31:21.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.038+0000 7f8c8c5c6740 10 client.0 _ll_register_callbacks cb 0x7f8c6c012540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722cf0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722cf0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45058/0 (socket says 172.21.15.3:45058) 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 -- 172.21.15.3:0/3349471402 learned_addr learned my addr 172.21.15.3:0/3349471402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c7a000640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce8700 0x55a722cf12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c7a000640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cf4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 -- 172.21.15.3:0/3349471402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 msgr2=0x55a722cf4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cf4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 -- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce8700 msgr2=0x55a722cf12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce8700 0x55a722cf12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 -- 172.21.15.3:0/3349471402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a722cf18a0 con 0x55a722ce3590 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c7a000640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cf4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:21.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c82a00640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722cf0da0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55a722ce8380 tx=0x7f8c74005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34dd6a9a95c9533 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.039+0000 7f8c70c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c70c00640 1 -- 172.21.15.3:0/3349471402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74057070 con 0x55a722ce3590 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c70c00640 1 -- 172.21.15.3:0/3349471402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8c740040c0 con 0x55a722ce3590 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c740035f0 con 0x55a722ce3590 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c70c00640 1 -- 172.21.15.3:0/3349471402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c74044040 con 0x55a722ce3590 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c8c5c6740 10 client.15459 fetch_fsmap learned FSMap version 22 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a722d55940 con 0x55a722ce3590 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c70c00640 1 -- 172.21.15.3:0/3349471402 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c740035f0 con 0x55a722ce3590 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.040+0000 7f8c70c00640 1 -- 172.21.15.3:0/3349471402 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8c74040550 con 0x55a722ce3590 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.041+0000 7f8c8c5c6740 10 client.15459 fetch_fsmap finished waiting for FSMap version 22 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.041+0000 7f8c8c5c6740 10 client.15459 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T08:31:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.041+0000 7f8c8c5c6740 20 client.15459 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.041+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722d598d0 0x55a722d55b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.041+0000 7f8c8c5c6740 4 client.15459 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.041+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a722c551d0 con 0x55a722d598d0 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.041+0000 7f8c7a000640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722d598d0 0x55a722d55b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.042+0000 7f8c7a000640 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722d598d0 0x55a722d55b10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee14b7f16c220a0 server_cookie=eab2de5f5f82e8fc in_seq=0 out_seq=0 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.042+0000 7f8c70c00640 10 client.15459 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.042+0000 7f8c70c00640 1 -- 172.21.15.3:0/3349471402 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a722c551d0 con 0x55a722d598d0 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.042+0000 7f8c70c00640 10 client.15459 handle_command_reply: tid=0 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.042+0000 7f8c8c5c6740 1 client.15459 shutdown 2024-03-21T08:31:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 msgr2=0x55a722cf0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722cf0da0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55a722ce8380 tx=0x7f8c74005d70 comp rx=0 tx=0).stop 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 shutdown_connections 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a722ce8700 0x55a722cf12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a722ce3590 0x55a722cf0da0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a722d598d0 0x55a722d55b10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 --2- 172.21.15.3:0/3349471402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a722ce2ba0 0x55a722cf4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 >> 172.21.15.3:0/3349471402 conn(0x55a722cde110 msgr2=0x55a722cecc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c70c00640 0 client.15459 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 shutdown_connections 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 1 -- 172.21.15.3:0/3349471402 wait complete. 2024-03-21T08:31:21.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.043+0000 7f8c8c5c6740 20 client.15459 trim_cache size 0 max 16384 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "544fc91e-e75b-11ee-95cd-87774f69a715", 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 34472, 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 164.33549815699999, 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 96, 2024-03-21T08:31:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 94, 2024-03-21T08:31:21.054 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 185.17778731199999 2024-03-21T08:31:21.054 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:31:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c680a2fa0 msgr2=0x7f8c680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c680a2fa0 0x7f8c680a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c098190 tx=0x7f8c7c00cf50 comp rx=0 tx=0).stop 2024-03-21T08:31:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84109890 msgr2=0x7f8c841a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84109890 0x7f8c841a54e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00eaa0 tx=0x7f8c7c00ef60 comp rx=0 tx=0).stop 2024-03-21T08:31:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 shutdown_connections 2024-03-21T08:31:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c84109890 0x7f8c841a54e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c680a2fa0 0x7f8c680a5460 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c84108ed0 0x7f8c8419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 --2- 172.21.15.3:0/703233963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c84107ce0 0x7f8c8419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 >> 172.21.15.3:0/703233963 conn(0x7f8c84076c90 msgr2=0x7f8c8410c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 shutdown_connections 2024-03-21T08:31:21.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:21.044+0000 7f8c89800640 1 -- 172.21.15.3:0/703233963 wait complete. 2024-03-21T08:31:21.063 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-21T08:31:21.071 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-21T08:31:21.071 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-21T08:31:21.071 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-21T08:31:21.071 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T08:31:21.072 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:21.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:21.072 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-03-21T08:31:21.128 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-21T08:31:21.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:21.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T08:31:21.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:21.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-03-21T08:31:21.310 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:21 smithi003 ceph-mon[31675]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:21.355 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-21T08:31:21.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T08:31:21.535 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:21 smithi171 ceph-mon[35388]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:21.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:21 smithi084 ceph-mon[35477]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:21.816 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:31:23.368 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:23 smithi003 ceph-mon[31675]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:23.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:23 smithi084 ceph-mon[35477]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:23.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:23 smithi171 ceph-mon[35388]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.836+0000 7ff2c1000700 1 -- 172.21.15.3:0/1935483365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 msgr2=0x7ff2bc106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.836+0000 7ff2c1000700 1 --2- 172.21.15.3:0/1935483365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc106dd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80099c0 tx=0x7ff2a8009cd0 comp rx=0 tx=0).stop 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.836+0000 7ff2c1000700 1 -- 172.21.15.3:0/1935483365 shutdown_connections 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.836+0000 7ff2c1000700 1 --2- 172.21.15.3:0/1935483365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2bc107500 0x7ff2bc10f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.836+0000 7ff2c1000700 1 --2- 172.21.15.3:0/1935483365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2bc06aa90 0x7ff2bc06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.836+0000 7ff2c1000700 1 --2- 172.21.15.3:0/1935483365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc106dd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.836+0000 7ff2c1000700 1 -- 172.21.15.3:0/1935483365 >> 172.21.15.3:0/1935483365 conn(0x7ff2bc078000 msgr2=0x7ff2bc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.837+0000 7ff2c1000700 1 -- 172.21.15.3:0/1935483365 shutdown_connections 2024-03-21T08:31:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.837+0000 7ff2c1000700 1 -- 172.21.15.3:0/1935483365 wait complete. 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 Processor -- start 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 -- start start 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2bc06aa90 0x7ff2bc101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2bc107500 0x7ff2bc101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc112350 con 0x7ff2bc06b440 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1121d0 con 0x7ff2bc06aa90 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2c1000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1124d0 con 0x7ff2bc107500 2024-03-21T08:31:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.838+0000 7ff2b9400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45072/0 (socket says 172.21.15.3:45072) 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 -- 172.21.15.3:0/2796654111 learned_addr learned my addr 172.21.15.3:0/2796654111 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9e00700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2bc06aa90 0x7ff2bc101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2ba800700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2bc107500 0x7ff2bc101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 -- 172.21.15.3:0/2796654111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2bc107500 msgr2=0x7ff2bc101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2bc107500 0x7ff2bc101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 -- 172.21.15.3:0/2796654111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2bc06aa90 msgr2=0x7ff2bc101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2bc06aa90 0x7ff2bc101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 -- 172.21.15.3:0/2796654111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc1a6ea0 con 0x7ff2bc06b440 2024-03-21T08:31:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.839+0000 7ff2b9400700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc101690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098e0 tx=0x7ff2a8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.840+0000 7ff2b2a00700 1 -- 172.21.15.3:0/2796654111 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8009e60 con 0x7ff2bc06b440 2024-03-21T08:31:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.840+0000 7ff2b2a00700 1 -- 172.21.15.3:0/2796654111 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2a8004830 con 0x7ff2bc06b440 2024-03-21T08:31:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.840+0000 7ff2b2a00700 1 -- 172.21.15.3:0/2796654111 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8005280 con 0x7ff2bc06b440 2024-03-21T08:31:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.840+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc1a7130 con 0x7ff2bc06b440 2024-03-21T08:31:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.841+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc1a7530 con 0x7ff2bc06b440 2024-03-21T08:31:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.842+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc1038b0 con 0x7ff2bc06b440 2024-03-21T08:31:23.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.844+0000 7ff2b2a00700 1 -- 172.21.15.3:0/2796654111 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7ff2a80049e0 con 0x7ff2bc06b440 2024-03-21T08:31:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.846+0000 7ff2b2a00700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff2a00a3030 0x7ff2a00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.846+0000 7ff2b2a00700 1 -- 172.21.15.3:0/2796654111 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff2a80d0220 con 0x7ff2bc06b440 2024-03-21T08:31:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.846+0000 7ff2b9e00700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff2a00a3030 0x7ff2a00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.847+0000 7ff2b9e00700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff2a00a3030 0x7ff2a00a54f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009bf0 tx=0x7ff2ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:23.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:23.851+0000 7ff2b2a00700 1 -- 172.21.15.3:0/2796654111 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff2a8013070 con 0x7ff2bc06b440 2024-03-21T08:31:24.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.127+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2bc02dad0 con 0x7ff2bc06b440 2024-03-21T08:31:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.128+0000 7ff2b2a00700 1 -- 172.21.15.3:0/2796654111 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff2a80213d0 con 0x7ff2bc06b440 2024-03-21T08:31:24.130 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T08:31:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff2a00a3030 msgr2=0x7ff2a00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff2a00a3030 0x7ff2a00a54f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009bf0 tx=0x7ff2ac009260 comp rx=0 tx=0).stop 2024-03-21T08:31:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 msgr2=0x7ff2bc101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc101690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098e0 tx=0x7ff2a8004690 comp rx=0 tx=0).stop 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 shutdown_connections 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff2a00a3030 0x7ff2a00a54f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2bc107500 0x7ff2bc101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2bc06aa90 0x7ff2bc101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 --2- 172.21.15.3:0/2796654111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2bc06b440 0x7ff2bc101690 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.131+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 >> 172.21.15.3:0/2796654111 conn(0x7ff2bc078000 msgr2=0x7ff2bc0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.132+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 shutdown_connections 2024-03-21T08:31:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:24.132+0000 7ff2c1000700 1 -- 172.21.15.3:0/2796654111 wait complete. 2024-03-21T08:31:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:24 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2796654111' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:31:24.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:24 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2796654111' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:31:24.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:24 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2796654111' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:31:24.925 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-21T08:31:24.925 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T08:31:24.925 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-03-21T08:31:24.925 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-21T08:31:24.926 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:24.926 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-21T08:31:24.926 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T08:31:24.926 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T08:31:24.926 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-21T08:31:24.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:24.926 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-03-21T08:31:24.989 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T08:31:24.989 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T08:31:24.989 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-03-21T08:31:24.990 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41724sec preferred_lft 41724sec 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-03-21T08:31:24.991 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-03-21T08:31:24.992 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T08:31:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:24.992 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:31:24.992 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:31:24.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-03-21T08:31:24.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-03-21T08:31:24.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-03-21T08:31:24.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T08:31:24.993 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:31:25.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:25.239 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T08:31:25.319 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-03-21T08:31:25.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:25.324 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-03-21T08:31:25.379 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-03-21T08:31:25.379 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T08:31:25.379 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T08:31:25.379 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T08:31:25.379 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T08:31:25.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:25.379 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:31:25.379 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:31:25.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T08:31:25.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T08:31:25.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T08:31:25.380 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:31:25.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:25 smithi003 ceph-mon[31675]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:25.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:25.574 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-03-21T08:31:25.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:25 smithi084 ceph-mon[35477]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:25.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:25 smithi171 ceph-mon[35388]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T08:31:25.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:25.677 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-03-21T08:31:25.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:25.735 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:31:25.735 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:31:25.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:31:25.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-21T08:31:25.736 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:31:25.890 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-21T08:31:25.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-21T08:31:25.890 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:31:26.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:26.150 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2024-03-21T08:31:26.150 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-03-21T08:31:26.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-03-21T08:31:26.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T08:31:26.151 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-03-21T08:31:26.330 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-21T08:31:26.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T08:31:26.330 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:26.371 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-21T08:31:26.371 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T08:31:26.371 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:26.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-03-21T08:31:26.590 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-03-21T08:31:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-03-21T08:31:26.646 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/eb1ed2432cfab63a4b72798347886aed47dc3ba69fef7b22c63c56f77899a1db/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7ad661500301612b71fa5ceea623c4b888e83eaff29188df8423fee2ec212813/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/b9244e850b1ee0c649d575f2fdee3f518ddafe0e2b860aa6d628fc386f11ece2/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/ba82b0150aae1b7b22cd0c1a93cc366a0346de48a14645dafbe478a5034d8383/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/2abedc3682b4d2a8a526875f63e14e8c524e036f6675003c7eef2a3f604290b7/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/dc79872ffba797f1b2411b7614686e9c6312bdb341dbbfb58f75e22706e78896/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d52178c390940302fa16cd17170672e4efd32de7adc4d4e17da2b92a351d6929/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7d0f2664a3a6bd2838e50b42462ffe6462e94a514442b1560ad834f4aa3dca76/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7c6b09c1743db251c08db3fbde3e05413fa0445a49aa70cf6e9f1ee920a875ad/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7bc152af8d338f0ee3e0f5f249984bc70707b6c24e83c45803c2ff641c74077d/merged 2024-03-21T08:31:26.647 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-03-21T08:31:26.648 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:31:26.648 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:31:26.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:26.649 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-03-21T08:31:26.697 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T08:31:26.698 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393 --client_fs=cephfs) 2024-03-21T08:31:26.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-03-21T08:31:26.776 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T08:31:26.820 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-03-21T08:31:26.821 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/eb1ed2432cfab63a4b72798347886aed47dc3ba69fef7b22c63c56f77899a1db/merged 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7ad661500301612b71fa5ceea623c4b888e83eaff29188df8423fee2ec212813/merged 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-03-21T08:31:26.822 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/b9244e850b1ee0c649d575f2fdee3f518ddafe0e2b860aa6d628fc386f11ece2/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/ba82b0150aae1b7b22cd0c1a93cc366a0346de48a14645dafbe478a5034d8383/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/2abedc3682b4d2a8a526875f63e14e8c524e036f6675003c7eef2a3f604290b7/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/dc79872ffba797f1b2411b7614686e9c6312bdb341dbbfb58f75e22706e78896/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d52178c390940302fa16cd17170672e4efd32de7adc4d4e17da2b92a351d6929/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7d0f2664a3a6bd2838e50b42462ffe6462e94a514442b1560ad834f4aa3dca76/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7c6b09c1743db251c08db3fbde3e05413fa0445a49aa70cf6e9f1ee920a875ad/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7bc152af8d338f0ee3e0f5f249984bc70707b6c24e83c45803c2ff641c74077d/merged 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:31:26.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:26.824 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-03-21T08:31:26.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-03-21T08:31:26.887+0000 7fdf6b647480 -1 init, newargv = 0x561a5b64a120 newargc=15 2024-03-21T08:31:26.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[85526]: starting ceph client 2024-03-21T08:31:26.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[85526]: starting fuse 2024-03-21T08:31:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:27 smithi003 ceph-mon[31675]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:31:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:27 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:27.594 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:27 smithi084 ceph-mon[35477]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:27.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:31:27.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:27.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:27 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:27.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:27 smithi171 ceph-mon[35388]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:31:27.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:31:27.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:27.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:27 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:28.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-03-21T08:31:28.107 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T08:31:28.148 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-03-21T08:31:28.149 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-03-21T08:31:28.150 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/eb1ed2432cfab63a4b72798347886aed47dc3ba69fef7b22c63c56f77899a1db/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7ad661500301612b71fa5ceea623c4b888e83eaff29188df8423fee2ec212813/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/b9244e850b1ee0c649d575f2fdee3f518ddafe0e2b860aa6d628fc386f11ece2/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/ba82b0150aae1b7b22cd0c1a93cc366a0346de48a14645dafbe478a5034d8383/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/2abedc3682b4d2a8a526875f63e14e8c524e036f6675003c7eef2a3f604290b7/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/dc79872ffba797f1b2411b7614686e9c6312bdb341dbbfb58f75e22706e78896/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d52178c390940302fa16cd17170672e4efd32de7adc4d4e17da2b92a351d6929/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7d0f2664a3a6bd2838e50b42462ffe6462e94a514442b1560ad834f4aa3dca76/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7c6b09c1743db251c08db3fbde3e05413fa0445a49aa70cf6e9f1ee920a875ad/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/7bc152af8d338f0ee3e0f5f249984bc70707b6c24e83c45803c2ff641c74077d/merged 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-03-21T08:31:28.151 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:31:28.152 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:31:28.152 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:28.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:28.153 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-03-21T08:31:28.238 INFO:teuthology.orchestra.run.smithi003.stdout:118 2024-03-21T08:31:28.238 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-03-21T08:31:28.239 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-03-21T08:31:28.240 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-03-21T08:31:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.85526.asok 2024-03-21T08:31:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T08:31:28.458 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.85526.asok 2024-03-21T08:31:28.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:28.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.85526.asok status 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "2edfee68ad7243bed422a049b6f5f58619300f11", 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "85526", 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393" 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:31:28.680 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 15471, 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 15471 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2998133843 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-03-21T08:31:28.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2998133843 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.15471 192.168.0.1:0/2998133843", 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2998133843", 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 5, 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 22, 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 96, 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-03-21T08:31:28.682 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:31:28.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:28.697 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-03-21T08:31:28.697 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:28.739 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-03-21T08:31:28.739 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:28.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:28.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:28.863 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T08:31:28.873 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T08:31:28.873 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:31:28.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 -- 172.21.15.3:0/266903093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1016f0 msgr2=0x7f690c101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 --2- 172.21.15.3:0/266903093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1016f0 0x7f690c101b10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6900009900 tx=0x7f690002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 -- 172.21.15.3:0/266903093 shutdown_connections 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 --2- 172.21.15.3:0/266903093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690c1032a0 0x7f690c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 --2- 172.21.15.3:0/266903093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f690c1028e0 0x7f690c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 --2- 172.21.15.3:0/266903093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1016f0 0x7f690c101b10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 -- 172.21.15.3:0/266903093 >> 172.21.15.3:0/266903093 conn(0x7f690c0fcc60 msgr2=0x7f690c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 -- 172.21.15.3:0/266903093 shutdown_connections 2024-03-21T08:31:29.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.074+0000 7f6914600640 1 -- 172.21.15.3:0/266903093 wait complete. 2024-03-21T08:31:29.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.076+0000 7f6914600640 1 Processor -- start 2024-03-21T08:31:29.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.076+0000 7f6914600640 1 -- start start 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.076+0000 7f6914600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690c1016f0 0x7f690c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.076+0000 7f6914600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f690c1028e0 0x7f690c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6914600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1032a0 0x7f690c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6914600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10d240 con 0x7f690c1032a0 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6914600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10d0c0 con 0x7f690c1016f0 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6914600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10d3c0 con 0x7f690c1028e0 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1032a0 0x7f690c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1032a0 0x7f690c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 -- 172.21.15.3:0/3775806105 learned_addr learned my addr 172.21.15.3:0/3775806105 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6911200640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f690c1028e0 0x7f690c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6911c00640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690c1016f0 0x7f690c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:29.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 -- 172.21.15.3:0/3775806105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f690c1028e0 msgr2=0x7f690c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f690c1028e0 0x7f690c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 -- 172.21.15.3:0/3775806105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690c1016f0 msgr2=0x7f690c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690c1016f0 0x7f690c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.077+0000 7f6912600640 1 -- 172.21.15.3:0/3775806105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c19f560 con 0x7f690c1032a0 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.078+0000 7f6912600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1032a0 0x7f690c19eef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f690400eaa0 tx=0x7f690400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.078+0000 7f690a000640 1 -- 172.21.15.3:0/3775806105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690400cdb0 con 0x7f690c1032a0 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.078+0000 7f690a000640 1 -- 172.21.15.3:0/3775806105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6904005020 con 0x7f690c1032a0 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.078+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c19f850 con 0x7f690c1032a0 2024-03-21T08:31:29.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.078+0000 7f690a000640 1 -- 172.21.15.3:0/3775806105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040116d0 con 0x7f690c1032a0 2024-03-21T08:31:29.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.078+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c19fd50 con 0x7f690c1032a0 2024-03-21T08:31:29.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.080+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c108a80 con 0x7f690c1032a0 2024-03-21T08:31:29.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.081+0000 7f690a000640 1 -- 172.21.15.3:0/3775806105 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f69040051d0 con 0x7f690c1032a0 2024-03-21T08:31:29.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.083+0000 7f690a000640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f68f40a2f10 0x7f68f40a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:29.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.083+0000 7f690a000640 1 -- 172.21.15.3:0/3775806105 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f69040cf9e0 con 0x7f690c1032a0 2024-03-21T08:31:29.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.083+0000 7f6911c00640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f68f40a2f10 0x7f68f40a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:29.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.084+0000 7f6911c00640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f68f40a2f10 0x7f68f40a53d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f690002c240 tx=0x7f690002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.089+0000 7f690a000640 1 -- 172.21.15.3:0/3775806105 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6904016030 con 0x7f690c1032a0 2024-03-21T08:31:29.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:29.349+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f690c0622a0 con 0x7f690c1032a0 2024-03-21T08:31:29.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:29 smithi084 ceph-mon[35477]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:29.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:29 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:29 smithi171 ceph-mon[35388]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:29.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:29 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:29 smithi003 ceph-mon[31675]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:31:29.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:29.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:29 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:30.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.215+0000 7f690a000640 1 -- 172.21.15.3:0/3775806105 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v23) v1 ==== 117+0+0 (secure 0 0 0) 0x7f690409bd20 con 0x7f690c1032a0 2024-03-21T08:31:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f68f40a2f10 msgr2=0x7f68f40a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f68f40a2f10 0x7f68f40a53d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f690002c240 tx=0x7f690002c7e0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1032a0 msgr2=0x7f690c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1032a0 0x7f690c19eef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f690400eaa0 tx=0x7f690400ef60 comp rx=0 tx=0).stop 2024-03-21T08:31:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 shutdown_connections 2024-03-21T08:31:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f690c1032a0 0x7f690c19eef0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f68f40a2f10 0x7f68f40a53d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f690c1028e0 0x7f690c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 --2- 172.21.15.3:0/3775806105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690c1016f0 0x7f690c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 >> 172.21.15.3:0/3775806105 conn(0x7f690c0fcc60 msgr2=0x7f690c105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 shutdown_connections 2024-03-21T08:31:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.219+0000 7f6914600640 1 -- 172.21.15.3:0/3775806105 wait complete. 2024-03-21T08:31:30.241 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-21T08:31:30.259 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-21T08:31:30.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:31:30.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 -- 172.21.15.3:0/3278921666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214107420 msgr2=0x7fe214109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 --2- 172.21.15.3:0/3278921666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214107420 0x7fe214109870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ca0 tx=0x7fe20c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 -- 172.21.15.3:0/3278921666 shutdown_connections 2024-03-21T08:31:30.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 --2- 172.21.15.3:0/3278921666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214107420 0x7fe214109870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 --2- 172.21.15.3:0/3278921666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214104af0 0x7fe214106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 --2- 172.21.15.3:0/3278921666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2141021c0 0x7fe2141045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 -- 172.21.15.3:0/3278921666 >> 172.21.15.3:0/3278921666 conn(0x7fe2140fbbd0 msgr2=0x7fe2140fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:30.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 -- 172.21.15.3:0/3278921666 shutdown_connections 2024-03-21T08:31:30.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.496+0000 7fe219e00640 1 -- 172.21.15.3:0/3278921666 wait complete. 2024-03-21T08:31:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.497+0000 7fe219e00640 1 Processor -- start 2024-03-21T08:31:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.497+0000 7fe219e00640 1 -- start start 2024-03-21T08:31:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.497+0000 7fe219e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2141021c0 0x7fe214199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe219e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214104af0 0x7fe21419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe219e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214107420 0x7fe2141a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe219e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410d220 con 0x7fe214104af0 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe219e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410d0a0 con 0x7fe2141021c0 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe219e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410d3a0 con 0x7fe214107420 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214104af0 0x7fe21419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214104af0 0x7fe21419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34068/0 (socket says 172.21.15.3:34068) 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 -- 172.21.15.3:0/197709725 learned_addr learned my addr 172.21.15.3:0/197709725 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe212a00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2141021c0 0x7fe214199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe213400640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214107420 0x7fe2141a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 -- 172.21.15.3:0/197709725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214107420 msgr2=0x7fe2141a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214107420 0x7fe2141a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 -- 172.21.15.3:0/197709725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2141021c0 msgr2=0x7fe214199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2141021c0 0x7fe214199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe20be00640 1 -- 172.21.15.3:0/197709725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2141a16f0 con 0x7fe214104af0 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.498+0000 7fe212a00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2141021c0 0x7fe214199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.499+0000 7fe20be00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214104af0 0x7fe21419a010 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00d740 tx=0x7fe1fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.499+0000 7fe218a00640 1 -- 172.21.15.3:0/197709725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc017070 con 0x7fe214104af0 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.499+0000 7fe218a00640 1 -- 172.21.15.3:0/197709725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1fc00de50 con 0x7fe214104af0 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.499+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2141a19e0 con 0x7fe214104af0 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.499+0000 7fe218a00640 1 -- 172.21.15.3:0/197709725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc004d60 con 0x7fe214104af0 2024-03-21T08:31:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.499+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141a1ee0 con 0x7fe214104af0 2024-03-21T08:31:30.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.500+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214108f10 con 0x7fe214104af0 2024-03-21T08:31:30.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.502+0000 7fe218a00640 1 -- 172.21.15.3:0/197709725 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fe1fc004f00 con 0x7fe214104af0 2024-03-21T08:31:30.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.504+0000 7fe218a00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1f80a2f20 0x7fe1f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.505+0000 7fe218a00640 1 -- 172.21.15.3:0/197709725 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe1fc0cf250 con 0x7fe214104af0 2024-03-21T08:31:30.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.505+0000 7fe212a00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1f80a2f20 0x7fe1f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.505+0000 7fe212a00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1f80a2f20 0x7fe1f80a53e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe204005d00 tx=0x7fe204005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:30.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.509+0000 7fe218a00640 1 -- 172.21.15.3:0/197709725 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe1fc09bbe0 con 0x7fe214104af0 2024-03-21T08:31:30.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:30 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3775806105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-21T08:31:30.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:30.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:30 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:30.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:30 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3775806105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-21T08:31:30.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:30.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:30 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:30.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:30 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3775806105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-21T08:31:30.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:30.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:30 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.714+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2140622d0 con 0x7fe214104af0 2024-03-21T08:31:30.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.716+0000 7fe218a00640 1 -- 172.21.15.3:0/197709725 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7fe1fc09b4e0 con 0x7fe214104af0 2024-03-21T08:31:30.719 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:30.719 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:31:30.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1f80a2f20 msgr2=0x7fe1f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1f80a2f20 0x7fe1f80a53e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe204005d00 tx=0x7fe204005c70 comp rx=0 tx=0).stop 2024-03-21T08:31:30.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214104af0 msgr2=0x7fe21419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214104af0 0x7fe21419a010 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00d740 tx=0x7fe1fc00dc00 comp rx=0 tx=0).stop 2024-03-21T08:31:30.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 shutdown_connections 2024-03-21T08:31:30.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe1f80a2f20 0x7fe1f80a53e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214107420 0x7fe2141a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe214104af0 0x7fe21419a010 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 --2- 172.21.15.3:0/197709725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2141021c0 0x7fe214199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.719+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 >> 172.21.15.3:0/197709725 conn(0x7fe2140fbbd0 msgr2=0x7fe214101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:30.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.720+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 shutdown_connections 2024-03-21T08:31:30.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.720+0000 7fe219e00640 1 -- 172.21.15.3:0/197709725 wait complete. 2024-03-21T08:31:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:31:30.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:31:30.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 -- 172.21.15.3:0/1743107547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 msgr2=0x7fa5e8101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 --2- 172.21.15.3:0/1743107547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8101ac0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009900 tx=0x7fa5dc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:30.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 -- 172.21.15.3:0/1743107547 shutdown_connections 2024-03-21T08:31:30.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 --2- 172.21.15.3:0/1743107547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5e8103250 0x7fa5e8109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 --2- 172.21.15.3:0/1743107547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5e8102890 0x7fa5e8102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 --2- 172.21.15.3:0/1743107547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8101ac0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 -- 172.21.15.3:0/1743107547 >> 172.21.15.3:0/1743107547 conn(0x7fa5e80fcbf0 msgr2=0x7fa5e80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:30.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 -- 172.21.15.3:0/1743107547 shutdown_connections 2024-03-21T08:31:30.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.965+0000 7fa5f0400640 1 -- 172.21.15.3:0/1743107547 wait complete. 2024-03-21T08:31:30.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.966+0000 7fa5f0400640 1 Processor -- start 2024-03-21T08:31:30.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5f0400640 1 -- start start 2024-03-21T08:31:30.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5f0400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5f0400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5e8102890 0x7fa5e819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5f0400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5e8103250 0x7fa5e81a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5f0400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e810d210 con 0x7fa5e81016a0 2024-03-21T08:31:30.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5f0400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e810d090 con 0x7fa5e8103250 2024-03-21T08:31:30.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5f0400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e810d390 con 0x7fa5e8102890 2024-03-21T08:31:30.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5eda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5eda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34090/0 (socket says 172.21.15.3:34090) 2024-03-21T08:31:30.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5eda00640 1 -- 172.21.15.3:0/247982398 learned_addr learned my addr 172.21.15.3:0/247982398 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:30.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.967+0000 7fa5ee400640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5e8103250 0x7fa5e81a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5ed000640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5e8102890 0x7fa5e819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5eda00640 1 -- 172.21.15.3:0/247982398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5e8102890 msgr2=0x7fa5e819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5eda00640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5e8102890 0x7fa5e819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5eda00640 1 -- 172.21.15.3:0/247982398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5e8103250 msgr2=0x7fa5e81a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5eda00640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5e8103250 0x7fa5e81a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5eda00640 1 -- 172.21.15.3:0/247982398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e81a1760 con 0x7fa5e81016a0 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5ed000640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5e8102890 0x7fa5e819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5eda00640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8199b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc000c00 tx=0x7fa5dc005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.968+0000 7fa5e6000640 1 -- 172.21.15.3:0/247982398 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc043020 con 0x7fa5e81016a0 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.969+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e81a19f0 con 0x7fa5e81016a0 2024-03-21T08:31:30.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.969+0000 7fa5e6000640 1 -- 172.21.15.3:0/247982398 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5dc0043d0 con 0x7fa5e81016a0 2024-03-21T08:31:30.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.969+0000 7fa5e6000640 1 -- 172.21.15.3:0/247982398 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc003bb0 con 0x7fa5e81016a0 2024-03-21T08:31:30.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.969+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e81a1f40 con 0x7fa5e81016a0 2024-03-21T08:31:30.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.970+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e81a2110 con 0x7fa5e81016a0 2024-03-21T08:31:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.972+0000 7fa5e6000640 1 -- 172.21.15.3:0/247982398 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fa5dc046020 con 0x7fa5e81016a0 2024-03-21T08:31:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.973+0000 7fa5e6000640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa5d00a2f20 0x7fa5d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.973+0000 7fa5e6000640 1 -- 172.21.15.3:0/247982398 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa5dc0f1a80 con 0x7fa5e81016a0 2024-03-21T08:31:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.973+0000 7fa5ed000640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa5d00a2f20 0x7fa5d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.974+0000 7fa5ed000640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa5d00a2f20 0x7fa5d00a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5e819b050 tx=0x7fa5d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:30.978+0000 7fa5e6000640 1 -- 172.21.15.3:0/247982398 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5dc0f3e20 con 0x7fa5e81016a0 2024-03-21T08:31:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.185+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5e81a2400 con 0x7fa5e81016a0 2024-03-21T08:31:31.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.186+0000 7fa5e6000640 1 -- 172.21.15.3:0/247982398 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5dc04d020 con 0x7fa5e81016a0 2024-03-21T08:31:31.188 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa5d00a2f20 msgr2=0x7fa5d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa5d00a2f20 0x7fa5d00a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5e819b050 tx=0x7fa5d4005f90 comp rx=0 tx=0).stop 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 msgr2=0x7fa5e8199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8199b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc000c00 tx=0x7fa5dc005e60 comp rx=0 tx=0).stop 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 shutdown_connections 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5e8103250 0x7fa5e81a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fa5d00a2f20 0x7fa5d00a53e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5e8102890 0x7fa5e819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 --2- 172.21.15.3:0/247982398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e81016a0 0x7fa5e8199b90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 >> 172.21.15.3:0/247982398 conn(0x7fa5e80fcbf0 msgr2=0x7fa5e80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:31.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 shutdown_connections 2024-03-21T08:31:31.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.189+0000 7fa5f0400640 1 -- 172.21.15.3:0/247982398 wait complete. 2024-03-21T08:31:31.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:31:31.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.440+0000 7f981aa00640 1 -- 172.21.15.3:0/732296861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f981410dac0 msgr2=0x7f981410feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.440+0000 7f981aa00640 1 --2- 172.21.15.3:0/732296861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f981410dac0 0x7f981410feb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f980c009cd0 tx=0x7f980c02c0e0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.440+0000 7f981aa00640 1 -- 172.21.15.3:0/732296861 shutdown_connections 2024-03-21T08:31:31.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.440+0000 7f981aa00640 1 --2- 172.21.15.3:0/732296861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f981410dac0 0x7f981410feb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.440+0000 7f981aa00640 1 --2- 172.21.15.3:0/732296861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9814074100 0x7f9814074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.440+0000 7f981aa00640 1 --2- 172.21.15.3:0/732296861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98140756e0 0x7f9814073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.440+0000 7f981aa00640 1 -- 172.21.15.3:0/732296861 >> 172.21.15.3:0/732296861 conn(0x7f98140fca60 msgr2=0x7f98140feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:31.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.441+0000 7f981aa00640 1 -- 172.21.15.3:0/732296861 shutdown_connections 2024-03-21T08:31:31.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.441+0000 7f981aa00640 1 -- 172.21.15.3:0/732296861 wait complete. 2024-03-21T08:31:31.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.441+0000 7f981aa00640 1 Processor -- start 2024-03-21T08:31:31.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f981aa00640 1 -- start start 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f981aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9814074100 0x7f981419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f981aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98140756e0 0x7f981419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f981aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f981410dac0 0x7f98141a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f981aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98141135f0 con 0x7f9814074100 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f981aa00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9814113470 con 0x7f98140756e0 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f981aa00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9814113770 con 0x7f981410dac0 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f9819600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9814074100 0x7f981419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f9819600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9814074100 0x7f981419dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34110/0 (socket says 172.21.15.3:34110) 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f9819600640 1 -- 172.21.15.3:0/187058325 learned_addr learned my addr 172.21.15.3:0/187058325 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.442+0000 7f9818c00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98140756e0 0x7f981419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f981a000640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f981410dac0 0x7f98141a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9819600640 1 -- 172.21.15.3:0/187058325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f981410dac0 msgr2=0x7f98141a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9819600640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f981410dac0 0x7f98141a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9819600640 1 -- 172.21.15.3:0/187058325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98140756e0 msgr2=0x7f981419e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9819600640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98140756e0 0x7f981419e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9819600640 1 -- 172.21.15.3:0/187058325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98141a58e0 con 0x7f9814074100 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9819600640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9814074100 0x7f981419dca0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f980400e9f0 tx=0x7f980400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9812000640 1 -- 172.21.15.3:0/187058325 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400cd80 con 0x7f9814074100 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f9812000640 1 -- 172.21.15.3:0/187058325 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98040108e0 con 0x7f9814074100 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.443+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98141a5bd0 con 0x7f9814074100 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.444+0000 7f9812000640 1 -- 172.21.15.3:0/187058325 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804018920 con 0x7f9814074100 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.444+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98141a6170 con 0x7f9814074100 2024-03-21T08:31:31.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.445+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9814103c90 con 0x7f9814074100 2024-03-21T08:31:31.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.449+0000 7f9812000640 1 -- 172.21.15.3:0/187058325 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f9804010a90 con 0x7f9814074100 2024-03-21T08:31:31.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.450+0000 7f9812000640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f97fc0a2fa0 0x7f97fc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.450+0000 7f9812000640 1 -- 172.21.15.3:0/187058325 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f98040cfb50 con 0x7f9814074100 2024-03-21T08:31:31.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.450+0000 7f9818c00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f97fc0a2fa0 0x7f97fc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.451+0000 7f9818c00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f97fc0a2fa0 0x7f97fc0a5460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f981419f1a0 tx=0x7f980800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:31.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.455+0000 7f9812000640 1 -- 172.21.15.3:0/187058325 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f980409c540 con 0x7f9814074100 2024-03-21T08:31:31.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: pgmap v458: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:31:31.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3775806105' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-21T08:31:31.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:31:31.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/197709725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:31.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:31:31.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:31:31.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:31.528 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:31 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/247982398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: pgmap v458: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3775806105' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/197709725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:31.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:31 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/247982398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:31.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.656+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9814062340 con 0x7f9814074100 2024-03-21T08:31:31.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.657+0000 7f9812000640 1 -- 172.21.15.3:0/187058325 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f980409be40 con 0x7f9814074100 2024-03-21T08:31:31.659 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:31.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f97fc0a2fa0 msgr2=0x7f97fc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f97fc0a2fa0 0x7f97fc0a5460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f981419f1a0 tx=0x7f980800b390 comp rx=0 tx=0).stop 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9814074100 msgr2=0x7f981419dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9814074100 0x7f981419dca0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f980400e9f0 tx=0x7f980400eeb0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 shutdown_connections 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f97fc0a2fa0 0x7f97fc0a5460 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f981410dac0 0x7f98141a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98140756e0 0x7f981419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.660+0000 7f981aa00640 1 --2- 172.21.15.3:0/187058325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9814074100 0x7f981419dca0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.661+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 >> 172.21.15.3:0/187058325 conn(0x7f98140fca60 msgr2=0x7f9814100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.661+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 shutdown_connections 2024-03-21T08:31:31.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.661+0000 7f981aa00640 1 -- 172.21.15.3:0/187058325 wait complete. 2024-03-21T08:31:31.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: pgmap v458: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:31:31.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3775806105' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-21T08:31:31.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:31:31.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/197709725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:31.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:31:31.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:31:31.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: from='mgr.14152 172.21.15.3:0/1560723832' entity='mgr.x' 2024-03-21T08:31:31.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:31 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/247982398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:31.684 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 -- 172.21.15.3:0/706053756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454107ce0 msgr2=0x7f0454108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 --2- 172.21.15.3:0/706053756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454107ce0 0x7f0454108100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0444009900 tx=0x7f044402bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 -- 172.21.15.3:0/706053756 shutdown_connections 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 --2- 172.21.15.3:0/706053756 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454109890 0x7f0454110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 --2- 172.21.15.3:0/706053756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0454108ed0 0x7f0454109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 --2- 172.21.15.3:0/706053756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454107ce0 0x7f0454108100 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 -- 172.21.15.3:0/706053756 >> 172.21.15.3:0/706053756 conn(0x7f0454076c90 msgr2=0x7f04540790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:31.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 -- 172.21.15.3:0/706053756 shutdown_connections 2024-03-21T08:31:31.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.915+0000 7f045a200640 1 -- 172.21.15.3:0/706053756 wait complete. 2024-03-21T08:31:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.916+0000 7f045a200640 1 Processor -- start 2024-03-21T08:31:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.916+0000 7f045a200640 1 -- start start 2024-03-21T08:31:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.916+0000 7f045a200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454107ce0 0x7f045419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f045a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454108ed0 0x7f045419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f045a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0454109890 0x7f04541a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f045a200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0454113860 con 0x7f0454108ed0 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f045a200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04541136e0 con 0x7f0454109890 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f045a200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04541139e0 con 0x7f0454107ce0 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454108ed0 0x7f045419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454108ed0 0x7f045419e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34120/0 (socket says 172.21.15.3:34120) 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 -- 172.21.15.3:0/3125262036 learned_addr learned my addr 172.21.15.3:0/3125262036 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0453400640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454107ce0 0x7f045419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0453e00640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0454109890 0x7f04541a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 -- 172.21.15.3:0/3125262036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454107ce0 msgr2=0x7f045419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454107ce0 0x7f045419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 -- 172.21.15.3:0/3125262036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0454109890 msgr2=0x7f04541a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0454109890 0x7f04541a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:31.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0452a00640 1 -- 172.21.15.3:0/3125262036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04541a5bc0 con 0x7f0454108ed0 2024-03-21T08:31:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.917+0000 7f0453400640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454107ce0 0x7f045419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.918+0000 7f0452a00640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454108ed0 0x7f045419e4e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f044000d950 tx=0x7f044000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.918+0000 7f044be00640 1 -- 172.21.15.3:0/3125262036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440017070 con 0x7f0454108ed0 2024-03-21T08:31:31.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.918+0000 7f044be00640 1 -- 172.21.15.3:0/3125262036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0440009c30 con 0x7f0454108ed0 2024-03-21T08:31:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.918+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04541a5eb0 con 0x7f0454108ed0 2024-03-21T08:31:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.918+0000 7f044be00640 1 -- 172.21.15.3:0/3125262036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044000f620 con 0x7f0454108ed0 2024-03-21T08:31:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.918+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04541a6440 con 0x7f0454108ed0 2024-03-21T08:31:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.919+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045410f120 con 0x7f0454108ed0 2024-03-21T08:31:31.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.922+0000 7f044be00640 1 -- 172.21.15.3:0/3125262036 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f04400109a0 con 0x7f0454108ed0 2024-03-21T08:31:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.923+0000 7f044be00640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04380a2fa0 0x7f04380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.923+0000 7f044be00640 1 -- 172.21.15.3:0/3125262036 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f04400cf4b0 con 0x7f0454108ed0 2024-03-21T08:31:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.924+0000 7f0453400640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04380a2fa0 0x7f04380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.924+0000 7f0453400640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04380a2fa0 0x7f04380a5460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f04440096c0 tx=0x7f0444000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:31.928+0000 7f044be00640 1 -- 172.21.15.3:0/3125262036 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0440009de0 con 0x7f0454108ed0 2024-03-21T08:31:32.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.132+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0454062340 con 0x7f0454108ed0 2024-03-21T08:31:32.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.133+0000 7f044be00640 1 -- 172.21.15.3:0/3125262036 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f044009be40 con 0x7f0454108ed0 2024-03-21T08:31:32.135 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04380a2fa0 msgr2=0x7f04380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04380a2fa0 0x7f04380a5460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f04440096c0 tx=0x7f0444000bc0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454108ed0 msgr2=0x7f045419e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454108ed0 0x7f045419e4e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f044000d950 tx=0x7f044000de10 comp rx=0 tx=0).stop 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 shutdown_connections 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0454109890 0x7f04541a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0454108ed0 0x7f045419e4e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f04380a2fa0 0x7f04380a5460 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.136+0000 7f045a200640 1 --2- 172.21.15.3:0/3125262036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454107ce0 0x7f045419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.137+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 >> 172.21.15.3:0/3125262036 conn(0x7f0454076c90 msgr2=0x7f045410c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.137+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 shutdown_connections 2024-03-21T08:31:32.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.137+0000 7f045a200640 1 -- 172.21.15.3:0/3125262036 wait complete. 2024-03-21T08:31:32.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:31:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 -- 172.21.15.3:0/1833679013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 msgr2=0x7f9204109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 --2- 172.21.15.3:0/1833679013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 0x7f9204109170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009900 tx=0x7f91ec02bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 -- 172.21.15.3:0/1833679013 shutdown_connections 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 --2- 172.21.15.3:0/1833679013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92041096b0 0x7f920410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 --2- 172.21.15.3:0/1833679013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 0x7f9204109170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 --2- 172.21.15.3:0/1833679013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 0x7f9204107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 -- 172.21.15.3:0/1833679013 >> 172.21.15.3:0/1833679013 conn(0x7f9204076a90 msgr2=0x7f9204078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 -- 172.21.15.3:0/1833679013 shutdown_connections 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.382+0000 7f9203e00640 1 -- 172.21.15.3:0/1833679013 wait complete. 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 Processor -- start 2024-03-21T08:31:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 -- start start 2024-03-21T08:31:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 0x7f920419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 0x7f920419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92041096b0 0x7f92041a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204113630 con 0x7f9204107b00 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041134b0 con 0x7f9204108cf0 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.384+0000 7f9203e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041137b0 con 0x7f92041096b0 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 0x7f920419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 0x7f920419dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34134/0 (socket says 172.21.15.3:34134) 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 -- 172.21.15.3:0/3994490759 learned_addr learned my addr 172.21.15.3:0/3994490759 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f91fbe00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 0x7f920419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9203400640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92041096b0 0x7f92041a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 -- 172.21.15.3:0/3994490759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92041096b0 msgr2=0x7f92041a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92041096b0 0x7f92041a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 -- 172.21.15.3:0/3994490759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 msgr2=0x7f920419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 0x7f920419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.385+0000 7f9202a00640 1 -- 172.21.15.3:0/3994490759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92041a5940 con 0x7f9204107b00 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.386+0000 7f9202a00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 0x7f920419dd30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f91f400e9c0 tx=0x7f91f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.386+0000 7f91fb400640 1 -- 172.21.15.3:0/3994490759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400cd80 con 0x7f9204107b00 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.386+0000 7f91fb400640 1 -- 172.21.15.3:0/3994490759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91f40035f0 con 0x7f9204107b00 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.386+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92041a5c30 con 0x7f9204107b00 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.386+0000 7f91fb400640 1 -- 172.21.15.3:0/3994490759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4010690 con 0x7f9204107b00 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.386+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92041a61d0 con 0x7f9204107b00 2024-03-21T08:31:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.387+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920410ee40 con 0x7f9204107b00 2024-03-21T08:31:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.389+0000 7f91fb400640 1 -- 172.21.15.3:0/3994490759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f91f40037a0 con 0x7f9204107b00 2024-03-21T08:31:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.391+0000 7f91fb400640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f91e80a2fa0 0x7f91e80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.391+0000 7f91fb400640 1 -- 172.21.15.3:0/3994490759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f91f40cfc90 con 0x7f9204107b00 2024-03-21T08:31:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.391+0000 7f91fbe00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f91e80a2fa0 0x7f91e80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.392+0000 7f91fbe00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f91e80a2fa0 0x7f91e80a5460 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009230 tx=0x7f91ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:32.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.396+0000 7f91fb400640 1 -- 172.21.15.3:0/3994490759 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f91f409c620 con 0x7f9204107b00 2024-03-21T08:31:32.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:32 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/187058325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:32.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:32 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3125262036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:32.548 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:32 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/187058325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:32.548 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:32 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3125262036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:32.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.600+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9204062340 con 0x7f9204107b00 2024-03-21T08:31:32.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.602+0000 7f91fb400640 1 -- 172.21.15.3:0/3994490759 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f91f409bf20 con 0x7f9204107b00 2024-03-21T08:31:32.604 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f91e80a2fa0 msgr2=0x7f91e80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f91e80a2fa0 0x7f91e80a5460 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009230 tx=0x7f91ec005fd0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 msgr2=0x7f920419dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 0x7f920419dd30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f91f400e9c0 tx=0x7f91f400ee80 comp rx=0 tx=0).stop 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 shutdown_connections 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f91e80a2fa0 0x7f91e80a5460 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92041096b0 0x7f92041a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9204108cf0 0x7f920419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 --2- 172.21.15.3:0/3994490759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9204107b00 0x7f920419dd30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 >> 172.21.15.3:0/3994490759 conn(0x7f9204076a90 msgr2=0x7f920410c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 shutdown_connections 2024-03-21T08:31:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.605+0000 7f9203e00640 1 -- 172.21.15.3:0/3994490759 wait complete. 2024-03-21T08:31:32.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:31:32.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:32 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/187058325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:32.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:32 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3125262036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.854+0000 7f8b27200640 1 -- 172.21.15.3:0/2790172163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28107bc0 msgr2=0x7f8b28107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.854+0000 7f8b27200640 1 --2- 172.21.15.3:0/2790172163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28107bc0 0x7f8b28107fe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009ca0 tx=0x7f8b1802c0f0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.855+0000 7f8b27200640 1 -- 172.21.15.3:0/2790172163 shutdown_connections 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.855+0000 7f8b27200640 1 --2- 172.21.15.3:0/2790172163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28109770 0x7f8b28110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.855+0000 7f8b27200640 1 --2- 172.21.15.3:0/2790172163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b28108db0 0x7f8b28109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.855+0000 7f8b27200640 1 --2- 172.21.15.3:0/2790172163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28107bc0 0x7f8b28107fe0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.855+0000 7f8b27200640 1 -- 172.21.15.3:0/2790172163 >> 172.21.15.3:0/2790172163 conn(0x7f8b28076b60 msgr2=0x7f8b28078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.855+0000 7f8b27200640 1 -- 172.21.15.3:0/2790172163 shutdown_connections 2024-03-21T08:31:32.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.855+0000 7f8b27200640 1 -- 172.21.15.3:0/2790172163 wait complete. 2024-03-21T08:31:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 Processor -- start 2024-03-21T08:31:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 -- start start 2024-03-21T08:31:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28107bc0 0x7f8b2819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28108db0 0x7f8b2819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b28109770 0x7f8b281a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281136e0 con 0x7f8b28108db0 2024-03-21T08:31:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28113560 con 0x7f8b28107bc0 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.856+0000 7f8b27200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28113860 con 0x7f8b28109770 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28108db0 0x7f8b2819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28107bc0 0x7f8b2819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28108db0 0x7f8b2819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34152/0 (socket says 172.21.15.3:34152) 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25400640 1 -- 172.21.15.3:0/3537280414 learned_addr learned my addr 172.21.15.3:0/3537280414 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b26800640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b28109770 0x7f8b281a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25e00640 1 -- 172.21.15.3:0/3537280414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b28109770 msgr2=0x7f8b281a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25e00640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b28109770 0x7f8b281a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25e00640 1 -- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28108db0 msgr2=0x7f8b2819e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25e00640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28108db0 0x7f8b2819e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25e00640 1 -- 172.21.15.3:0/3537280414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b281a5a20 con 0x7f8b28107bc0 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.857+0000 7f8b25400640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28108db0 0x7f8b2819e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.858+0000 7f8b25e00640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28107bc0 0x7f8b2819de00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b1802c630 tx=0x7f8b180328a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.858+0000 7f8b1ea00640 1 -- 172.21.15.3:0/3537280414 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18041020 con 0x7f8b28107bc0 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.858+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b281a5cb0 con 0x7f8b28107bc0 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.858+0000 7f8b1ea00640 1 -- 172.21.15.3:0/3537280414 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b18003e20 con 0x7f8b28107bc0 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.858+0000 7f8b1ea00640 1 -- 172.21.15.3:0/3537280414 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18039040 con 0x7f8b28107bc0 2024-03-21T08:31:32.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.858+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b281a61c0 con 0x7f8b28107bc0 2024-03-21T08:31:32.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.859+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2810f000 con 0x7f8b28107bc0 2024-03-21T08:31:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.862+0000 7f8b1ea00640 1 -- 172.21.15.3:0/3537280414 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8b18044020 con 0x7f8b28107bc0 2024-03-21T08:31:32.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.864+0000 7f8b1ea00640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b0c0a2f90 0x7f8b0c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:32.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.864+0000 7f8b1ea00640 1 -- 172.21.15.3:0/3537280414 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8b180f1d30 con 0x7f8b28107bc0 2024-03-21T08:31:32.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.864+0000 7f8b25400640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b0c0a2f90 0x7f8b0c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:32.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.865+0000 7f8b25400640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b0c0a2f90 0x7f8b0c0a5450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8b2819f310 tx=0x7f8b10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:32.872+0000 7f8b1ea00640 1 -- 172.21.15.3:0/3537280414 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b1804b020 con 0x7f8b28107bc0 2024-03-21T08:31:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.080+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b2802d1f0 con 0x7f8b28107bc0 2024-03-21T08:31:33.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.084+0000 7f8b1ea00640 1 -- 172.21.15.3:0/3537280414 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7f8b180be6c0 con 0x7f8b28107bc0 2024-03-21T08:31:33.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:33.086 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b0c0a2f90 msgr2=0x7f8b0c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b0c0a2f90 0x7f8b0c0a5450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8b2819f310 tx=0x7f8b10005f90 comp rx=0 tx=0).stop 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28107bc0 msgr2=0x7f8b2819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28107bc0 0x7f8b2819de00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b1802c630 tx=0x7f8b180328a0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 shutdown_connections 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b0c0a2f90 0x7f8b0c0a5450 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b28109770 0x7f8b281a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b28108db0 0x7f8b2819e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 --2- 172.21.15.3:0/3537280414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28107bc0 0x7f8b2819de00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 >> 172.21.15.3:0/3537280414 conn(0x7f8b28076b60 msgr2=0x7f8b2810c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:33.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 shutdown_connections 2024-03-21T08:31:33.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.087+0000 7f8b27200640 1 -- 172.21.15.3:0/3537280414 wait complete. 2024-03-21T08:31:33.109 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:31:33.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:31:33.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 -- 172.21.15.3:0/449836205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0048104ab0 msgr2=0x7f0048104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 --2- 172.21.15.3:0/449836205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0048104ab0 0x7f0048104f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0034009900 tx=0x7f003402bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:33.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 -- 172.21.15.3:0/449836205 shutdown_connections 2024-03-21T08:31:33.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 --2- 172.21.15.3:0/449836205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048105470 0x7f004810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 --2- 172.21.15.3:0/449836205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0048104ab0 0x7f0048104f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 --2- 172.21.15.3:0/449836205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00481038c0 0x7f0048103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 -- 172.21.15.3:0/449836205 >> 172.21.15.3:0/449836205 conn(0x7f00480fee50 msgr2=0x7f0048101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:33.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 -- 172.21.15.3:0/449836205 shutdown_connections 2024-03-21T08:31:33.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.333+0000 7f0050600640 1 -- 172.21.15.3:0/449836205 wait complete. 2024-03-21T08:31:33.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 Processor -- start 2024-03-21T08:31:33.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 -- start start 2024-03-21T08:31:33.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00481038c0 0x7f0048199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048104ab0 0x7f004819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048105470 0x7f00481a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004810f450 con 0x7f0048105470 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004810f2d0 con 0x7f00481038c0 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f0050600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004810f5d0 con 0x7f0048104ab0 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.335+0000 7f004e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048105470 0x7f00481a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004dc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00481038c0 0x7f0048199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048105470 0x7f00481a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34156/0 (socket says 172.21.15.3:34156) 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 -- 172.21.15.3:0/76324805 learned_addr learned my addr 172.21.15.3:0/76324805 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:33.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004d200640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048104ab0 0x7f004819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 -- 172.21.15.3:0/76324805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048104ab0 msgr2=0x7f004819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048104ab0 0x7f004819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 -- 172.21.15.3:0/76324805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00481038c0 msgr2=0x7f0048199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00481038c0 0x7f0048199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 -- 172.21.15.3:0/76324805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00481a17a0 con 0x7f0048105470 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.336+0000 7f004e600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048105470 0x7f00481a1130 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f004000b970 tx=0x7f004000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.337+0000 7f0046000640 1 -- 172.21.15.3:0/76324805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040019070 con 0x7f0048105470 2024-03-21T08:31:33.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.337+0000 7f0046000640 1 -- 172.21.15.3:0/76324805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0040014440 con 0x7f0048105470 2024-03-21T08:31:33.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.337+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00481a1a90 con 0x7f0048105470 2024-03-21T08:31:33.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.337+0000 7f0046000640 1 -- 172.21.15.3:0/76324805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400075d0 con 0x7f0048105470 2024-03-21T08:31:33.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.337+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00481a2060 con 0x7f0048105470 2024-03-21T08:31:33.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.338+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004810ac50 con 0x7f0048105470 2024-03-21T08:31:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.340+0000 7f0046000640 1 -- 172.21.15.3:0/76324805 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f004001c070 con 0x7f0048105470 2024-03-21T08:31:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.341+0000 7f0046000640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f00300a2fa0 0x7f00300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.342+0000 7f0046000640 1 -- 172.21.15.3:0/76324805 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f00400ced70 con 0x7f0048105470 2024-03-21T08:31:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.342+0000 7f004dc00640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f00300a2fa0 0x7f00300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.342+0000 7f004dc00640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f00300a2fa0 0x7f00300a5460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0048104910 tx=0x7f003c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:33.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.346+0000 7f0046000640 1 -- 172.21.15.3:0/76324805 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f004009b0b0 con 0x7f0048105470 2024-03-21T08:31:33.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:33 smithi003 ceph-mon[31675]: pgmap v459: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-21T08:31:33.424 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:33 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3994490759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:33.424 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:33 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3537280414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:33.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.554+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00480622a0 con 0x7f0048105470 2024-03-21T08:31:33.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.555+0000 7f0046000640 1 -- 172.21.15.3:0/76324805 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f004009f150 con 0x7f0048105470 2024-03-21T08:31:33.557 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f00300a2fa0 msgr2=0x7f00300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f00300a2fa0 0x7f00300a5460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0048104910 tx=0x7f003c005f90 comp rx=0 tx=0).stop 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048105470 msgr2=0x7f00481a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048105470 0x7f00481a1130 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f004000b970 tx=0x7f004000be30 comp rx=0 tx=0).stop 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 shutdown_connections 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048105470 0x7f00481a1130 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f00300a2fa0 0x7f00300a5460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048104ab0 0x7f004819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 --2- 172.21.15.3:0/76324805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00481038c0 0x7f0048199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 >> 172.21.15.3:0/76324805 conn(0x7f00480fee50 msgr2=0x7f0048107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 shutdown_connections 2024-03-21T08:31:33.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.558+0000 7f0050600640 1 -- 172.21.15.3:0/76324805 wait complete. 2024-03-21T08:31:33.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:31:33.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:33 smithi084 ceph-mon[35477]: pgmap v459: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-21T08:31:33.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:33 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3994490759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:33.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:33 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3537280414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:33 smithi171 ceph-mon[35388]: pgmap v459: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-21T08:31:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:33 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3994490759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:33.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:33 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3537280414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 -- 172.21.15.3:0/2024709002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 msgr2=0x7fd380102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 --2- 172.21.15.3:0/2024709002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd380102190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd370009ca0 tx=0x7fd37002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 -- 172.21.15.3:0/2024709002 shutdown_connections 2024-03-21T08:31:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 --2- 172.21.15.3:0/2024709002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd380105080 0x7fd380105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 --2- 172.21.15.3:0/2024709002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3801026d0 0x7fd380104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 --2- 172.21.15.3:0/2024709002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd380102190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 -- 172.21.15.3:0/2024709002 >> 172.21.15.3:0/2024709002 conn(0x7fd3800fb980 msgr2=0x7fd3800fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 -- 172.21.15.3:0/2024709002 shutdown_connections 2024-03-21T08:31:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.804+0000 7fd386e00640 1 -- 172.21.15.3:0/2024709002 wait complete. 2024-03-21T08:31:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.805+0000 7fd386e00640 1 Processor -- start 2024-03-21T08:31:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd386e00640 1 -- start start 2024-03-21T08:31:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd386e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd38019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd386e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3801026d0 0x7fd38019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd386e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd380105080 0x7fd3801a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd386e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801135f0 con 0x7fd3800ffd60 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd386e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380113470 con 0x7fd3801026d0 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd386e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380113770 con 0x7fd380105080 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.806+0000 7fd385a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd38019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd38019dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34164/0 (socket says 172.21.15.3:34164) 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 -- 172.21.15.3:0/3163918460 learned_addr learned my addr 172.21.15.3:0/3163918460 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385000640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3801026d0 0x7fd38019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd386400640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd380105080 0x7fd3801a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 -- 172.21.15.3:0/3163918460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd380105080 msgr2=0x7fd3801a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd380105080 0x7fd3801a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 -- 172.21.15.3:0/3163918460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3801026d0 msgr2=0x7fd38019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3801026d0 0x7fd38019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 -- 172.21.15.3:0/3163918460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3801a5930 con 0x7fd3800ffd60 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.807+0000 7fd385a00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd38019dd20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd370009dc0 tx=0x7fd3700328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.808+0000 7fd37e000640 1 -- 172.21.15.3:0/3163918460 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370041020 con 0x7fd3800ffd60 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.808+0000 7fd37e000640 1 -- 172.21.15.3:0/3163918460 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd370003e20 con 0x7fd3800ffd60 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.808+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3801a5bc0 con 0x7fd3800ffd60 2024-03-21T08:31:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.808+0000 7fd37e000640 1 -- 172.21.15.3:0/3163918460 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370039040 con 0x7fd3800ffd60 2024-03-21T08:31:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.808+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3801a60b0 con 0x7fd3800ffd60 2024-03-21T08:31:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.809+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3800fd470 con 0x7fd3800ffd60 2024-03-21T08:31:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.811+0000 7fd37e000640 1 -- 172.21.15.3:0/3163918460 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fd370044020 con 0x7fd3800ffd60 2024-03-21T08:31:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.813+0000 7fd37e000640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3640a2f20 0x7fd3640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:33.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.813+0000 7fd37e000640 1 -- 172.21.15.3:0/3163918460 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd3700f1b60 con 0x7fd3800ffd60 2024-03-21T08:31:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.813+0000 7fd385000640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3640a2f20 0x7fd3640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:33.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.814+0000 7fd385000640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3640a2f20 0x7fd3640a53e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009bd0 tx=0x7fd36c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:33.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:33.817+0000 7fd37e000640 1 -- 172.21.15.3:0/3163918460 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd37004b020 con 0x7fd3800ffd60 2024-03-21T08:31:34.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.024+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3800622d0 con 0x7fd3800ffd60 2024-03-21T08:31:34.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.025+0000 7fd37e000640 1 -- 172.21.15.3:0/3163918460 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3700be550 con 0x7fd3800ffd60 2024-03-21T08:31:34.027 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3640a2f20 msgr2=0x7fd3640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3640a2f20 0x7fd3640a53e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009bd0 tx=0x7fd36c005f90 comp rx=0 tx=0).stop 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 msgr2=0x7fd38019dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd38019dd20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd370009dc0 tx=0x7fd3700328a0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 shutdown_connections 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fd3640a2f20 0x7fd3640a53e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd380105080 0x7fd3801a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3801026d0 0x7fd38019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 --2- 172.21.15.3:0/3163918460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3800ffd60 0x7fd38019dd20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 >> 172.21.15.3:0/3163918460 conn(0x7fd3800fb980 msgr2=0x7fd3801031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:34.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 shutdown_connections 2024-03-21T08:31:34.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.028+0000 7fd386e00640 1 -- 172.21.15.3:0/3163918460 wait complete. 2024-03-21T08:31:34.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:31:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 -- 172.21.15.3:0/2715873112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 msgr2=0x7f5838109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 --2- 172.21.15.3:0/2715873112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f5838109af0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5830009900 tx=0x7f583002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 -- 172.21.15.3:0/2715873112 shutdown_connections 2024-03-21T08:31:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 --2- 172.21.15.3:0/2715873112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f5838109af0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 --2- 172.21.15.3:0/2715873112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58381028a0 0x7f5838102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 --2- 172.21.15.3:0/2715873112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381016b0 0x7f5838101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 -- 172.21.15.3:0/2715873112 >> 172.21.15.3:0/2715873112 conn(0x7f58380fcc20 msgr2=0x7f58380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 -- 172.21.15.3:0/2715873112 shutdown_connections 2024-03-21T08:31:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.274+0000 7f583fc00640 1 -- 172.21.15.3:0/2715873112 wait complete. 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 Processor -- start 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 -- start start 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58381016b0 0x7f5838199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381028a0 0x7f5838199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f58381a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583810d230 con 0x7f5838103260 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583810d0b0 con 0x7f58381028a0 2024-03-21T08:31:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.275+0000 7f583fc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583810d3b0 con 0x7f58381016b0 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f583dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f58381a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381028a0 0x7f5838199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f583dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f58381a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34176/0 (socket says 172.21.15.3:34176) 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381028a0 0x7f5838199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55178/0 (socket says 172.21.15.3:55178) 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f583dc00640 1 -- 172.21.15.3:0/953330569 learned_addr learned my addr 172.21.15.3:0/953330569 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f583d200640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58381016b0 0x7f5838199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 -- 172.21.15.3:0/953330569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58381016b0 msgr2=0x7f5838199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58381016b0 0x7f5838199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 -- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 msgr2=0x7f58381a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f58381a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 -- 172.21.15.3:0/953330569 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58381a16a0 con 0x7f58381028a0 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f583dc00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f58381a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.276+0000 7f5837e00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381028a0 0x7f5838199fc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f582000d720 tx=0x7f582000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.277+0000 7f5835600640 1 -- 172.21.15.3:0/953330569 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820017070 con 0x7f58381028a0 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.277+0000 7f5835600640 1 -- 172.21.15.3:0/953330569 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f582000de30 con 0x7f58381028a0 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.277+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58381a1990 con 0x7f58381028a0 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.277+0000 7f5835600640 1 -- 172.21.15.3:0/953330569 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200047a0 con 0x7f58381028a0 2024-03-21T08:31:34.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.277+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58381a1ee0 con 0x7f58381028a0 2024-03-21T08:31:34.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.278+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58381a2110 con 0x7f58381028a0 2024-03-21T08:31:34.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.280+0000 7f5835600640 1 -- 172.21.15.3:0/953330569 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f58200099d0 con 0x7f58381028a0 2024-03-21T08:31:34.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.282+0000 7f5835600640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f581c0a2fa0 0x7f581c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.282+0000 7f5835600640 1 -- 172.21.15.3:0/953330569 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f58200cf780 con 0x7f58381028a0 2024-03-21T08:31:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.282+0000 7f583d200640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f581c0a2fa0 0x7f581c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.283+0000 7f583d200640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f581c0a2fa0 0x7f581c0a5460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5838102700 tx=0x7f5828009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:34.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.285+0000 7f5835600640 1 -- 172.21.15.3:0/953330569 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5820020020 con 0x7f58381028a0 2024-03-21T08:31:34.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:34 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/76324805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:34.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:34 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3163918460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:34.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.486+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58381a22e0 con 0x7f58381028a0 2024-03-21T08:31:34.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.488+0000 7f5835600640 1 -- 172.21.15.3:0/953330569 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f582009c190 con 0x7f58381028a0 2024-03-21T08:31:34.489 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f581c0a2fa0 msgr2=0x7f581c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f581c0a2fa0 0x7f581c0a5460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5838102700 tx=0x7f5828009500 comp rx=0 tx=0).stop 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381028a0 msgr2=0x7f5838199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381028a0 0x7f5838199fc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f582000d720 tx=0x7f582000dbe0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 shutdown_connections 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5838103260 0x7f58381a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58381028a0 0x7f5838199fc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f581c0a2fa0 0x7f581c0a5460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 --2- 172.21.15.3:0/953330569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58381016b0 0x7f5838199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.490+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 >> 172.21.15.3:0/953330569 conn(0x7f58380fcc20 msgr2=0x7f58380738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:34.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.491+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 shutdown_connections 2024-03-21T08:31:34.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.491+0000 7f583fc00640 1 -- 172.21.15.3:0/953330569 wait complete. 2024-03-21T08:31:34.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:31:34.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:34 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/76324805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:34.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:34 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3163918460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:34.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:34 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/76324805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:34.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:34 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3163918460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 -- 172.21.15.3:0/759458761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 msgr2=0x7f1944101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 --2- 172.21.15.3:0/759458761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f1944101930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1934009ca0 tx=0x7f193402c100 comp rx=0 tx=0).stop 2024-03-21T08:31:34.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 -- 172.21.15.3:0/759458761 shutdown_connections 2024-03-21T08:31:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 --2- 172.21.15.3:0/759458761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19441030c0 0x7f1944109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 --2- 172.21.15.3:0/759458761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1944102700 0x7f1944102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 --2- 172.21.15.3:0/759458761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f1944101930 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 -- 172.21.15.3:0/759458761 >> 172.21.15.3:0/759458761 conn(0x7f19440fca80 msgr2=0x7f19440feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 -- 172.21.15.3:0/759458761 shutdown_connections 2024-03-21T08:31:34.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.721+0000 7f1949200640 1 -- 172.21.15.3:0/759458761 wait complete. 2024-03-21T08:31:34.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.722+0000 7f1949200640 1 Processor -- start 2024-03-21T08:31:34.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.722+0000 7f1949200640 1 -- start start 2024-03-21T08:31:34.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1949200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f19441976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1949200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1944102700 0x7f1944197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1949200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19441030c0 0x7f194419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1949200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194410d050 con 0x7f1944101510 2024-03-21T08:31:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1949200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194410ced0 con 0x7f19441030c0 2024-03-21T08:31:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1949200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194410d1d0 con 0x7f1944102700 2024-03-21T08:31:34.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1943400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f19441976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1943400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f19441976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34194/0 (socket says 172.21.15.3:34194) 2024-03-21T08:31:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1943400640 1 -- 172.21.15.3:0/618550649 learned_addr learned my addr 172.21.15.3:0/618550649 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.723+0000 7f1943e00640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19441030c0 0x7f194419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f193aa00640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1944102700 0x7f1944197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1943400640 1 -- 172.21.15.3:0/618550649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1944102700 msgr2=0x7f1944197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1943400640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1944102700 0x7f1944197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1943400640 1 -- 172.21.15.3:0/618550649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19441030c0 msgr2=0x7f194419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1943400640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19441030c0 0x7f194419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1943400640 1 -- 172.21.15.3:0/618550649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194419f2d0 con 0x7f1944101510 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f193aa00640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1944102700 0x7f1944197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1943400640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f19441976b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1934032380 tx=0x7f1934035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1940c00640 1 -- 172.21.15.3:0/618550649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934039040 con 0x7f1944101510 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1940c00640 1 -- 172.21.15.3:0/618550649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1934044070 con 0x7f1944101510 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1940c00640 1 -- 172.21.15.3:0/618550649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19340045a0 con 0x7f1944101510 2024-03-21T08:31:34.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.724+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194419f560 con 0x7f1944101510 2024-03-21T08:31:34.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.725+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194419fa00 con 0x7f1944101510 2024-03-21T08:31:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.726+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1944108830 con 0x7f1944101510 2024-03-21T08:31:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.728+0000 7f1940c00640 1 -- 172.21.15.3:0/618550649 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f1934003970 con 0x7f1944101510 2024-03-21T08:31:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.729+0000 7f1940c00640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19280a2f20 0x7f19280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.729+0000 7f1940c00640 1 -- 172.21.15.3:0/618550649 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f19340f1de0 con 0x7f1944101510 2024-03-21T08:31:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.729+0000 7f193aa00640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19280a2f20 0x7f19280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:34.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.730+0000 7f193aa00640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19280a2f20 0x7f19280a53e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1944198bc0 tx=0x7f192c005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.733+0000 7f1940c00640 1 -- 172.21.15.3:0/618550649 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f193404e020 con 0x7f1944101510 2024-03-21T08:31:34.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.937+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f19440622a0 con 0x7f1944101510 2024-03-21T08:31:34.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.939+0000 7f1940c00640 1 -- 172.21.15.3:0/618550649 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19340be770 con 0x7f1944101510 2024-03-21T08:31:34.941 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:34.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19280a2f20 msgr2=0x7f19280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19280a2f20 0x7f19280a53e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1944198bc0 tx=0x7f192c005d90 comp rx=0 tx=0).stop 2024-03-21T08:31:34.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 msgr2=0x7f19441976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f19441976b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1934032380 tx=0x7f1934035ce0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 shutdown_connections 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19441030c0 0x7f194419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f19280a2f20 0x7f19280a53e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1944102700 0x7f1944197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 --2- 172.21.15.3:0/618550649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1944101510 0x7f19441976b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.941+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 >> 172.21.15.3:0/618550649 conn(0x7f19440fca80 msgr2=0x7f1944105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.942+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 shutdown_connections 2024-03-21T08:31:34.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:34.942+0000 7f1949200640 1 -- 172.21.15.3:0/618550649 wait complete. 2024-03-21T08:31:34.962 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-21T08:31:34.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:31:35.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 -- 172.21.15.3:0/987773810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64105370 msgr2=0x7f8c6410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 --2- 172.21.15.3:0/987773810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64105370 0x7f8c6410bc00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009900 tx=0x7f8c5c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:35.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 -- 172.21.15.3:0/987773810 shutdown_connections 2024-03-21T08:31:35.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 --2- 172.21.15.3:0/987773810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64105370 0x7f8c6410bc00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 --2- 172.21.15.3:0/987773810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c641049b0 0x7f8c64104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 --2- 172.21.15.3:0/987773810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c641037c0 0x7f8c64103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 -- 172.21.15.3:0/987773810 >> 172.21.15.3:0/987773810 conn(0x7f8c640fed50 msgr2=0x7f8c64101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:35.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 -- 172.21.15.3:0/987773810 shutdown_connections 2024-03-21T08:31:35.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.172+0000 7f8c63e00640 1 -- 172.21.15.3:0/987773810 wait complete. 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.173+0000 7f8c63e00640 1 Processor -- start 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.173+0000 7f8c63e00640 1 -- start start 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c641037c0 0x7f8c641998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c641049b0 0x7f8c64199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c64105370 0x7f8c641a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c63e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6410f2d0 con 0x7f8c641049b0 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c63e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6410f150 con 0x7f8c641037c0 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c63e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6410f450 con 0x7f8c64105370 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c62000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c641049b0 0x7f8c64199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c62000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c641049b0 0x7f8c64199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34216/0 (socket says 172.21.15.3:34216) 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c62000640 1 -- 172.21.15.3:0/3493495700 learned_addr learned my addr 172.21.15.3:0/3493495700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.174+0000 7f8c62a00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c641037c0 0x7f8c641998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c63400640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c64105370 0x7f8c641a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c62000640 1 -- 172.21.15.3:0/3493495700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c64105370 msgr2=0x7f8c641a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c62000640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c64105370 0x7f8c641a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c62000640 1 -- 172.21.15.3:0/3493495700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c641037c0 msgr2=0x7f8c641998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c62000640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c641037c0 0x7f8c641998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c62000640 1 -- 172.21.15.3:0/3493495700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c641a14e0 con 0x7f8c641049b0 2024-03-21T08:31:35.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c62000640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c641049b0 0x7f8c64199e00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000d950 tx=0x7f8c5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:35.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c5b400640 1 -- 172.21.15.3:0/3493495700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50017070 con 0x7f8c641049b0 2024-03-21T08:31:35.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c5b400640 1 -- 172.21.15.3:0/3493495700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c50009c30 con 0x7f8c641049b0 2024-03-21T08:31:35.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c641a17d0 con 0x7f8c641049b0 2024-03-21T08:31:35.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.175+0000 7f8c5b400640 1 -- 172.21.15.3:0/3493495700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5000f620 con 0x7f8c641049b0 2024-03-21T08:31:35.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.176+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c641a1d20 con 0x7f8c641049b0 2024-03-21T08:31:35.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.177+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6410aae0 con 0x7f8c641049b0 2024-03-21T08:31:35.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.179+0000 7f8c5b400640 1 -- 172.21.15.3:0/3493495700 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f8c500109a0 con 0x7f8c641049b0 2024-03-21T08:31:35.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.181+0000 7f8c5b400640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c480a2f10 0x7f8c480a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.181+0000 7f8c5b400640 1 -- 172.21.15.3:0/3493495700 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8c500cf4d0 con 0x7f8c641049b0 2024-03-21T08:31:35.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.181+0000 7f8c62a00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c480a2f10 0x7f8c480a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.182+0000 7f8c62a00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c480a2f10 0x7f8c480a53d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8c64104810 tx=0x7f8c54007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:35.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.185+0000 7f8c5b400640 1 -- 172.21.15.3:0/3493495700 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c50009de0 con 0x7f8c641049b0 2024-03-21T08:31:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.431+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c640622a0 con 0x7f8c641049b0 2024-03-21T08:31:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.433+0000 7f8c5b400640 1 -- 172.21.15.3:0/3493495700 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f8c5009bec0 con 0x7f8c641049b0 2024-03-21T08:31:35.435 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:35.435 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:31:29.351404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34472,34409]},"id":1}]} 2024-03-21T08:31:35.435 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-03-21T08:31:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c480a2f10 msgr2=0x7f8c480a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c480a2f10 0x7f8c480a53d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8c64104810 tx=0x7f8c54007600 comp rx=0 tx=0).stop 2024-03-21T08:31:35.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c641049b0 msgr2=0x7f8c64199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c641049b0 0x7f8c64199e00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000d950 tx=0x7f8c5000de10 comp rx=0 tx=0).stop 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 shutdown_connections 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8c480a2f10 0x7f8c480a53d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c64105370 0x7f8c641a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c641049b0 0x7f8c64199e00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 --2- 172.21.15.3:0/3493495700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c641037c0 0x7f8c641998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.435+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 >> 172.21.15.3:0/3493495700 conn(0x7f8c640fed50 msgr2=0x7f8c64107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.436+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 shutdown_connections 2024-03-21T08:31:35.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.436+0000 7f8c63e00640 1 -- 172.21.15.3:0/3493495700 wait complete. 2024-03-21T08:31:35.456 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-21T08:31:35.457 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-21T08:31:35.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T08:31:35.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:35 smithi003 ceph-mon[31675]: pgmap v460: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-21T08:31:35.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:35 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/953330569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:35.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:35 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/618550649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:35.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:35 smithi084 ceph-mon[35477]: pgmap v460: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-21T08:31:35.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:35 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/953330569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:35.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:35 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/618550649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:35.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:35 smithi171 ceph-mon[35388]: pgmap v460: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-21T08:31:35.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:35 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/953330569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:35.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:35 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/618550649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:35.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 -- 172.21.15.3:0/3168346706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404109770 msgr2=0x7ff404110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 --2- 172.21.15.3:0/3168346706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404109770 0x7ff404110000 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009900 tx=0x7ff3fc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 -- 172.21.15.3:0/3168346706 shutdown_connections 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 --2- 172.21.15.3:0/3168346706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404109770 0x7ff404110000 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 --2- 172.21.15.3:0/3168346706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404108db0 0x7ff404109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 --2- 172.21.15.3:0/3168346706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404107bc0 0x7ff404107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 -- 172.21.15.3:0/3168346706 >> 172.21.15.3:0/3168346706 conn(0x7ff404076b60 msgr2=0x7ff404078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 -- 172.21.15.3:0/3168346706 shutdown_connections 2024-03-21T08:31:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.682+0000 7ff403200640 1 -- 172.21.15.3:0/3168346706 wait complete. 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 Processor -- start 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 -- start start 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404107bc0 0x7ff40419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404108db0 0x7ff40419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404109770 0x7ff4041a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404113740 con 0x7ff404107bc0 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4041135c0 con 0x7ff404108db0 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff403200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4041138c0 con 0x7ff404109770 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff401e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404107bc0 0x7ff40419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff401e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404107bc0 0x7ff40419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34232/0 (socket says 172.21.15.3:34232) 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.683+0000 7ff401e00640 1 -- 172.21.15.3:0/598337010 learned_addr learned my addr 172.21.15.3:0/598337010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff401400640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404108db0 0x7ff40419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff402800640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404109770 0x7ff4041a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff401e00640 1 -- 172.21.15.3:0/598337010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404109770 msgr2=0x7ff4041a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff401e00640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404109770 0x7ff4041a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff401e00640 1 -- 172.21.15.3:0/598337010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404108db0 msgr2=0x7ff40419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff401e00640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404108db0 0x7ff40419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff401e00640 1 -- 172.21.15.3:0/598337010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4041a59b0 con 0x7ff404107bc0 2024-03-21T08:31:35.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.684+0000 7ff401e00640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404107bc0 0x7ff40419dda0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:35.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.685+0000 7ff3faa00640 1 -- 172.21.15.3:0/598337010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff404107bc0 2024-03-21T08:31:35.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.685+0000 7ff3faa00640 1 -- 172.21.15.3:0/598337010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff3f4009c30 con 0x7ff404107bc0 2024-03-21T08:31:35.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.685+0000 7ff3faa00640 1 -- 172.21.15.3:0/598337010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400f620 con 0x7ff404107bc0 2024-03-21T08:31:35.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.685+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4041a5ca0 con 0x7ff404107bc0 2024-03-21T08:31:35.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.685+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4041a61a0 con 0x7ff404107bc0 2024-03-21T08:31:35.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.686+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40410ee80 con 0x7ff404107bc0 2024-03-21T08:31:35.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.688+0000 7ff3faa00640 1 -- 172.21.15.3:0/598337010 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7ff3f401a070 con 0x7ff404107bc0 2024-03-21T08:31:35.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.689+0000 7ff3faa00640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff3e40a2f90 0x7ff3e40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:35.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.690+0000 7ff3faa00640 1 -- 172.21.15.3:0/598337010 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff3f40cf310 con 0x7ff404107bc0 2024-03-21T08:31:35.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.690+0000 7ff401400640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff3e40a2f90 0x7ff3e40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:35.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.691+0000 7ff401400640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff3e40a2f90 0x7ff3e40a5450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec005fd0 tx=0x7ff3ec009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:35.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.694+0000 7ff3faa00640 1 -- 172.21.15.3:0/598337010 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff3f4009de0 con 0x7ff404107bc0 2024-03-21T08:31:35.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.894+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff404061ae0 con 0x7ff3e40a2f90 2024-03-21T08:31:35.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.912+0000 7ff3faa00640 1 -- 172.21.15.3:0/598337010 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340532 (secure 0 0 0) 0x7ff404061ae0 con 0x7ff3e40a2f90 2024-03-21T08:31:35.919 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:35.921 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-03-21T08:31:35.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff3e40a2f90 msgr2=0x7ff3e40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff3e40a2f90 0x7ff3e40a5450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec005fd0 tx=0x7ff3ec009760 comp rx=0 tx=0).stop 2024-03-21T08:31:35.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404107bc0 msgr2=0x7ff40419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:35.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404107bc0 0x7ff40419dda0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).stop 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 shutdown_connections 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7ff3e40a2f90 0x7ff3e40a5450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404109770 0x7ff4041a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff404108db0 0x7ff40419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 --2- 172.21.15.3:0/598337010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff404107bc0 0x7ff40419dda0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 >> 172.21.15.3:0/598337010 conn(0x7ff404076b60 msgr2=0x7ff40410c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.917+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 shutdown_connections 2024-03-21T08:31:35.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:35.918+0000 7ff403200640 1 -- 172.21.15.3:0/598337010 wait complete. 2024-03-21T08:31:35.943 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-03-21T08:31:35.414486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663119,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1040,"num_read_kb":649,"num_write":364,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372344,"kb_used_data":14872,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569480,"statfs":{"total":1151940427776,"available":1151559147520,"internally_reserved":0,"allocated":15228928,"data_stored":8373173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005448"},"pg_stats":[{"pgid":"3.3f","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332311+0000","last_change":"2024-03-21T08:28:36.667851+0000","last_active":"2024-03-21T08:30:35.332311+0000","last_peered":"2024-03-21T08:30:35.332311+0000","last_clean":"2024-03-21T08:30:35.332311+0000","last_became_active":"2024-03-21T08:28:36.665713+0000","last_became_peered":"2024-03-21T08:28:36.665713+0000","last_unstale":"2024-03-21T08:30:35.332311+0000","last_undegraded":"2024-03-21T08:30:35.332311+0000","last_fullsized":"2024-03-21T08:30:35.332311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:02.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329013+0000","last_change":"2024-03-21T08:28:36.650986+0000","last_active":"2024-03-21T08:30:35.329013+0000","last_peered":"2024-03-21T08:30:35.329013+0000","last_clean":"2024-03-21T08:30:35.329013+0000","last_became_active":"2024-03-21T08:28:36.650633+0000","last_became_peered":"2024-03-21T08:28:36.650633+0000","last_unstale":"2024-03-21T08:30:35.329013+0000","last_undegraded":"2024-03-21T08:30:35.329013+0000","last_fullsized":"2024-03-21T08:30:35.329013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:40:20.801814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.335139+0000","last_change":"2024-03-21T08:28:36.662984+0000","last_active":"2024-03-21T08:30:35.335139+0000","last_peered":"2024-03-21T08:30:35.335139+0000","last_clean":"2024-03-21T08:30:35.335139+0000","last_became_active":"2024-03-21T08:28:36.662657+0000","last_became_peered":"2024-03-21T08:28:36.662657+0000","last_unstale":"2024-03-21T08:30:35.335139+0000","last_undegraded":"2024-03-21T08:30:35.335139+0000","last_fullsized":"2024-03-21T08:30:35.335139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:41.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.299562+0000","last_change":"2024-03-21T08:28:36.665958+0000","last_active":"2024-03-21T08:30:35.299562+0000","last_peered":"2024-03-21T08:30:35.299562+0000","last_clean":"2024-03-21T08:30:35.299562+0000","last_became_active":"2024-03-21T08:28:36.664309+0000","last_became_peered":"2024-03-21T08:28:36.664309+0000","last_unstale":"2024-03-21T08:30:35.299562+0000","last_undegraded":"2024-03-21T08:30:35.299562+0000","last_fullsized":"2024-03-21T08:30:35.299562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:16.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315553+0000","last_change":"2024-03-21T08:28:36.657767+0000","last_active":"2024-03-21T08:30:35.315553+0000","last_peered":"2024-03-21T08:30:35.315553+0000","last_clean":"2024-03-21T08:30:35.315553+0000","last_became_active":"2024-03-21T08:28:36.657350+0000","last_became_peered":"2024-03-21T08:28:36.657350+0000","last_unstale":"2024-03-21T08:30:35.315553+0000","last_undegraded":"2024-03-21T08:30:35.315553+0000","last_fullsized":"2024-03-21T08:30:35.315553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:47.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312622+0000","last_change":"2024-03-21T08:29:06.916971+0000","last_active":"2024-03-21T08:30:35.312622+0000","last_peered":"2024-03-21T08:30:35.312622+0000","last_clean":"2024-03-21T08:30:35.312622+0000","last_became_active":"2024-03-21T08:29:06.915997+0000","last_became_peered":"2024-03-21T08:29:06.915997+0000","last_unstale":"2024-03-21T08:30:35.312622+0000","last_undegraded":"2024-03-21T08:30:35.312622+0000","last_fullsized":"2024-03-21T08:30:35.312622+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:02.525190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312579+0000","last_change":"2024-03-21T08:28:36.667787+0000","last_active":"2024-03-21T08:30:35.312579+0000","last_peered":"2024-03-21T08:30:35.312579+0000","last_clean":"2024-03-21T08:30:35.312579+0000","last_became_active":"2024-03-21T08:28:36.667451+0000","last_became_peered":"2024-03-21T08:28:36.667451+0000","last_unstale":"2024-03-21T08:30:35.312579+0000","last_undegraded":"2024-03-21T08:30:35.312579+0000","last_fullsized":"2024-03-21T08:30:35.312579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:01.874897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328289+0000","last_change":"2024-03-21T08:28:36.662445+0000","last_active":"2024-03-21T08:30:35.328289+0000","last_peered":"2024-03-21T08:30:35.328289+0000","last_clean":"2024-03-21T08:30:35.328289+0000","last_became_active":"2024-03-21T08:28:36.662113+0000","last_became_peered":"2024-03-21T08:28:36.662113+0000","last_unstale":"2024-03-21T08:30:35.328289+0000","last_undegraded":"2024-03-21T08:30:35.328289+0000","last_fullsized":"2024-03-21T08:30:35.328289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:21.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331556+0000","last_change":"2024-03-21T08:28:36.666084+0000","last_active":"2024-03-21T08:30:35.331556+0000","last_peered":"2024-03-21T08:30:35.331556+0000","last_clean":"2024-03-21T08:30:35.331556+0000","last_became_active":"2024-03-21T08:28:36.665354+0000","last_became_peered":"2024-03-21T08:28:36.665354+0000","last_unstale":"2024-03-21T08:30:35.331556+0000","last_undegraded":"2024-03-21T08:30:35.331556+0000","last_fullsized":"2024-03-21T08:30:35.331556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:02:50.160765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319234+0000","last_change":"2024-03-21T08:29:06.919231+0000","last_active":"2024-03-21T08:30:35.319234+0000","last_peered":"2024-03-21T08:30:35.319234+0000","last_clean":"2024-03-21T08:30:35.319234+0000","last_became_active":"2024-03-21T08:29:06.918199+0000","last_became_peered":"2024-03-21T08:29:06.918199+0000","last_unstale":"2024-03-21T08:30:35.319234+0000","last_undegraded":"2024-03-21T08:30:35.319234+0000","last_fullsized":"2024-03-21T08:30:35.319234+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:44.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333943+0000","last_change":"2024-03-21T08:28:36.656385+0000","last_active":"2024-03-21T08:30:35.333943+0000","last_peered":"2024-03-21T08:30:35.333943+0000","last_clean":"2024-03-21T08:30:35.333943+0000","last_became_active":"2024-03-21T08:28:36.656014+0000","last_became_peered":"2024-03-21T08:28:36.656014+0000","last_unstale":"2024-03-21T08:30:35.333943+0000","last_undegraded":"2024-03-21T08:30:35.333943+0000","last_fullsized":"2024-03-21T08:30:35.333943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:46.000495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319565+0000","last_change":"2024-03-21T08:28:36.666254+0000","last_active":"2024-03-21T08:30:35.319565+0000","last_peered":"2024-03-21T08:30:35.319565+0000","last_clean":"2024-03-21T08:30:35.319565+0000","last_became_active":"2024-03-21T08:28:36.664698+0000","last_became_peered":"2024-03-21T08:28:36.664698+0000","last_unstale":"2024-03-21T08:30:35.319565+0000","last_undegraded":"2024-03-21T08:30:35.319565+0000","last_fullsized":"2024-03-21T08:30:35.319565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:03:08.358159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.327841+0000","last_change":"2024-03-21T08:29:06.919930+0000","last_active":"2024-03-21T08:30:35.327841+0000","last_peered":"2024-03-21T08:30:35.327841+0000","last_clean":"2024-03-21T08:30:35.327841+0000","last_became_active":"2024-03-21T08:29:06.917852+0000","last_became_peered":"2024-03-21T08:29:06.917852+0000","last_unstale":"2024-03-21T08:30:35.327841+0000","last_undegraded":"2024-03-21T08:30:35.327841+0000","last_fullsized":"2024-03-21T08:30:35.327841+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:05:01.185388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328029+0000","last_change":"2024-03-21T08:28:36.655229+0000","last_active":"2024-03-21T08:30:35.328029+0000","last_peered":"2024-03-21T08:30:35.328029+0000","last_clean":"2024-03-21T08:30:35.328029+0000","last_became_active":"2024-03-21T08:28:36.653949+0000","last_became_peered":"2024-03-21T08:28:36.653949+0000","last_unstale":"2024-03-21T08:30:35.328029+0000","last_undegraded":"2024-03-21T08:30:35.328029+0000","last_fullsized":"2024-03-21T08:30:35.328029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:57:38.370897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328261+0000","last_change":"2024-03-21T08:28:36.664894+0000","last_active":"2024-03-21T08:30:35.328261+0000","last_peered":"2024-03-21T08:30:35.328261+0000","last_clean":"2024-03-21T08:30:35.328261+0000","last_became_active":"2024-03-21T08:28:36.664523+0000","last_became_peered":"2024-03-21T08:28:36.664523+0000","last_unstale":"2024-03-21T08:30:35.328261+0000","last_undegraded":"2024-03-21T08:30:35.328261+0000","last_fullsized":"2024-03-21T08:30:35.328261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:52:54.147459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333446+0000","last_change":"2024-03-21T08:28:36.662374+0000","last_active":"2024-03-21T08:30:35.333446+0000","last_peered":"2024-03-21T08:30:35.333446+0000","last_clean":"2024-03-21T08:30:35.333446+0000","last_became_active":"2024-03-21T08:28:36.662066+0000","last_became_peered":"2024-03-21T08:28:36.662066+0000","last_unstale":"2024-03-21T08:30:35.333446+0000","last_undegraded":"2024-03-21T08:30:35.333446+0000","last_fullsized":"2024-03-21T08:30:35.333446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:06.082925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332398+0000","last_change":"2024-03-21T08:28:36.672172+0000","last_active":"2024-03-21T08:30:35.332398+0000","last_peered":"2024-03-21T08:30:35.332398+0000","last_clean":"2024-03-21T08:30:35.332398+0000","last_became_active":"2024-03-21T08:28:36.671657+0000","last_became_peered":"2024-03-21T08:28:36.671657+0000","last_unstale":"2024-03-21T08:30:35.332398+0000","last_undegraded":"2024-03-21T08:30:35.332398+0000","last_fullsized":"2024-03-21T08:30:35.332398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:56.892818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332036+0000","last_change":"2024-03-21T08:28:36.666973+0000","last_active":"2024-03-21T08:30:35.332036+0000","last_peered":"2024-03-21T08:30:35.332036+0000","last_clean":"2024-03-21T08:30:35.332036+0000","last_became_active":"2024-03-21T08:28:36.663770+0000","last_became_peered":"2024-03-21T08:28:36.663770+0000","last_unstale":"2024-03-21T08:30:35.332036+0000","last_undegraded":"2024-03-21T08:30:35.332036+0000","last_fullsized":"2024-03-21T08:30:35.332036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:54:26.701009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.318007+0000","last_change":"2024-03-21T08:28:36.668793+0000","last_active":"2024-03-21T08:30:35.318007+0000","last_peered":"2024-03-21T08:30:35.318007+0000","last_clean":"2024-03-21T08:30:35.318007+0000","last_became_active":"2024-03-21T08:28:36.668452+0000","last_became_peered":"2024-03-21T08:28:36.668452+0000","last_unstale":"2024-03-21T08:30:35.318007+0000","last_undegraded":"2024-03-21T08:30:35.318007+0000","last_fullsized":"2024-03-21T08:30:35.318007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:45.191849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645032+0000","last_change":"2024-03-21T08:28:40.687618+0000","last_active":"2024-03-21T08:30:06.645032+0000","last_peered":"2024-03-21T08:30:06.645032+0000","last_clean":"2024-03-21T08:30:06.645032+0000","last_became_active":"2024-03-21T08:28:40.687155+0000","last_became_peered":"2024-03-21T08:28:40.687155+0000","last_unstale":"2024-03-21T08:30:06.645032+0000","last_undegraded":"2024-03-21T08:30:06.645032+0000","last_fullsized":"2024-03-21T08:30:06.645032+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:08.014790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'9","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.673458+0000","last_change":"2024-03-21T08:30:06.672911+0000","last_active":"2024-03-21T08:30:06.673458+0000","last_peered":"2024-03-21T08:30:06.673458+0000","last_clean":"2024-03-21T08:30:06.673458+0000","last_became_active":"2024-03-21T08:30:06.661974+0000","last_became_peered":"2024-03-21T08:30:06.661974+0000","last_unstale":"2024-03-21T08:30:06.673458+0000","last_undegraded":"2024-03-21T08:30:06.673458+0000","last_fullsized":"2024-03-21T08:30:06.673458+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:37:25.848279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313894+0000","last_change":"2024-03-21T08:28:36.670325+0000","last_active":"2024-03-21T08:30:35.313894+0000","last_peered":"2024-03-21T08:30:35.313894+0000","last_clean":"2024-03-21T08:30:35.313894+0000","last_became_active":"2024-03-21T08:28:36.669783+0000","last_became_peered":"2024-03-21T08:28:36.669783+0000","last_unstale":"2024-03-21T08:30:35.313894+0000","last_undegraded":"2024-03-21T08:30:35.313894+0000","last_fullsized":"2024-03-21T08:30:35.313894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:24.166567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663706+0000","last_change":"2024-03-21T08:28:40.689849+0000","last_active":"2024-03-21T08:30:06.663706+0000","last_peered":"2024-03-21T08:30:06.663706+0000","last_clean":"2024-03-21T08:30:06.663706+0000","last_became_active":"2024-03-21T08:28:40.689307+0000","last_became_peered":"2024-03-21T08:28:40.689307+0000","last_unstale":"2024-03-21T08:30:06.663706+0000","last_undegraded":"2024-03-21T08:30:06.663706+0000","last_fullsized":"2024-03-21T08:30:06.663706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:34:05.660317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647944+0000","last_change":"2024-03-21T08:28:35.674685+0000","last_active":"2024-03-21T08:30:06.647944+0000","last_peered":"2024-03-21T08:30:06.647944+0000","last_clean":"2024-03-21T08:30:06.647944+0000","last_became_active":"2024-03-21T08:28:35.674312+0000","last_became_peered":"2024-03-21T08:28:35.674312+0000","last_unstale":"2024-03-21T08:30:06.647944+0000","last_undegraded":"2024-03-21T08:30:06.647944+0000","last_fullsized":"2024-03-21T08:30:06.647944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:27.644111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.335342+0000","last_change":"2024-03-21T08:28:36.667744+0000","last_active":"2024-03-21T08:30:35.335342+0000","last_peered":"2024-03-21T08:30:35.335342+0000","last_clean":"2024-03-21T08:30:35.335342+0000","last_became_active":"2024-03-21T08:28:36.667279+0000","last_became_peered":"2024-03-21T08:28:36.667279+0000","last_unstale":"2024-03-21T08:30:35.335342+0000","last_undegraded":"2024-03-21T08:30:35.335342+0000","last_fullsized":"2024-03-21T08:30:35.335342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:14:24.974683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664879+0000","last_change":"2024-03-21T08:28:40.688864+0000","last_active":"2024-03-21T08:30:06.664879+0000","last_peered":"2024-03-21T08:30:06.664879+0000","last_clean":"2024-03-21T08:30:06.664879+0000","last_became_active":"2024-03-21T08:28:40.688029+0000","last_became_peered":"2024-03-21T08:28:40.688029+0000","last_unstale":"2024-03-21T08:30:06.664879+0000","last_undegraded":"2024-03-21T08:30:06.664879+0000","last_fullsized":"2024-03-21T08:30:06.664879+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:39.235807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.681016+0000","last_change":"2024-03-21T08:30:06.680224+0000","last_active":"2024-03-21T08:30:06.681016+0000","last_peered":"2024-03-21T08:30:06.681016+0000","last_clean":"2024-03-21T08:30:06.681016+0000","last_became_active":"2024-03-21T08:30:06.665898+0000","last_became_peered":"2024-03-21T08:30:06.665898+0000","last_unstale":"2024-03-21T08:30:06.681016+0000","last_undegraded":"2024-03-21T08:30:06.681016+0000","last_fullsized":"2024-03-21T08:30:06.681016+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:21.690576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333879+0000","last_change":"2024-03-21T08:28:36.667244+0000","last_active":"2024-03-21T08:30:35.333879+0000","last_peered":"2024-03-21T08:30:35.333879+0000","last_clean":"2024-03-21T08:30:35.333879+0000","last_became_active":"2024-03-21T08:28:36.666829+0000","last_became_peered":"2024-03-21T08:28:36.666829+0000","last_unstale":"2024-03-21T08:30:35.333879+0000","last_undegraded":"2024-03-21T08:30:35.333879+0000","last_fullsized":"2024-03-21T08:30:35.333879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:12:56.842084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657793+0000","last_change":"2024-03-21T08:28:40.692621+0000","last_active":"2024-03-21T08:30:06.657793+0000","last_peered":"2024-03-21T08:30:06.657793+0000","last_clean":"2024-03-21T08:30:06.657793+0000","last_became_active":"2024-03-21T08:28:40.691984+0000","last_became_peered":"2024-03-21T08:28:40.691984+0000","last_unstale":"2024-03-21T08:30:06.657793+0000","last_undegraded":"2024-03-21T08:30:06.657793+0000","last_fullsized":"2024-03-21T08:30:06.657793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:29.330702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651368+0000","last_change":"2024-03-21T08:28:35.676421+0000","last_active":"2024-03-21T08:30:06.651368+0000","last_peered":"2024-03-21T08:30:06.651368+0000","last_clean":"2024-03-21T08:30:06.651368+0000","last_became_active":"2024-03-21T08:28:35.674957+0000","last_became_peered":"2024-03-21T08:28:35.674957+0000","last_unstale":"2024-03-21T08:30:06.651368+0000","last_undegraded":"2024-03-21T08:30:06.651368+0000","last_fullsized":"2024-03-21T08:30:06.651368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:01:38.397757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331569+0000","last_change":"2024-03-21T08:28:36.664662+0000","last_active":"2024-03-21T08:30:35.331569+0000","last_peered":"2024-03-21T08:30:35.331569+0000","last_clean":"2024-03-21T08:30:35.331569+0000","last_became_active":"2024-03-21T08:28:36.664324+0000","last_became_peered":"2024-03-21T08:28:36.664324+0000","last_unstale":"2024-03-21T08:30:35.331569+0000","last_undegraded":"2024-03-21T08:30:35.331569+0000","last_fullsized":"2024-03-21T08:30:35.331569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:57.325869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645495+0000","last_change":"2024-03-21T08:28:40.676297+0000","last_active":"2024-03-21T08:30:06.645495+0000","last_peered":"2024-03-21T08:30:06.645495+0000","last_clean":"2024-03-21T08:30:06.645495+0000","last_became_active":"2024-03-21T08:28:40.675337+0000","last_became_peered":"2024-03-21T08:28:40.675337+0000","last_unstale":"2024-03-21T08:30:06.645495+0000","last_undegraded":"2024-03-21T08:30:06.645495+0000","last_fullsized":"2024-03-21T08:30:06.645495+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:05.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'8","reported_seq":189,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:31.737556+0000","last_change":"2024-03-21T08:28:35.687651+0000","last_active":"2024-03-21T08:31:31.737556+0000","last_peered":"2024-03-21T08:31:31.737556+0000","last_clean":"2024-03-21T08:31:31.737556+0000","last_became_active":"2024-03-21T08:28:35.686284+0000","last_became_peered":"2024-03-21T08:28:35.686284+0000","last_unstale":"2024-03-21T08:31:31.737556+0000","last_undegraded":"2024-03-21T08:31:31.737556+0000","last_fullsized":"2024-03-21T08:31:31.737556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:56.121798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315336+0000","last_change":"2024-03-21T08:29:07.033436+0000","last_active":"2024-03-21T08:30:35.315336+0000","last_peered":"2024-03-21T08:30:35.315336+0000","last_clean":"2024-03-21T08:30:35.315336+0000","last_became_active":"2024-03-21T08:29:07.032012+0000","last_became_peered":"2024-03-21T08:29:07.032012+0000","last_unstale":"2024-03-21T08:30:35.315336+0000","last_undegraded":"2024-03-21T08:30:35.315336+0000","last_fullsized":"2024-03-21T08:30:35.315336+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:44:47.009078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645774+0000","last_change":"2024-03-21T08:29:06.882449+0000","last_active":"2024-03-21T08:30:06.645774+0000","last_peered":"2024-03-21T08:30:06.645774+0000","last_clean":"2024-03-21T08:30:06.645774+0000","last_became_active":"2024-03-21T08:29:06.880682+0000","last_became_peered":"2024-03-21T08:29:06.880682+0000","last_unstale":"2024-03-21T08:30:06.645774+0000","last_undegraded":"2024-03-21T08:30:06.645774+0000","last_fullsized":"2024-03-21T08:30:06.645774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:09.464388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.661857+0000","last_change":"2024-03-21T08:30:06.659993+0000","last_active":"2024-03-21T08:30:06.661857+0000","last_peered":"2024-03-21T08:30:06.661857+0000","last_clean":"2024-03-21T08:30:06.661857+0000","last_became_active":"2024-03-21T08:30:06.653383+0000","last_became_peered":"2024-03-21T08:30:06.653383+0000","last_unstale":"2024-03-21T08:30:06.661857+0000","last_undegraded":"2024-03-21T08:30:06.661857+0000","last_fullsized":"2024-03-21T08:30:06.661857+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:43.836881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317829+0000","last_change":"2024-03-21T08:28:36.670176+0000","last_active":"2024-03-21T08:30:35.317829+0000","last_peered":"2024-03-21T08:30:35.317829+0000","last_clean":"2024-03-21T08:30:35.317829+0000","last_became_active":"2024-03-21T08:28:36.669477+0000","last_became_peered":"2024-03-21T08:28:36.669477+0000","last_unstale":"2024-03-21T08:30:35.317829+0000","last_undegraded":"2024-03-21T08:30:35.317829+0000","last_fullsized":"2024-03-21T08:30:35.317829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:10:21.289484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654777+0000","last_change":"2024-03-21T08:29:06.784996+0000","last_active":"2024-03-21T08:30:06.654777+0000","last_peered":"2024-03-21T08:30:06.654777+0000","last_clean":"2024-03-21T08:30:06.654777+0000","last_became_active":"2024-03-21T08:29:06.783403+0000","last_became_peered":"2024-03-21T08:29:06.783403+0000","last_unstale":"2024-03-21T08:30:06.654777+0000","last_undegraded":"2024-03-21T08:30:06.654777+0000","last_fullsized":"2024-03-21T08:30:06.654777+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:18:59.912956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641118+0000","last_change":"2024-03-21T08:28:35.667502+0000","last_active":"2024-03-21T08:30:06.641118+0000","last_peered":"2024-03-21T08:30:06.641118+0000","last_clean":"2024-03-21T08:30:06.641118+0000","last_became_active":"2024-03-21T08:28:35.666667+0000","last_became_peered":"2024-03-21T08:28:35.666667+0000","last_unstale":"2024-03-21T08:30:06.641118+0000","last_undegraded":"2024-03-21T08:30:06.641118+0000","last_fullsized":"2024-03-21T08:30:06.641118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:42.196361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.318414+0000","last_change":"2024-03-21T08:28:36.670855+0000","last_active":"2024-03-21T08:30:35.318414+0000","last_peered":"2024-03-21T08:30:35.318414+0000","last_clean":"2024-03-21T08:30:35.318414+0000","last_became_active":"2024-03-21T08:28:36.666012+0000","last_became_peered":"2024-03-21T08:28:36.666012+0000","last_unstale":"2024-03-21T08:30:35.318414+0000","last_undegraded":"2024-03-21T08:30:35.318414+0000","last_fullsized":"2024-03-21T08:30:35.318414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:19.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652911+0000","last_change":"2024-03-21T08:28:40.700037+0000","last_active":"2024-03-21T08:30:06.652911+0000","last_peered":"2024-03-21T08:30:06.652911+0000","last_clean":"2024-03-21T08:30:06.652911+0000","last_became_active":"2024-03-21T08:28:40.698127+0000","last_became_peered":"2024-03-21T08:28:40.698127+0000","last_unstale":"2024-03-21T08:30:06.652911+0000","last_undegraded":"2024-03-21T08:30:06.652911+0000","last_fullsized":"2024-03-21T08:30:06.652911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:48:57.135690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664533+0000","last_change":"2024-03-21T08:28:35.677071+0000","last_active":"2024-03-21T08:30:06.664533+0000","last_peered":"2024-03-21T08:30:06.664533+0000","last_clean":"2024-03-21T08:30:06.664533+0000","last_became_active":"2024-03-21T08:28:35.676652+0000","last_became_peered":"2024-03-21T08:28:35.676652+0000","last_unstale":"2024-03-21T08:30:06.664533+0000","last_undegraded":"2024-03-21T08:30:06.664533+0000","last_fullsized":"2024-03-21T08:30:06.664533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:24:54.097952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.321747+0000","last_change":"2024-03-21T08:28:36.677981+0000","last_active":"2024-03-21T08:30:35.321747+0000","last_peered":"2024-03-21T08:30:35.321747+0000","last_clean":"2024-03-21T08:30:35.321747+0000","last_became_active":"2024-03-21T08:28:36.677512+0000","last_became_peered":"2024-03-21T08:28:36.677512+0000","last_unstale":"2024-03-21T08:30:35.321747+0000","last_undegraded":"2024-03-21T08:30:35.321747+0000","last_fullsized":"2024-03-21T08:30:35.321747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:50.742928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646728+0000","last_change":"2024-03-21T08:28:40.692672+0000","last_active":"2024-03-21T08:30:06.646728+0000","last_peered":"2024-03-21T08:30:06.646728+0000","last_clean":"2024-03-21T08:30:06.646728+0000","last_became_active":"2024-03-21T08:28:40.688940+0000","last_became_peered":"2024-03-21T08:28:40.688940+0000","last_unstale":"2024-03-21T08:30:06.646728+0000","last_undegraded":"2024-03-21T08:30:06.646728+0000","last_fullsized":"2024-03-21T08:30:06.646728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:39.569709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:32.739007+0000","last_change":"2024-03-21T08:28:35.681970+0000","last_active":"2024-03-21T08:31:32.739007+0000","last_peered":"2024-03-21T08:31:32.739007+0000","last_clean":"2024-03-21T08:31:32.739007+0000","last_became_active":"2024-03-21T08:28:35.681252+0000","last_became_peered":"2024-03-21T08:28:35.681252+0000","last_unstale":"2024-03-21T08:31:32.739007+0000","last_undegraded":"2024-03-21T08:31:32.739007+0000","last_fullsized":"2024-03-21T08:31:32.739007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:46.345359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.334525+0000","last_change":"2024-03-21T08:28:36.675419+0000","last_active":"2024-03-21T08:30:35.334525+0000","last_peered":"2024-03-21T08:30:35.334525+0000","last_clean":"2024-03-21T08:30:35.334525+0000","last_became_active":"2024-03-21T08:28:36.674039+0000","last_became_peered":"2024-03-21T08:28:36.674039+0000","last_unstale":"2024-03-21T08:30:35.334525+0000","last_undegraded":"2024-03-21T08:30:35.334525+0000","last_fullsized":"2024-03-21T08:30:35.334525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:09.667634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648585+0000","last_change":"2024-03-21T08:28:40.690076+0000","last_active":"2024-03-21T08:30:06.648585+0000","last_peered":"2024-03-21T08:30:06.648585+0000","last_clean":"2024-03-21T08:30:06.648585+0000","last_became_active":"2024-03-21T08:28:40.683817+0000","last_became_peered":"2024-03-21T08:28:40.683817+0000","last_unstale":"2024-03-21T08:30:06.648585+0000","last_undegraded":"2024-03-21T08:30:06.648585+0000","last_fullsized":"2024-03-21T08:30:06.648585+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:33:00.948879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654217+0000","last_change":"2024-03-21T08:28:35.690693+0000","last_active":"2024-03-21T08:30:06.654217+0000","last_peered":"2024-03-21T08:30:06.654217+0000","last_clean":"2024-03-21T08:30:06.654217+0000","last_became_active":"2024-03-21T08:28:35.690250+0000","last_became_peered":"2024-03-21T08:28:35.690250+0000","last_unstale":"2024-03-21T08:30:06.654217+0000","last_undegraded":"2024-03-21T08:30:06.654217+0000","last_fullsized":"2024-03-21T08:30:06.654217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:44:21.307772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330814+0000","last_change":"2024-03-21T08:28:36.661872+0000","last_active":"2024-03-21T08:30:35.330814+0000","last_peered":"2024-03-21T08:30:35.330814+0000","last_clean":"2024-03-21T08:30:35.330814+0000","last_became_active":"2024-03-21T08:28:36.661483+0000","last_became_peered":"2024-03-21T08:28:36.661483+0000","last_unstale":"2024-03-21T08:30:35.330814+0000","last_undegraded":"2024-03-21T08:30:35.330814+0000","last_fullsized":"2024-03-21T08:30:35.330814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:03:35.630141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665016+0000","last_change":"2024-03-21T08:28:40.682753+0000","last_active":"2024-03-21T08:30:06.665016+0000","last_peered":"2024-03-21T08:30:06.665016+0000","last_clean":"2024-03-21T08:30:06.665016+0000","last_became_active":"2024-03-21T08:28:40.682278+0000","last_became_peered":"2024-03-21T08:28:40.682278+0000","last_unstale":"2024-03-21T08:30:06.665016+0000","last_undegraded":"2024-03-21T08:30:06.665016+0000","last_fullsized":"2024-03-21T08:30:06.665016+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:20.280442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666308+0000","last_change":"2024-03-21T08:28:35.672524+0000","last_active":"2024-03-21T08:30:06.666308+0000","last_peered":"2024-03-21T08:30:06.666308+0000","last_clean":"2024-03-21T08:30:06.666308+0000","last_became_active":"2024-03-21T08:28:35.672051+0000","last_became_peered":"2024-03-21T08:28:35.672051+0000","last_unstale":"2024-03-21T08:30:06.666308+0000","last_undegraded":"2024-03-21T08:30:06.666308+0000","last_fullsized":"2024-03-21T08:30:06.666308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:53.336320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329907+0000","last_change":"2024-03-21T08:29:06.532172+0000","last_active":"2024-03-21T08:30:35.329907+0000","last_peered":"2024-03-21T08:30:35.329907+0000","last_clean":"2024-03-21T08:30:35.329907+0000","last_became_active":"2024-03-21T08:29:06.531439+0000","last_became_peered":"2024-03-21T08:29:06.531439+0000","last_unstale":"2024-03-21T08:30:35.329907+0000","last_undegraded":"2024-03-21T08:30:35.329907+0000","last_fullsized":"2024-03-21T08:30:35.329907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:27.222359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648624+0000","last_change":"2024-03-21T08:28:40.682294+0000","last_active":"2024-03-21T08:30:06.648624+0000","last_peered":"2024-03-21T08:30:06.648624+0000","last_clean":"2024-03-21T08:30:06.648624+0000","last_became_active":"2024-03-21T08:28:40.681855+0000","last_became_peered":"2024-03-21T08:28:40.681855+0000","last_unstale":"2024-03-21T08:30:06.648624+0000","last_undegraded":"2024-03-21T08:30:06.648624+0000","last_fullsized":"2024-03-21T08:30:06.648624+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:49.756322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'40","reported_seq":374,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:33.736869+0000","last_change":"2024-03-21T08:28:35.682947+0000","last_active":"2024-03-21T08:31:33.736869+0000","last_peered":"2024-03-21T08:31:33.736869+0000","last_clean":"2024-03-21T08:31:33.736869+0000","last_became_active":"2024-03-21T08:28:35.682395+0000","last_became_peered":"2024-03-21T08:28:35.682395+0000","last_unstale":"2024-03-21T08:31:33.736869+0000","last_undegraded":"2024-03-21T08:31:33.736869+0000","last_fullsized":"2024-03-21T08:31:33.736869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:07:48.072988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67389,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":218,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.310604+0000","last_change":"2024-03-21T08:28:36.651140+0000","last_active":"2024-03-21T08:30:35.310604+0000","last_peered":"2024-03-21T08:30:35.310604+0000","last_clean":"2024-03-21T08:30:35.310604+0000","last_became_active":"2024-03-21T08:28:36.650551+0000","last_became_peered":"2024-03-21T08:28:36.650551+0000","last_unstale":"2024-03-21T08:30:35.310604+0000","last_undegraded":"2024-03-21T08:30:35.310604+0000","last_fullsized":"2024-03-21T08:30:35.310604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:53.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.667084+0000","last_change":"2024-03-21T08:28:40.691814+0000","last_active":"2024-03-21T08:30:06.667084+0000","last_peered":"2024-03-21T08:30:06.667084+0000","last_clean":"2024-03-21T08:30:06.667084+0000","last_became_active":"2024-03-21T08:28:40.691372+0000","last_became_peered":"2024-03-21T08:28:40.691372+0000","last_unstale":"2024-03-21T08:30:06.667084+0000","last_undegraded":"2024-03-21T08:30:06.667084+0000","last_fullsized":"2024-03-21T08:30:06.667084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:21:42.612037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645742+0000","last_change":"2024-03-21T08:28:35.687178+0000","last_active":"2024-03-21T08:30:06.645742+0000","last_peered":"2024-03-21T08:30:06.645742+0000","last_clean":"2024-03-21T08:30:06.645742+0000","last_became_active":"2024-03-21T08:28:35.686771+0000","last_became_peered":"2024-03-21T08:28:35.686771+0000","last_unstale":"2024-03-21T08:30:06.645742+0000","last_undegraded":"2024-03-21T08:30:06.645742+0000","last_fullsized":"2024-03-21T08:30:06.645742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:54.665866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317525+0000","last_change":"2024-03-21T08:28:36.662952+0000","last_active":"2024-03-21T08:30:35.317525+0000","last_peered":"2024-03-21T08:30:35.317525+0000","last_clean":"2024-03-21T08:30:35.317525+0000","last_became_active":"2024-03-21T08:28:36.662468+0000","last_became_peered":"2024-03-21T08:28:36.662468+0000","last_unstale":"2024-03-21T08:30:35.317525+0000","last_undegraded":"2024-03-21T08:30:35.317525+0000","last_fullsized":"2024-03-21T08:30:35.317525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:40.486998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656409+0000","last_change":"2024-03-21T08:28:40.693999+0000","last_active":"2024-03-21T08:30:06.656409+0000","last_peered":"2024-03-21T08:30:06.656409+0000","last_clean":"2024-03-21T08:30:06.656409+0000","last_became_active":"2024-03-21T08:28:40.693507+0000","last_became_peered":"2024-03-21T08:28:40.693507+0000","last_unstale":"2024-03-21T08:30:06.656409+0000","last_undegraded":"2024-03-21T08:30:06.656409+0000","last_fullsized":"2024-03-21T08:30:06.656409+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:14.159379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.668728+0000","last_change":"2024-03-21T08:28:35.677536+0000","last_active":"2024-03-21T08:30:06.668728+0000","last_peered":"2024-03-21T08:30:06.668728+0000","last_clean":"2024-03-21T08:30:06.668728+0000","last_became_active":"2024-03-21T08:28:35.677100+0000","last_became_peered":"2024-03-21T08:28:35.677100+0000","last_unstale":"2024-03-21T08:30:06.668728+0000","last_undegraded":"2024-03-21T08:30:06.668728+0000","last_fullsized":"2024-03-21T08:30:06.668728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:52:47.275869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.322854+0000","last_change":"2024-03-21T08:28:36.667190+0000","last_active":"2024-03-21T08:30:35.322854+0000","last_peered":"2024-03-21T08:30:35.322854+0000","last_clean":"2024-03-21T08:30:35.322854+0000","last_became_active":"2024-03-21T08:28:36.665336+0000","last_became_peered":"2024-03-21T08:28:36.665336+0000","last_unstale":"2024-03-21T08:30:35.322854+0000","last_undegraded":"2024-03-21T08:30:35.322854+0000","last_fullsized":"2024-03-21T08:30:35.322854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:21.888426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649761+0000","last_change":"2024-03-21T08:28:40.696582+0000","last_active":"2024-03-21T08:30:06.649761+0000","last_peered":"2024-03-21T08:30:06.649761+0000","last_clean":"2024-03-21T08:30:06.649761+0000","last_became_active":"2024-03-21T08:28:40.689583+0000","last_became_peered":"2024-03-21T08:28:40.689583+0000","last_unstale":"2024-03-21T08:30:06.649761+0000","last_undegraded":"2024-03-21T08:30:06.649761+0000","last_fullsized":"2024-03-21T08:30:06.649761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:57.329533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'4","reported_seq":185,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:33.736977+0000","last_change":"2024-03-21T08:28:35.675560+0000","last_active":"2024-03-21T08:31:33.736977+0000","last_peered":"2024-03-21T08:31:33.736977+0000","last_clean":"2024-03-21T08:31:33.736977+0000","last_became_active":"2024-03-21T08:28:35.673496+0000","last_became_peered":"2024-03-21T08:28:35.673496+0000","last_unstale":"2024-03-21T08:31:33.736977+0000","last_undegraded":"2024-03-21T08:31:33.736977+0000","last_fullsized":"2024-03-21T08:31:33.736977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:41:27.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.322318+0000","last_change":"2024-03-21T08:28:36.660869+0000","last_active":"2024-03-21T08:30:35.322318+0000","last_peered":"2024-03-21T08:30:35.322318+0000","last_clean":"2024-03-21T08:30:35.322318+0000","last_became_active":"2024-03-21T08:28:36.659291+0000","last_became_peered":"2024-03-21T08:28:36.659291+0000","last_unstale":"2024-03-21T08:30:35.322318+0000","last_undegraded":"2024-03-21T08:30:35.322318+0000","last_fullsized":"2024-03-21T08:30:35.322318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:06.534214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.643153+0000","last_change":"2024-03-21T08:28:40.701351+0000","last_active":"2024-03-21T08:30:06.643153+0000","last_peered":"2024-03-21T08:30:06.643153+0000","last_clean":"2024-03-21T08:30:06.643153+0000","last_became_active":"2024-03-21T08:28:40.700989+0000","last_became_peered":"2024-03-21T08:28:40.700989+0000","last_unstale":"2024-03-21T08:30:06.643153+0000","last_undegraded":"2024-03-21T08:30:06.643153+0000","last_fullsized":"2024-03-21T08:30:06.643153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:35.697013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650538+0000","last_change":"2024-03-21T08:28:35.685024+0000","last_active":"2024-03-21T08:30:06.650538+0000","last_peered":"2024-03-21T08:30:06.650538+0000","last_clean":"2024-03-21T08:30:06.650538+0000","last_became_active":"2024-03-21T08:28:35.684597+0000","last_became_peered":"2024-03-21T08:28:35.684597+0000","last_unstale":"2024-03-21T08:30:06.650538+0000","last_undegraded":"2024-03-21T08:30:06.650538+0000","last_fullsized":"2024-03-21T08:30:06.650538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:09:46.262782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332203+0000","last_change":"2024-03-21T08:28:36.664688+0000","last_active":"2024-03-21T08:30:35.332203+0000","last_peered":"2024-03-21T08:30:35.332203+0000","last_clean":"2024-03-21T08:30:35.332203+0000","last_became_active":"2024-03-21T08:28:36.663990+0000","last_became_peered":"2024-03-21T08:28:36.663990+0000","last_unstale":"2024-03-21T08:30:35.332203+0000","last_undegraded":"2024-03-21T08:30:35.332203+0000","last_fullsized":"2024-03-21T08:30:35.332203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:39.656290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664426+0000","last_change":"2024-03-21T08:28:40.695269+0000","last_active":"2024-03-21T08:30:06.664426+0000","last_peered":"2024-03-21T08:30:06.664426+0000","last_clean":"2024-03-21T08:30:06.664426+0000","last_became_active":"2024-03-21T08:28:40.694820+0000","last_became_peered":"2024-03-21T08:28:40.694820+0000","last_unstale":"2024-03-21T08:30:06.664426+0000","last_undegraded":"2024-03-21T08:30:06.664426+0000","last_fullsized":"2024-03-21T08:30:06.664426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:47:25.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.673554+0000","last_change":"2024-03-21T08:30:06.672662+0000","last_active":"2024-03-21T08:30:06.673554+0000","last_peered":"2024-03-21T08:30:06.673554+0000","last_clean":"2024-03-21T08:30:06.673554+0000","last_became_active":"2024-03-21T08:30:06.660161+0000","last_became_peered":"2024-03-21T08:30:06.660161+0000","last_unstale":"2024-03-21T08:30:06.673554+0000","last_undegraded":"2024-03-21T08:30:06.673554+0000","last_fullsized":"2024-03-21T08:30:06.673554+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:41.163709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,4],"acting":[8,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315069+0000","last_change":"2024-03-21T08:28:36.667605+0000","last_active":"2024-03-21T08:30:35.315069+0000","last_peered":"2024-03-21T08:30:35.315069+0000","last_clean":"2024-03-21T08:30:35.315069+0000","last_became_active":"2024-03-21T08:28:36.667300+0000","last_became_peered":"2024-03-21T08:28:36.667300+0000","last_unstale":"2024-03-21T08:30:35.315069+0000","last_undegraded":"2024-03-21T08:30:35.315069+0000","last_fullsized":"2024-03-21T08:30:35.315069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:49.268935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652659+0000","last_change":"2024-03-21T08:28:40.884964+0000","last_active":"2024-03-21T08:30:06.652659+0000","last_peered":"2024-03-21T08:30:06.652659+0000","last_clean":"2024-03-21T08:30:06.652659+0000","last_became_active":"2024-03-21T08:28:40.883445+0000","last_became_peered":"2024-03-21T08:28:40.883445+0000","last_unstale":"2024-03-21T08:30:06.652659+0000","last_undegraded":"2024-03-21T08:30:06.652659+0000","last_fullsized":"2024-03-21T08:30:06.652659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:36:54.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647966+0000","last_change":"2024-03-21T08:28:35.682581+0000","last_active":"2024-03-21T08:30:06.647966+0000","last_peered":"2024-03-21T08:30:06.647966+0000","last_clean":"2024-03-21T08:30:06.647966+0000","last_became_active":"2024-03-21T08:28:35.682150+0000","last_became_peered":"2024-03-21T08:28:35.682150+0000","last_unstale":"2024-03-21T08:30:06.647966+0000","last_undegraded":"2024-03-21T08:30:06.647966+0000","last_fullsized":"2024-03-21T08:30:06.647966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:08.729710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316730+0000","last_change":"2024-03-21T08:28:36.666632+0000","last_active":"2024-03-21T08:30:35.316730+0000","last_peered":"2024-03-21T08:30:35.316730+0000","last_clean":"2024-03-21T08:30:35.316730+0000","last_became_active":"2024-03-21T08:28:36.666305+0000","last_became_peered":"2024-03-21T08:28:36.666305+0000","last_unstale":"2024-03-21T08:30:35.316730+0000","last_undegraded":"2024-03-21T08:30:35.316730+0000","last_fullsized":"2024-03-21T08:30:35.316730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:15.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642891+0000","last_change":"2024-03-21T08:28:40.700382+0000","last_active":"2024-03-21T08:30:06.642891+0000","last_peered":"2024-03-21T08:30:06.642891+0000","last_clean":"2024-03-21T08:30:06.642891+0000","last_became_active":"2024-03-21T08:28:40.699762+0000","last_became_peered":"2024-03-21T08:28:40.699762+0000","last_unstale":"2024-03-21T08:30:06.642891+0000","last_undegraded":"2024-03-21T08:30:06.642891+0000","last_fullsized":"2024-03-21T08:30:06.642891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:53.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657107+0000","last_change":"2024-03-21T08:28:35.680310+0000","last_active":"2024-03-21T08:30:06.657107+0000","last_peered":"2024-03-21T08:30:06.657107+0000","last_clean":"2024-03-21T08:30:06.657107+0000","last_became_active":"2024-03-21T08:28:35.678634+0000","last_became_peered":"2024-03-21T08:28:35.678634+0000","last_unstale":"2024-03-21T08:30:06.657107+0000","last_undegraded":"2024-03-21T08:30:06.657107+0000","last_fullsized":"2024-03-21T08:30:06.657107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:56:42.872521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319411+0000","last_change":"2024-03-21T08:28:36.669527+0000","last_active":"2024-03-21T08:30:35.319411+0000","last_peered":"2024-03-21T08:30:35.319411+0000","last_clean":"2024-03-21T08:30:35.319411+0000","last_became_active":"2024-03-21T08:28:36.669196+0000","last_became_peered":"2024-03-21T08:28:36.669196+0000","last_unstale":"2024-03-21T08:30:35.319411+0000","last_undegraded":"2024-03-21T08:30:35.319411+0000","last_fullsized":"2024-03-21T08:30:35.319411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:07:34.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644070+0000","last_change":"2024-03-21T08:28:40.882949+0000","last_active":"2024-03-21T08:30:06.644070+0000","last_peered":"2024-03-21T08:30:06.644070+0000","last_clean":"2024-03-21T08:30:06.644070+0000","last_became_active":"2024-03-21T08:28:40.882432+0000","last_became_peered":"2024-03-21T08:28:40.882432+0000","last_unstale":"2024-03-21T08:30:06.644070+0000","last_undegraded":"2024-03-21T08:30:06.644070+0000","last_fullsized":"2024-03-21T08:30:06.644070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:51.171365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.321838+0000","last_change":"2024-03-21T08:28:36.665164+0000","last_active":"2024-03-21T08:30:35.321838+0000","last_peered":"2024-03-21T08:30:35.321838+0000","last_clean":"2024-03-21T08:30:35.321838+0000","last_became_active":"2024-03-21T08:28:36.664794+0000","last_became_peered":"2024-03-21T08:28:36.664794+0000","last_unstale":"2024-03-21T08:30:35.321838+0000","last_undegraded":"2024-03-21T08:30:35.321838+0000","last_fullsized":"2024-03-21T08:30:35.321838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:25:39.355633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'5","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:32.733229+0000","last_change":"2024-03-21T08:28:35.671754+0000","last_active":"2024-03-21T08:31:32.733229+0000","last_peered":"2024-03-21T08:31:32.733229+0000","last_clean":"2024-03-21T08:31:32.733229+0000","last_became_active":"2024-03-21T08:28:35.671363+0000","last_became_peered":"2024-03-21T08:28:35.671363+0000","last_unstale":"2024-03-21T08:31:32.733229+0000","last_undegraded":"2024-03-21T08:31:32.733229+0000","last_fullsized":"2024-03-21T08:31:32.733229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:22:11.147325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657862+0000","last_change":"2024-03-21T08:28:40.689038+0000","last_active":"2024-03-21T08:30:06.657862+0000","last_peered":"2024-03-21T08:30:06.657862+0000","last_clean":"2024-03-21T08:30:06.657862+0000","last_became_active":"2024-03-21T08:28:40.688538+0000","last_became_peered":"2024-03-21T08:28:40.688538+0000","last_unstale":"2024-03-21T08:30:06.657862+0000","last_undegraded":"2024-03-21T08:30:06.657862+0000","last_fullsized":"2024-03-21T08:30:06.657862+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:14.833594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331670+0000","last_change":"2024-03-21T08:28:36.667725+0000","last_active":"2024-03-21T08:30:35.331670+0000","last_peered":"2024-03-21T08:30:35.331670+0000","last_clean":"2024-03-21T08:30:35.331670+0000","last_became_active":"2024-03-21T08:28:36.667303+0000","last_became_peered":"2024-03-21T08:28:36.667303+0000","last_unstale":"2024-03-21T08:30:35.331670+0000","last_undegraded":"2024-03-21T08:30:35.331670+0000","last_fullsized":"2024-03-21T08:30:35.331670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:32:18.990943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664955+0000","last_change":"2024-03-21T08:28:35.673497+0000","last_active":"2024-03-21T08:30:06.664955+0000","last_peered":"2024-03-21T08:30:06.664955+0000","last_clean":"2024-03-21T08:30:06.664955+0000","last_became_active":"2024-03-21T08:28:35.673045+0000","last_became_peered":"2024-03-21T08:28:35.673045+0000","last_unstale":"2024-03-21T08:30:06.664955+0000","last_undegraded":"2024-03-21T08:30:06.664955+0000","last_fullsized":"2024-03-21T08:30:06.664955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:23.972672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":370,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653387+0000","last_change":"2024-03-21T08:26:58.139094+0000","last_active":"2024-03-21T08:30:06.653387+0000","last_peered":"2024-03-21T08:30:06.653387+0000","last_clean":"2024-03-21T08:30:06.653387+0000","last_became_active":"2024-03-21T08:26:58.137648+0000","last_became_peered":"2024-03-21T08:26:58.137648+0000","last_unstale":"2024-03-21T08:30:06.653387+0000","last_undegraded":"2024-03-21T08:30:06.653387+0000","last_fullsized":"2024-03-21T08:30:06.653387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_clean_scrub_stamp":"2024-03-21T08:22:17.977314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:57.950111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665774+0000","last_change":"2024-03-21T08:28:40.887865+0000","last_active":"2024-03-21T08:30:06.665774+0000","last_peered":"2024-03-21T08:30:06.665774+0000","last_clean":"2024-03-21T08:30:06.665774+0000","last_became_active":"2024-03-21T08:28:40.885538+0000","last_became_peered":"2024-03-21T08:28:40.885538+0000","last_unstale":"2024-03-21T08:30:06.665774+0000","last_undegraded":"2024-03-21T08:30:06.665774+0000","last_fullsized":"2024-03-21T08:30:06.665774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:26.532005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.309634+0000","last_change":"2024-03-21T08:28:36.668600+0000","last_active":"2024-03-21T08:30:35.309634+0000","last_peered":"2024-03-21T08:30:35.309634+0000","last_clean":"2024-03-21T08:30:35.309634+0000","last_became_active":"2024-03-21T08:28:36.668188+0000","last_became_peered":"2024-03-21T08:28:36.668188+0000","last_unstale":"2024-03-21T08:30:35.309634+0000","last_undegraded":"2024-03-21T08:30:35.309634+0000","last_fullsized":"2024-03-21T08:30:35.309634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:13.536003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.674753+0000","last_change":"2024-03-21T08:30:06.673425+0000","last_active":"2024-03-21T08:30:06.674753+0000","last_peered":"2024-03-21T08:30:06.674753+0000","last_clean":"2024-03-21T08:30:06.674753+0000","last_became_active":"2024-03-21T08:30:06.666858+0000","last_became_peered":"2024-03-21T08:30:06.666858+0000","last_unstale":"2024-03-21T08:30:06.674753+0000","last_undegraded":"2024-03-21T08:30:06.674753+0000","last_fullsized":"2024-03-21T08:30:06.674753+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:37.159978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655579+0000","last_change":"2024-03-21T08:29:07.032622+0000","last_active":"2024-03-21T08:30:06.655579+0000","last_peered":"2024-03-21T08:30:06.655579+0000","last_clean":"2024-03-21T08:30:06.655579+0000","last_became_active":"2024-03-21T08:29:07.030884+0000","last_became_peered":"2024-03-21T08:29:07.030884+0000","last_unstale":"2024-03-21T08:30:06.655579+0000","last_undegraded":"2024-03-21T08:30:06.655579+0000","last_fullsized":"2024-03-21T08:30:06.655579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:47.615776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654098+0000","last_change":"2024-03-21T08:28:35.672948+0000","last_active":"2024-03-21T08:30:06.654098+0000","last_peered":"2024-03-21T08:30:06.654098+0000","last_clean":"2024-03-21T08:30:06.654098+0000","last_became_active":"2024-03-21T08:28:35.671576+0000","last_became_peered":"2024-03-21T08:28:35.671576+0000","last_unstale":"2024-03-21T08:30:06.654098+0000","last_undegraded":"2024-03-21T08:30:06.654098+0000","last_fullsized":"2024-03-21T08:30:06.654098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:36:25.620392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.308174+0000","last_change":"2024-03-21T08:28:36.671173+0000","last_active":"2024-03-21T08:30:35.308174+0000","last_peered":"2024-03-21T08:30:35.308174+0000","last_clean":"2024-03-21T08:30:35.308174+0000","last_became_active":"2024-03-21T08:28:36.666264+0000","last_became_peered":"2024-03-21T08:28:36.666264+0000","last_unstale":"2024-03-21T08:30:35.308174+0000","last_undegraded":"2024-03-21T08:30:35.308174+0000","last_fullsized":"2024-03-21T08:30:35.308174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:35.081487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646351+0000","last_change":"2024-03-21T08:28:40.883801+0000","last_active":"2024-03-21T08:30:06.646351+0000","last_peered":"2024-03-21T08:30:06.646351+0000","last_clean":"2024-03-21T08:30:06.646351+0000","last_became_active":"2024-03-21T08:28:40.882556+0000","last_became_peered":"2024-03-21T08:28:40.882556+0000","last_unstale":"2024-03-21T08:30:06.646351+0000","last_undegraded":"2024-03-21T08:30:06.646351+0000","last_fullsized":"2024-03-21T08:30:06.646351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:24.211625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'10","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:49.542985+0000","last_change":"2024-03-21T08:28:35.668657+0000","last_active":"2024-03-21T08:30:49.542985+0000","last_peered":"2024-03-21T08:30:49.542985+0000","last_clean":"2024-03-21T08:30:49.542985+0000","last_became_active":"2024-03-21T08:28:35.668284+0000","last_became_peered":"2024-03-21T08:28:35.668284+0000","last_unstale":"2024-03-21T08:30:49.542985+0000","last_undegraded":"2024-03-21T08:30:49.542985+0000","last_fullsized":"2024-03-21T08:30:49.542985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:26:07.791388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316596+0000","last_change":"2024-03-21T08:28:36.652383+0000","last_active":"2024-03-21T08:30:35.316596+0000","last_peered":"2024-03-21T08:30:35.316596+0000","last_clean":"2024-03-21T08:30:35.316596+0000","last_became_active":"2024-03-21T08:28:36.652046+0000","last_became_peered":"2024-03-21T08:28:36.652046+0000","last_unstale":"2024-03-21T08:30:35.316596+0000","last_undegraded":"2024-03-21T08:30:35.316596+0000","last_fullsized":"2024-03-21T08:30:35.316596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:11:08.699391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644497+0000","last_change":"2024-03-21T08:28:40.686122+0000","last_active":"2024-03-21T08:30:06.644497+0000","last_peered":"2024-03-21T08:30:06.644497+0000","last_clean":"2024-03-21T08:30:06.644497+0000","last_became_active":"2024-03-21T08:28:40.685736+0000","last_became_peered":"2024-03-21T08:28:40.685736+0000","last_unstale":"2024-03-21T08:30:06.644497+0000","last_undegraded":"2024-03-21T08:30:06.644497+0000","last_fullsized":"2024-03-21T08:30:06.644497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:52.941896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665257+0000","last_change":"2024-03-21T08:28:35.675197+0000","last_active":"2024-03-21T08:30:06.665257+0000","last_peered":"2024-03-21T08:30:06.665257+0000","last_clean":"2024-03-21T08:30:06.665257+0000","last_became_active":"2024-03-21T08:28:35.674688+0000","last_became_peered":"2024-03-21T08:28:35.674688+0000","last_unstale":"2024-03-21T08:30:06.665257+0000","last_undegraded":"2024-03-21T08:30:06.665257+0000","last_fullsized":"2024-03-21T08:30:06.665257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:01:11.489294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328753+0000","last_change":"2024-03-21T08:28:36.668160+0000","last_active":"2024-03-21T08:30:35.328753+0000","last_peered":"2024-03-21T08:30:35.328753+0000","last_clean":"2024-03-21T08:30:35.328753+0000","last_became_active":"2024-03-21T08:28:36.665152+0000","last_became_peered":"2024-03-21T08:28:36.665152+0000","last_unstale":"2024-03-21T08:30:35.328753+0000","last_undegraded":"2024-03-21T08:30:35.328753+0000","last_fullsized":"2024-03-21T08:30:35.328753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:43:41.803822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663608+0000","last_change":"2024-03-21T08:28:40.884365+0000","last_active":"2024-03-21T08:30:06.663608+0000","last_peered":"2024-03-21T08:30:06.663608+0000","last_clean":"2024-03-21T08:30:06.663608+0000","last_became_active":"2024-03-21T08:28:40.883654+0000","last_became_peered":"2024-03-21T08:28:40.883654+0000","last_unstale":"2024-03-21T08:30:06.663608+0000","last_undegraded":"2024-03-21T08:30:06.663608+0000","last_fullsized":"2024-03-21T08:30:06.663608+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:46:43.888121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672790+0000","last_change":"2024-03-21T08:30:06.672111+0000","last_active":"2024-03-21T08:30:06.672790+0000","last_peered":"2024-03-21T08:30:06.672790+0000","last_clean":"2024-03-21T08:30:06.672790+0000","last_became_active":"2024-03-21T08:30:06.665627+0000","last_became_peered":"2024-03-21T08:30:06.665627+0000","last_unstale":"2024-03-21T08:30:06.672790+0000","last_undegraded":"2024-03-21T08:30:06.672790+0000","last_fullsized":"2024-03-21T08:30:06.672790+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:05.512276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.307537+0000","last_change":"2024-03-21T08:28:36.653797+0000","last_active":"2024-03-21T08:30:35.307537+0000","last_peered":"2024-03-21T08:30:35.307537+0000","last_clean":"2024-03-21T08:30:35.307537+0000","last_became_active":"2024-03-21T08:28:36.651872+0000","last_became_peered":"2024-03-21T08:28:36.651872+0000","last_unstale":"2024-03-21T08:30:35.307537+0000","last_undegraded":"2024-03-21T08:30:35.307537+0000","last_fullsized":"2024-03-21T08:30:35.307537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:09.467573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649747+0000","last_change":"2024-03-21T08:28:40.692996+0000","last_active":"2024-03-21T08:30:06.649747+0000","last_peered":"2024-03-21T08:30:06.649747+0000","last_clean":"2024-03-21T08:30:06.649747+0000","last_became_active":"2024-03-21T08:28:40.687598+0000","last_became_peered":"2024-03-21T08:28:40.687598+0000","last_unstale":"2024-03-21T08:30:06.649747+0000","last_undegraded":"2024-03-21T08:30:06.649747+0000","last_fullsized":"2024-03-21T08:30:06.649747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:09.557272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645955+0000","last_change":"2024-03-21T08:28:35.672567+0000","last_active":"2024-03-21T08:30:06.645955+0000","last_peered":"2024-03-21T08:30:06.645955+0000","last_clean":"2024-03-21T08:30:06.645955+0000","last_became_active":"2024-03-21T08:28:35.671849+0000","last_became_peered":"2024-03-21T08:28:35.671849+0000","last_unstale":"2024-03-21T08:30:06.645955+0000","last_undegraded":"2024-03-21T08:30:06.645955+0000","last_fullsized":"2024-03-21T08:30:06.645955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:13.050227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.320698+0000","last_change":"2024-03-21T08:28:36.664110+0000","last_active":"2024-03-21T08:30:35.320698+0000","last_peered":"2024-03-21T08:30:35.320698+0000","last_clean":"2024-03-21T08:30:35.320698+0000","last_became_active":"2024-03-21T08:28:36.663297+0000","last_became_peered":"2024-03-21T08:28:36.663297+0000","last_unstale":"2024-03-21T08:30:35.320698+0000","last_undegraded":"2024-03-21T08:30:35.320698+0000","last_fullsized":"2024-03-21T08:30:35.320698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:55.450399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663838+0000","last_change":"2024-03-21T08:28:40.682371+0000","last_active":"2024-03-21T08:30:06.663838+0000","last_peered":"2024-03-21T08:30:06.663838+0000","last_clean":"2024-03-21T08:30:06.663838+0000","last_became_active":"2024-03-21T08:28:40.681786+0000","last_became_peered":"2024-03-21T08:28:40.681786+0000","last_unstale":"2024-03-21T08:30:06.663838+0000","last_undegraded":"2024-03-21T08:30:06.663838+0000","last_fullsized":"2024-03-21T08:30:06.663838+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:15:38.130807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"96'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:46.232715+0000","last_change":"2024-03-21T08:28:35.685312+0000","last_active":"2024-03-21T08:30:46.232715+0000","last_peered":"2024-03-21T08:30:46.232715+0000","last_clean":"2024-03-21T08:30:46.232715+0000","last_became_active":"2024-03-21T08:28:35.684327+0000","last_became_peered":"2024-03-21T08:28:35.684327+0000","last_unstale":"2024-03-21T08:30:46.232715+0000","last_undegraded":"2024-03-21T08:30:46.232715+0000","last_fullsized":"2024-03-21T08:30:46.232715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:12.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330747+0000","last_change":"2024-03-21T08:28:36.671186+0000","last_active":"2024-03-21T08:30:35.330747+0000","last_peered":"2024-03-21T08:30:35.330747+0000","last_clean":"2024-03-21T08:30:35.330747+0000","last_became_active":"2024-03-21T08:28:36.662726+0000","last_became_peered":"2024-03-21T08:28:36.662726+0000","last_unstale":"2024-03-21T08:30:35.330747+0000","last_undegraded":"2024-03-21T08:30:35.330747+0000","last_fullsized":"2024-03-21T08:30:35.330747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:29.129014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663764+0000","last_change":"2024-03-21T08:28:40.885203+0000","last_active":"2024-03-21T08:30:06.663764+0000","last_peered":"2024-03-21T08:30:06.663764+0000","last_clean":"2024-03-21T08:30:06.663764+0000","last_became_active":"2024-03-21T08:28:40.883810+0000","last_became_peered":"2024-03-21T08:28:40.883810+0000","last_unstale":"2024-03-21T08:30:06.663764+0000","last_undegraded":"2024-03-21T08:30:06.663764+0000","last_fullsized":"2024-03-21T08:30:06.663764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:12:58.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651247+0000","last_change":"2024-03-21T08:28:35.690494+0000","last_active":"2024-03-21T08:30:06.651247+0000","last_peered":"2024-03-21T08:30:06.651247+0000","last_clean":"2024-03-21T08:30:06.651247+0000","last_became_active":"2024-03-21T08:28:35.690071+0000","last_became_peered":"2024-03-21T08:28:35.690071+0000","last_unstale":"2024-03-21T08:30:06.651247+0000","last_undegraded":"2024-03-21T08:30:06.651247+0000","last_fullsized":"2024-03-21T08:30:06.651247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:04:46.713006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333944+0000","last_change":"2024-03-21T08:28:36.665135+0000","last_active":"2024-03-21T08:30:35.333944+0000","last_peered":"2024-03-21T08:30:35.333944+0000","last_clean":"2024-03-21T08:30:35.333944+0000","last_became_active":"2024-03-21T08:28:36.663700+0000","last_became_peered":"2024-03-21T08:28:36.663700+0000","last_unstale":"2024-03-21T08:30:35.333944+0000","last_undegraded":"2024-03-21T08:30:35.333944+0000","last_fullsized":"2024-03-21T08:30:35.333944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:50:00.604958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649998+0000","last_change":"2024-03-21T08:28:40.696076+0000","last_active":"2024-03-21T08:30:06.649998+0000","last_peered":"2024-03-21T08:30:06.649998+0000","last_clean":"2024-03-21T08:30:06.649998+0000","last_became_active":"2024-03-21T08:28:40.695556+0000","last_became_peered":"2024-03-21T08:28:40.695556+0000","last_unstale":"2024-03-21T08:30:06.649998+0000","last_undegraded":"2024-03-21T08:30:06.649998+0000","last_fullsized":"2024-03-21T08:30:06.649998+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:42.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646454+0000","last_change":"2024-03-21T08:28:35.685545+0000","last_active":"2024-03-21T08:30:06.646454+0000","last_peered":"2024-03-21T08:30:06.646454+0000","last_clean":"2024-03-21T08:30:06.646454+0000","last_became_active":"2024-03-21T08:28:35.684795+0000","last_became_peered":"2024-03-21T08:28:35.684795+0000","last_unstale":"2024-03-21T08:30:06.646454+0000","last_undegraded":"2024-03-21T08:30:06.646454+0000","last_fullsized":"2024-03-21T08:30:06.646454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:40.308014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.334201+0000","last_change":"2024-03-21T08:28:36.656276+0000","last_active":"2024-03-21T08:30:35.334201+0000","last_peered":"2024-03-21T08:30:35.334201+0000","last_clean":"2024-03-21T08:30:35.334201+0000","last_became_active":"2024-03-21T08:28:36.655871+0000","last_became_peered":"2024-03-21T08:28:36.655871+0000","last_unstale":"2024-03-21T08:30:35.334201+0000","last_undegraded":"2024-03-21T08:30:35.334201+0000","last_fullsized":"2024-03-21T08:30:35.334201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:22.362165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651972+0000","last_change":"2024-03-21T08:29:06.783155+0000","last_active":"2024-03-21T08:30:06.651972+0000","last_peered":"2024-03-21T08:30:06.651972+0000","last_clean":"2024-03-21T08:30:06.651972+0000","last_became_active":"2024-03-21T08:29:06.782546+0000","last_became_peered":"2024-03-21T08:29:06.782546+0000","last_unstale":"2024-03-21T08:30:06.651972+0000","last_undegraded":"2024-03-21T08:30:06.651972+0000","last_fullsized":"2024-03-21T08:30:06.651972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:16.489888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666681+0000","last_change":"2024-03-21T08:28:35.672783+0000","last_active":"2024-03-21T08:30:06.666681+0000","last_peered":"2024-03-21T08:30:06.666681+0000","last_clean":"2024-03-21T08:30:06.666681+0000","last_became_active":"2024-03-21T08:28:35.672161+0000","last_became_peered":"2024-03-21T08:28:35.672161+0000","last_unstale":"2024-03-21T08:30:06.666681+0000","last_undegraded":"2024-03-21T08:30:06.666681+0000","last_fullsized":"2024-03-21T08:30:06.666681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:37.180625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315686+0000","last_change":"2024-03-21T08:28:36.659447+0000","last_active":"2024-03-21T08:30:35.315686+0000","last_peered":"2024-03-21T08:30:35.315686+0000","last_clean":"2024-03-21T08:30:35.315686+0000","last_became_active":"2024-03-21T08:28:36.657773+0000","last_became_peered":"2024-03-21T08:28:36.657773+0000","last_unstale":"2024-03-21T08:30:35.315686+0000","last_undegraded":"2024-03-21T08:30:35.315686+0000","last_fullsized":"2024-03-21T08:30:35.315686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:05.324498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664330+0000","last_change":"2024-03-21T08:28:40.679806+0000","last_active":"2024-03-21T08:30:06.664330+0000","last_peered":"2024-03-21T08:30:06.664330+0000","last_clean":"2024-03-21T08:30:06.664330+0000","last_became_active":"2024-03-21T08:28:40.679329+0000","last_became_peered":"2024-03-21T08:28:40.679329+0000","last_unstale":"2024-03-21T08:30:06.664330+0000","last_undegraded":"2024-03-21T08:30:06.664330+0000","last_fullsized":"2024-03-21T08:30:06.664330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:30:58.297581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.675639+0000","last_change":"2024-03-21T08:30:06.675069+0000","last_active":"2024-03-21T08:30:06.675639+0000","last_peered":"2024-03-21T08:30:06.675639+0000","last_clean":"2024-03-21T08:30:06.675639+0000","last_became_active":"2024-03-21T08:30:06.665699+0000","last_became_peered":"2024-03-21T08:30:06.665699+0000","last_unstale":"2024-03-21T08:30:06.675639+0000","last_undegraded":"2024-03-21T08:30:06.675639+0000","last_fullsized":"2024-03-21T08:30:06.675639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:30:07.336423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319014+0000","last_change":"2024-03-21T08:28:36.649194+0000","last_active":"2024-03-21T08:30:35.319014+0000","last_peered":"2024-03-21T08:30:35.319014+0000","last_clean":"2024-03-21T08:30:35.319014+0000","last_became_active":"2024-03-21T08:28:36.648663+0000","last_became_peered":"2024-03-21T08:28:36.648663+0000","last_unstale":"2024-03-21T08:30:35.319014+0000","last_undegraded":"2024-03-21T08:30:35.319014+0000","last_fullsized":"2024-03-21T08:30:35.319014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:16.296886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313728+0000","last_change":"2024-03-21T08:28:36.664473+0000","last_active":"2024-03-21T08:30:35.313728+0000","last_peered":"2024-03-21T08:30:35.313728+0000","last_clean":"2024-03-21T08:30:35.313728+0000","last_became_active":"2024-03-21T08:28:36.663785+0000","last_became_peered":"2024-03-21T08:28:36.663785+0000","last_unstale":"2024-03-21T08:30:35.313728+0000","last_undegraded":"2024-03-21T08:30:35.313728+0000","last_fullsized":"2024-03-21T08:30:35.313728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:00.409224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.309044+0000","last_change":"2024-03-21T08:28:36.668372+0000","last_active":"2024-03-21T08:30:35.309044+0000","last_peered":"2024-03-21T08:30:35.309044+0000","last_clean":"2024-03-21T08:30:35.309044+0000","last_became_active":"2024-03-21T08:28:36.666227+0000","last_became_peered":"2024-03-21T08:28:36.666227+0000","last_unstale":"2024-03-21T08:30:35.309044+0000","last_undegraded":"2024-03-21T08:30:35.309044+0000","last_fullsized":"2024-03-21T08:30:35.309044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:40:56.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317658+0000","last_change":"2024-03-21T08:28:36.670770+0000","last_active":"2024-03-21T08:30:35.317658+0000","last_peered":"2024-03-21T08:30:35.317658+0000","last_clean":"2024-03-21T08:30:35.317658+0000","last_became_active":"2024-03-21T08:28:36.670467+0000","last_became_peered":"2024-03-21T08:28:36.670467+0000","last_unstale":"2024-03-21T08:30:35.317658+0000","last_undegraded":"2024-03-21T08:30:35.317658+0000","last_fullsized":"2024-03-21T08:30:35.317658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:48.075189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329527+0000","last_change":"2024-03-21T08:28:36.669395+0000","last_active":"2024-03-21T08:30:35.329527+0000","last_peered":"2024-03-21T08:30:35.329527+0000","last_clean":"2024-03-21T08:30:35.329527+0000","last_became_active":"2024-03-21T08:28:36.668788+0000","last_became_peered":"2024-03-21T08:28:36.668788+0000","last_unstale":"2024-03-21T08:30:35.329527+0000","last_undegraded":"2024-03-21T08:30:35.329527+0000","last_fullsized":"2024-03-21T08:30:35.329527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:03.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.326209+0000","last_change":"2024-03-21T08:30:06.655160+0000","last_active":"2024-03-21T08:30:35.326209+0000","last_peered":"2024-03-21T08:30:35.326209+0000","last_clean":"2024-03-21T08:30:35.326209+0000","last_became_active":"2024-03-21T08:30:06.654744+0000","last_became_peered":"2024-03-21T08:30:06.654744+0000","last_unstale":"2024-03-21T08:30:35.326209+0000","last_undegraded":"2024-03-21T08:30:35.326209+0000","last_fullsized":"2024-03-21T08:30:35.326209+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:19:54.690562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.308832+0000","last_change":"2024-03-21T08:28:36.666836+0000","last_active":"2024-03-21T08:30:35.308832+0000","last_peered":"2024-03-21T08:30:35.308832+0000","last_clean":"2024-03-21T08:30:35.308832+0000","last_became_active":"2024-03-21T08:28:36.666538+0000","last_became_peered":"2024-03-21T08:28:36.666538+0000","last_unstale":"2024-03-21T08:30:35.308832+0000","last_undegraded":"2024-03-21T08:30:35.308832+0000","last_fullsized":"2024-03-21T08:30:35.308832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:43.353269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.325089+0000","last_change":"2024-03-21T08:28:36.667823+0000","last_active":"2024-03-21T08:30:35.325089+0000","last_peered":"2024-03-21T08:30:35.325089+0000","last_clean":"2024-03-21T08:30:35.325089+0000","last_became_active":"2024-03-21T08:28:36.666096+0000","last_became_peered":"2024-03-21T08:28:36.666096+0000","last_unstale":"2024-03-21T08:30:35.325089+0000","last_undegraded":"2024-03-21T08:30:35.325089+0000","last_fullsized":"2024-03-21T08:30:35.325089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:58.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312965+0000","last_change":"2024-03-21T08:28:36.672164+0000","last_active":"2024-03-21T08:30:35.312965+0000","last_peered":"2024-03-21T08:30:35.312965+0000","last_clean":"2024-03-21T08:30:35.312965+0000","last_became_active":"2024-03-21T08:28:36.671812+0000","last_became_peered":"2024-03-21T08:28:36.671812+0000","last_unstale":"2024-03-21T08:30:35.312965+0000","last_undegraded":"2024-03-21T08:30:35.312965+0000","last_fullsized":"2024-03-21T08:30:35.312965+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:24:19.687108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316596+0000","last_change":"2024-03-21T08:28:36.675056+0000","last_active":"2024-03-21T08:30:35.316596+0000","last_peered":"2024-03-21T08:30:35.316596+0000","last_clean":"2024-03-21T08:30:35.316596+0000","last_became_active":"2024-03-21T08:28:36.674586+0000","last_became_peered":"2024-03-21T08:28:36.674586+0000","last_unstale":"2024-03-21T08:30:35.316596+0000","last_undegraded":"2024-03-21T08:30:35.316596+0000","last_fullsized":"2024-03-21T08:30:35.316596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:04:51.207066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313977+0000","last_change":"2024-03-21T08:28:36.666370+0000","last_active":"2024-03-21T08:30:35.313977+0000","last_peered":"2024-03-21T08:30:35.313977+0000","last_clean":"2024-03-21T08:30:35.313977+0000","last_became_active":"2024-03-21T08:28:36.665950+0000","last_became_peered":"2024-03-21T08:28:36.665950+0000","last_unstale":"2024-03-21T08:30:35.313977+0000","last_undegraded":"2024-03-21T08:30:35.313977+0000","last_fullsized":"2024-03-21T08:30:35.313977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:49:31.945890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330531+0000","last_change":"2024-03-21T08:29:07.032862+0000","last_active":"2024-03-21T08:30:35.330531+0000","last_peered":"2024-03-21T08:30:35.330531+0000","last_clean":"2024-03-21T08:30:35.330531+0000","last_became_active":"2024-03-21T08:29:07.031784+0000","last_became_peered":"2024-03-21T08:29:07.031784+0000","last_unstale":"2024-03-21T08:30:35.330531+0000","last_undegraded":"2024-03-21T08:30:35.330531+0000","last_fullsized":"2024-03-21T08:30:35.330531+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:34.037495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330945+0000","last_change":"2024-03-21T08:28:36.660812+0000","last_active":"2024-03-21T08:30:35.330945+0000","last_peered":"2024-03-21T08:30:35.330945+0000","last_clean":"2024-03-21T08:30:35.330945+0000","last_became_active":"2024-03-21T08:28:36.660044+0000","last_became_peered":"2024-03-21T08:28:36.660044+0000","last_unstale":"2024-03-21T08:30:35.330945+0000","last_undegraded":"2024-03-21T08:30:35.330945+0000","last_fullsized":"2024-03-21T08:30:35.330945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:31.598678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316333+0000","last_change":"2024-03-21T08:28:36.669441+0000","last_active":"2024-03-21T08:30:35.316333+0000","last_peered":"2024-03-21T08:30:35.316333+0000","last_clean":"2024-03-21T08:30:35.316333+0000","last_became_active":"2024-03-21T08:28:36.668668+0000","last_became_peered":"2024-03-21T08:28:36.668668+0000","last_unstale":"2024-03-21T08:30:35.316333+0000","last_undegraded":"2024-03-21T08:30:35.316333+0000","last_fullsized":"2024-03-21T08:30:35.316333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:04.828297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72517,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":904,"num_read_kb":530,"num_write":181,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":217551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285621,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":534350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547258,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1189500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710786,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":538682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972425,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":533924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266769,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712316,"statfs":{"total":95995035648,"available":95961411584,"internally_reserved":0,"allocated":1699840,"data_stored":1131542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":529784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":597643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":531150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":595122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":529956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":534905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1126615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:31:35.950 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T08:31:36.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.209+0000 7f4a0dc00640 1 -- 172.21.15.3:0/830279946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 msgr2=0x7f4a08109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.209+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/830279946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a08109230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009900 tx=0x7f49f002bc40 comp rx=0 tx=0).stop 2024-03-21T08:31:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.209+0000 7f4a0dc00640 1 -- 172.21.15.3:0/830279946 shutdown_connections 2024-03-21T08:31:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.209+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/830279946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a08109770 0x7f4a08110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.209+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/830279946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a08109230 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.209+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/830279946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a08107bc0 0x7f4a08107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.209+0000 7f4a0dc00640 1 -- 172.21.15.3:0/830279946 >> 172.21.15.3:0/830279946 conn(0x7f4a08076b60 msgr2=0x7f4a08078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.210+0000 7f4a0dc00640 1 -- 172.21.15.3:0/830279946 shutdown_connections 2024-03-21T08:31:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.210+0000 7f4a0dc00640 1 -- 172.21.15.3:0/830279946 wait complete. 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.210+0000 7f4a0dc00640 1 Processor -- start 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0dc00640 1 -- start start 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0dc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a08107bc0 0x7f4a0819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a0819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0dc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a08109770 0x7f4a081a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0dc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081136d0 con 0x7f4a08108db0 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0dc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a08113550 con 0x7f4a08109770 2024-03-21T08:31:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0dc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a08113850 con 0x7f4a08107bc0 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f49ffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a0819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f49ffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a0819e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34254/0 (socket says 172.21.15.3:34254) 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f49ffe00640 1 -- 172.21.15.3:0/1396731347 learned_addr learned my addr 172.21.15.3:0/1396731347 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a07e00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a08107bc0 0x7f4a0819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.211+0000 7f4a0d200640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a08109770 0x7f4a081a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f49ffe00640 1 -- 172.21.15.3:0/1396731347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a08107bc0 msgr2=0x7f4a0819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f49ffe00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a08107bc0 0x7f4a0819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f49ffe00640 1 -- 172.21.15.3:0/1396731347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a08109770 msgr2=0x7f4a081a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f49ffe00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a08109770 0x7f4a081a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f49ffe00640 1 -- 172.21.15.3:0/1396731347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a081a59b0 con 0x7f4a08108db0 2024-03-21T08:31:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f49ffe00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a0819e2e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006010 tx=0x7f49f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f4a05600640 1 -- 172.21.15.3:0/1396731347 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0043020 con 0x7f4a08108db0 2024-03-21T08:31:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a081a5c40 con 0x7f4a08108db0 2024-03-21T08:31:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f4a05600640 1 -- 172.21.15.3:0/1396731347 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f49f0004aa0 con 0x7f4a08108db0 2024-03-21T08:31:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.212+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a081a6190 con 0x7f4a08108db0 2024-03-21T08:31:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.213+0000 7f4a05600640 1 -- 172.21.15.3:0/1396731347 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0003c20 con 0x7f4a08108db0 2024-03-21T08:31:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.214+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0810eee0 con 0x7f4a08108db0 2024-03-21T08:31:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.215+0000 7f4a05600640 1 -- 172.21.15.3:0/1396731347 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f49f0046020 con 0x7f4a08108db0 2024-03-21T08:31:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.217+0000 7f4a05600640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49ec0a2f10 0x7f49ec0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.217+0000 7f4a05600640 1 -- 172.21.15.3:0/1396731347 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f49f00f1d30 con 0x7f4a08108db0 2024-03-21T08:31:36.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.217+0000 7f4a07e00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49ec0a2f10 0x7f49ec0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.218+0000 7f4a07e00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49ec0a2f10 0x7f49ec0a53d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005d00 tx=0x7f49f8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.221+0000 7f4a05600640 1 -- 172.21.15.3:0/1396731347 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49f004d020 con 0x7f4a08108db0 2024-03-21T08:31:36.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.424+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a08061ae0 con 0x7f49ec0a2f10 2024-03-21T08:31:36.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.453+0000 7f4a05600640 1 -- 172.21.15.3:0/1396731347 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340532 (secure 0 0 0) 0x7f4a08061ae0 con 0x7f49ec0a2f10 2024-03-21T08:31:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:36.463 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49ec0a2f10 msgr2=0x7f49ec0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49ec0a2f10 0x7f49ec0a53d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005d00 tx=0x7f49f8005c70 comp rx=0 tx=0).stop 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 msgr2=0x7f4a0819e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a0819e2e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006010 tx=0x7f49f00042f0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 shutdown_connections 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a08109770 0x7f4a081a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a08108db0 0x7f4a0819e2e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f49ec0a2f10 0x7f49ec0a53d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 --2- 172.21.15.3:0/1396731347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a08107bc0 0x7f4a0819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 >> 172.21.15.3:0/1396731347 conn(0x7f4a08076b60 msgr2=0x7f4a0810c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 shutdown_connections 2024-03-21T08:31:36.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.461+0000 7f4a0dc00640 1 -- 172.21.15.3:0/1396731347 wait complete. 2024-03-21T08:31:36.484 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-03-21T08:31:35.414486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663119,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1040,"num_read_kb":649,"num_write":364,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372344,"kb_used_data":14872,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569480,"statfs":{"total":1151940427776,"available":1151559147520,"internally_reserved":0,"allocated":15228928,"data_stored":8373173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005448"},"pg_stats":[{"pgid":"3.3f","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332311+0000","last_change":"2024-03-21T08:28:36.667851+0000","last_active":"2024-03-21T08:30:35.332311+0000","last_peered":"2024-03-21T08:30:35.332311+0000","last_clean":"2024-03-21T08:30:35.332311+0000","last_became_active":"2024-03-21T08:28:36.665713+0000","last_became_peered":"2024-03-21T08:28:36.665713+0000","last_unstale":"2024-03-21T08:30:35.332311+0000","last_undegraded":"2024-03-21T08:30:35.332311+0000","last_fullsized":"2024-03-21T08:30:35.332311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:02.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329013+0000","last_change":"2024-03-21T08:28:36.650986+0000","last_active":"2024-03-21T08:30:35.329013+0000","last_peered":"2024-03-21T08:30:35.329013+0000","last_clean":"2024-03-21T08:30:35.329013+0000","last_became_active":"2024-03-21T08:28:36.650633+0000","last_became_peered":"2024-03-21T08:28:36.650633+0000","last_unstale":"2024-03-21T08:30:35.329013+0000","last_undegraded":"2024-03-21T08:30:35.329013+0000","last_fullsized":"2024-03-21T08:30:35.329013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:40:20.801814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.335139+0000","last_change":"2024-03-21T08:28:36.662984+0000","last_active":"2024-03-21T08:30:35.335139+0000","last_peered":"2024-03-21T08:30:35.335139+0000","last_clean":"2024-03-21T08:30:35.335139+0000","last_became_active":"2024-03-21T08:28:36.662657+0000","last_became_peered":"2024-03-21T08:28:36.662657+0000","last_unstale":"2024-03-21T08:30:35.335139+0000","last_undegraded":"2024-03-21T08:30:35.335139+0000","last_fullsized":"2024-03-21T08:30:35.335139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:41.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.299562+0000","last_change":"2024-03-21T08:28:36.665958+0000","last_active":"2024-03-21T08:30:35.299562+0000","last_peered":"2024-03-21T08:30:35.299562+0000","last_clean":"2024-03-21T08:30:35.299562+0000","last_became_active":"2024-03-21T08:28:36.664309+0000","last_became_peered":"2024-03-21T08:28:36.664309+0000","last_unstale":"2024-03-21T08:30:35.299562+0000","last_undegraded":"2024-03-21T08:30:35.299562+0000","last_fullsized":"2024-03-21T08:30:35.299562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:16.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315553+0000","last_change":"2024-03-21T08:28:36.657767+0000","last_active":"2024-03-21T08:30:35.315553+0000","last_peered":"2024-03-21T08:30:35.315553+0000","last_clean":"2024-03-21T08:30:35.315553+0000","last_became_active":"2024-03-21T08:28:36.657350+0000","last_became_peered":"2024-03-21T08:28:36.657350+0000","last_unstale":"2024-03-21T08:30:35.315553+0000","last_undegraded":"2024-03-21T08:30:35.315553+0000","last_fullsized":"2024-03-21T08:30:35.315553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:47.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312622+0000","last_change":"2024-03-21T08:29:06.916971+0000","last_active":"2024-03-21T08:30:35.312622+0000","last_peered":"2024-03-21T08:30:35.312622+0000","last_clean":"2024-03-21T08:30:35.312622+0000","last_became_active":"2024-03-21T08:29:06.915997+0000","last_became_peered":"2024-03-21T08:29:06.915997+0000","last_unstale":"2024-03-21T08:30:35.312622+0000","last_undegraded":"2024-03-21T08:30:35.312622+0000","last_fullsized":"2024-03-21T08:30:35.312622+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:02.525190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312579+0000","last_change":"2024-03-21T08:28:36.667787+0000","last_active":"2024-03-21T08:30:35.312579+0000","last_peered":"2024-03-21T08:30:35.312579+0000","last_clean":"2024-03-21T08:30:35.312579+0000","last_became_active":"2024-03-21T08:28:36.667451+0000","last_became_peered":"2024-03-21T08:28:36.667451+0000","last_unstale":"2024-03-21T08:30:35.312579+0000","last_undegraded":"2024-03-21T08:30:35.312579+0000","last_fullsized":"2024-03-21T08:30:35.312579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:01.874897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328289+0000","last_change":"2024-03-21T08:28:36.662445+0000","last_active":"2024-03-21T08:30:35.328289+0000","last_peered":"2024-03-21T08:30:35.328289+0000","last_clean":"2024-03-21T08:30:35.328289+0000","last_became_active":"2024-03-21T08:28:36.662113+0000","last_became_peered":"2024-03-21T08:28:36.662113+0000","last_unstale":"2024-03-21T08:30:35.328289+0000","last_undegraded":"2024-03-21T08:30:35.328289+0000","last_fullsized":"2024-03-21T08:30:35.328289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:21.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331556+0000","last_change":"2024-03-21T08:28:36.666084+0000","last_active":"2024-03-21T08:30:35.331556+0000","last_peered":"2024-03-21T08:30:35.331556+0000","last_clean":"2024-03-21T08:30:35.331556+0000","last_became_active":"2024-03-21T08:28:36.665354+0000","last_became_peered":"2024-03-21T08:28:36.665354+0000","last_unstale":"2024-03-21T08:30:35.331556+0000","last_undegraded":"2024-03-21T08:30:35.331556+0000","last_fullsized":"2024-03-21T08:30:35.331556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:02:50.160765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319234+0000","last_change":"2024-03-21T08:29:06.919231+0000","last_active":"2024-03-21T08:30:35.319234+0000","last_peered":"2024-03-21T08:30:35.319234+0000","last_clean":"2024-03-21T08:30:35.319234+0000","last_became_active":"2024-03-21T08:29:06.918199+0000","last_became_peered":"2024-03-21T08:29:06.918199+0000","last_unstale":"2024-03-21T08:30:35.319234+0000","last_undegraded":"2024-03-21T08:30:35.319234+0000","last_fullsized":"2024-03-21T08:30:35.319234+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:44.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333943+0000","last_change":"2024-03-21T08:28:36.656385+0000","last_active":"2024-03-21T08:30:35.333943+0000","last_peered":"2024-03-21T08:30:35.333943+0000","last_clean":"2024-03-21T08:30:35.333943+0000","last_became_active":"2024-03-21T08:28:36.656014+0000","last_became_peered":"2024-03-21T08:28:36.656014+0000","last_unstale":"2024-03-21T08:30:35.333943+0000","last_undegraded":"2024-03-21T08:30:35.333943+0000","last_fullsized":"2024-03-21T08:30:35.333943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:46.000495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319565+0000","last_change":"2024-03-21T08:28:36.666254+0000","last_active":"2024-03-21T08:30:35.319565+0000","last_peered":"2024-03-21T08:30:35.319565+0000","last_clean":"2024-03-21T08:30:35.319565+0000","last_became_active":"2024-03-21T08:28:36.664698+0000","last_became_peered":"2024-03-21T08:28:36.664698+0000","last_unstale":"2024-03-21T08:30:35.319565+0000","last_undegraded":"2024-03-21T08:30:35.319565+0000","last_fullsized":"2024-03-21T08:30:35.319565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:03:08.358159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.327841+0000","last_change":"2024-03-21T08:29:06.919930+0000","last_active":"2024-03-21T08:30:35.327841+0000","last_peered":"2024-03-21T08:30:35.327841+0000","last_clean":"2024-03-21T08:30:35.327841+0000","last_became_active":"2024-03-21T08:29:06.917852+0000","last_became_peered":"2024-03-21T08:29:06.917852+0000","last_unstale":"2024-03-21T08:30:35.327841+0000","last_undegraded":"2024-03-21T08:30:35.327841+0000","last_fullsized":"2024-03-21T08:30:35.327841+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:05:01.185388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328029+0000","last_change":"2024-03-21T08:28:36.655229+0000","last_active":"2024-03-21T08:30:35.328029+0000","last_peered":"2024-03-21T08:30:35.328029+0000","last_clean":"2024-03-21T08:30:35.328029+0000","last_became_active":"2024-03-21T08:28:36.653949+0000","last_became_peered":"2024-03-21T08:28:36.653949+0000","last_unstale":"2024-03-21T08:30:35.328029+0000","last_undegraded":"2024-03-21T08:30:35.328029+0000","last_fullsized":"2024-03-21T08:30:35.328029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:57:38.370897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328261+0000","last_change":"2024-03-21T08:28:36.664894+0000","last_active":"2024-03-21T08:30:35.328261+0000","last_peered":"2024-03-21T08:30:35.328261+0000","last_clean":"2024-03-21T08:30:35.328261+0000","last_became_active":"2024-03-21T08:28:36.664523+0000","last_became_peered":"2024-03-21T08:28:36.664523+0000","last_unstale":"2024-03-21T08:30:35.328261+0000","last_undegraded":"2024-03-21T08:30:35.328261+0000","last_fullsized":"2024-03-21T08:30:35.328261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:52:54.147459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333446+0000","last_change":"2024-03-21T08:28:36.662374+0000","last_active":"2024-03-21T08:30:35.333446+0000","last_peered":"2024-03-21T08:30:35.333446+0000","last_clean":"2024-03-21T08:30:35.333446+0000","last_became_active":"2024-03-21T08:28:36.662066+0000","last_became_peered":"2024-03-21T08:28:36.662066+0000","last_unstale":"2024-03-21T08:30:35.333446+0000","last_undegraded":"2024-03-21T08:30:35.333446+0000","last_fullsized":"2024-03-21T08:30:35.333446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:06.082925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332398+0000","last_change":"2024-03-21T08:28:36.672172+0000","last_active":"2024-03-21T08:30:35.332398+0000","last_peered":"2024-03-21T08:30:35.332398+0000","last_clean":"2024-03-21T08:30:35.332398+0000","last_became_active":"2024-03-21T08:28:36.671657+0000","last_became_peered":"2024-03-21T08:28:36.671657+0000","last_unstale":"2024-03-21T08:30:35.332398+0000","last_undegraded":"2024-03-21T08:30:35.332398+0000","last_fullsized":"2024-03-21T08:30:35.332398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:56.892818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332036+0000","last_change":"2024-03-21T08:28:36.666973+0000","last_active":"2024-03-21T08:30:35.332036+0000","last_peered":"2024-03-21T08:30:35.332036+0000","last_clean":"2024-03-21T08:30:35.332036+0000","last_became_active":"2024-03-21T08:28:36.663770+0000","last_became_peered":"2024-03-21T08:28:36.663770+0000","last_unstale":"2024-03-21T08:30:35.332036+0000","last_undegraded":"2024-03-21T08:30:35.332036+0000","last_fullsized":"2024-03-21T08:30:35.332036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:54:26.701009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.318007+0000","last_change":"2024-03-21T08:28:36.668793+0000","last_active":"2024-03-21T08:30:35.318007+0000","last_peered":"2024-03-21T08:30:35.318007+0000","last_clean":"2024-03-21T08:30:35.318007+0000","last_became_active":"2024-03-21T08:28:36.668452+0000","last_became_peered":"2024-03-21T08:28:36.668452+0000","last_unstale":"2024-03-21T08:30:35.318007+0000","last_undegraded":"2024-03-21T08:30:35.318007+0000","last_fullsized":"2024-03-21T08:30:35.318007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:45.191849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645032+0000","last_change":"2024-03-21T08:28:40.687618+0000","last_active":"2024-03-21T08:30:06.645032+0000","last_peered":"2024-03-21T08:30:06.645032+0000","last_clean":"2024-03-21T08:30:06.645032+0000","last_became_active":"2024-03-21T08:28:40.687155+0000","last_became_peered":"2024-03-21T08:28:40.687155+0000","last_unstale":"2024-03-21T08:30:06.645032+0000","last_undegraded":"2024-03-21T08:30:06.645032+0000","last_fullsized":"2024-03-21T08:30:06.645032+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:08.014790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'9","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.673458+0000","last_change":"2024-03-21T08:30:06.672911+0000","last_active":"2024-03-21T08:30:06.673458+0000","last_peered":"2024-03-21T08:30:06.673458+0000","last_clean":"2024-03-21T08:30:06.673458+0000","last_became_active":"2024-03-21T08:30:06.661974+0000","last_became_peered":"2024-03-21T08:30:06.661974+0000","last_unstale":"2024-03-21T08:30:06.673458+0000","last_undegraded":"2024-03-21T08:30:06.673458+0000","last_fullsized":"2024-03-21T08:30:06.673458+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:37:25.848279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313894+0000","last_change":"2024-03-21T08:28:36.670325+0000","last_active":"2024-03-21T08:30:35.313894+0000","last_peered":"2024-03-21T08:30:35.313894+0000","last_clean":"2024-03-21T08:30:35.313894+0000","last_became_active":"2024-03-21T08:28:36.669783+0000","last_became_peered":"2024-03-21T08:28:36.669783+0000","last_unstale":"2024-03-21T08:30:35.313894+0000","last_undegraded":"2024-03-21T08:30:35.313894+0000","last_fullsized":"2024-03-21T08:30:35.313894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:24.166567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663706+0000","last_change":"2024-03-21T08:28:40.689849+0000","last_active":"2024-03-21T08:30:06.663706+0000","last_peered":"2024-03-21T08:30:06.663706+0000","last_clean":"2024-03-21T08:30:06.663706+0000","last_became_active":"2024-03-21T08:28:40.689307+0000","last_became_peered":"2024-03-21T08:28:40.689307+0000","last_unstale":"2024-03-21T08:30:06.663706+0000","last_undegraded":"2024-03-21T08:30:06.663706+0000","last_fullsized":"2024-03-21T08:30:06.663706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:34:05.660317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647944+0000","last_change":"2024-03-21T08:28:35.674685+0000","last_active":"2024-03-21T08:30:06.647944+0000","last_peered":"2024-03-21T08:30:06.647944+0000","last_clean":"2024-03-21T08:30:06.647944+0000","last_became_active":"2024-03-21T08:28:35.674312+0000","last_became_peered":"2024-03-21T08:28:35.674312+0000","last_unstale":"2024-03-21T08:30:06.647944+0000","last_undegraded":"2024-03-21T08:30:06.647944+0000","last_fullsized":"2024-03-21T08:30:06.647944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:27.644111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.335342+0000","last_change":"2024-03-21T08:28:36.667744+0000","last_active":"2024-03-21T08:30:35.335342+0000","last_peered":"2024-03-21T08:30:35.335342+0000","last_clean":"2024-03-21T08:30:35.335342+0000","last_became_active":"2024-03-21T08:28:36.667279+0000","last_became_peered":"2024-03-21T08:28:36.667279+0000","last_unstale":"2024-03-21T08:30:35.335342+0000","last_undegraded":"2024-03-21T08:30:35.335342+0000","last_fullsized":"2024-03-21T08:30:35.335342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:14:24.974683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664879+0000","last_change":"2024-03-21T08:28:40.688864+0000","last_active":"2024-03-21T08:30:06.664879+0000","last_peered":"2024-03-21T08:30:06.664879+0000","last_clean":"2024-03-21T08:30:06.664879+0000","last_became_active":"2024-03-21T08:28:40.688029+0000","last_became_peered":"2024-03-21T08:28:40.688029+0000","last_unstale":"2024-03-21T08:30:06.664879+0000","last_undegraded":"2024-03-21T08:30:06.664879+0000","last_fullsized":"2024-03-21T08:30:06.664879+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:39.235807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.681016+0000","last_change":"2024-03-21T08:30:06.680224+0000","last_active":"2024-03-21T08:30:06.681016+0000","last_peered":"2024-03-21T08:30:06.681016+0000","last_clean":"2024-03-21T08:30:06.681016+0000","last_became_active":"2024-03-21T08:30:06.665898+0000","last_became_peered":"2024-03-21T08:30:06.665898+0000","last_unstale":"2024-03-21T08:30:06.681016+0000","last_undegraded":"2024-03-21T08:30:06.681016+0000","last_fullsized":"2024-03-21T08:30:06.681016+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:21.690576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333879+0000","last_change":"2024-03-21T08:28:36.667244+0000","last_active":"2024-03-21T08:30:35.333879+0000","last_peered":"2024-03-21T08:30:35.333879+0000","last_clean":"2024-03-21T08:30:35.333879+0000","last_became_active":"2024-03-21T08:28:36.666829+0000","last_became_peered":"2024-03-21T08:28:36.666829+0000","last_unstale":"2024-03-21T08:30:35.333879+0000","last_undegraded":"2024-03-21T08:30:35.333879+0000","last_fullsized":"2024-03-21T08:30:35.333879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:12:56.842084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657793+0000","last_change":"2024-03-21T08:28:40.692621+0000","last_active":"2024-03-21T08:30:06.657793+0000","last_peered":"2024-03-21T08:30:06.657793+0000","last_clean":"2024-03-21T08:30:06.657793+0000","last_became_active":"2024-03-21T08:28:40.691984+0000","last_became_peered":"2024-03-21T08:28:40.691984+0000","last_unstale":"2024-03-21T08:30:06.657793+0000","last_undegraded":"2024-03-21T08:30:06.657793+0000","last_fullsized":"2024-03-21T08:30:06.657793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:29.330702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651368+0000","last_change":"2024-03-21T08:28:35.676421+0000","last_active":"2024-03-21T08:30:06.651368+0000","last_peered":"2024-03-21T08:30:06.651368+0000","last_clean":"2024-03-21T08:30:06.651368+0000","last_became_active":"2024-03-21T08:28:35.674957+0000","last_became_peered":"2024-03-21T08:28:35.674957+0000","last_unstale":"2024-03-21T08:30:06.651368+0000","last_undegraded":"2024-03-21T08:30:06.651368+0000","last_fullsized":"2024-03-21T08:30:06.651368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:01:38.397757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331569+0000","last_change":"2024-03-21T08:28:36.664662+0000","last_active":"2024-03-21T08:30:35.331569+0000","last_peered":"2024-03-21T08:30:35.331569+0000","last_clean":"2024-03-21T08:30:35.331569+0000","last_became_active":"2024-03-21T08:28:36.664324+0000","last_became_peered":"2024-03-21T08:28:36.664324+0000","last_unstale":"2024-03-21T08:30:35.331569+0000","last_undegraded":"2024-03-21T08:30:35.331569+0000","last_fullsized":"2024-03-21T08:30:35.331569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:57.325869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645495+0000","last_change":"2024-03-21T08:28:40.676297+0000","last_active":"2024-03-21T08:30:06.645495+0000","last_peered":"2024-03-21T08:30:06.645495+0000","last_clean":"2024-03-21T08:30:06.645495+0000","last_became_active":"2024-03-21T08:28:40.675337+0000","last_became_peered":"2024-03-21T08:28:40.675337+0000","last_unstale":"2024-03-21T08:30:06.645495+0000","last_undegraded":"2024-03-21T08:30:06.645495+0000","last_fullsized":"2024-03-21T08:30:06.645495+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:05.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'8","reported_seq":189,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:31.737556+0000","last_change":"2024-03-21T08:28:35.687651+0000","last_active":"2024-03-21T08:31:31.737556+0000","last_peered":"2024-03-21T08:31:31.737556+0000","last_clean":"2024-03-21T08:31:31.737556+0000","last_became_active":"2024-03-21T08:28:35.686284+0000","last_became_peered":"2024-03-21T08:28:35.686284+0000","last_unstale":"2024-03-21T08:31:31.737556+0000","last_undegraded":"2024-03-21T08:31:31.737556+0000","last_fullsized":"2024-03-21T08:31:31.737556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:56.121798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315336+0000","last_change":"2024-03-21T08:29:07.033436+0000","last_active":"2024-03-21T08:30:35.315336+0000","last_peered":"2024-03-21T08:30:35.315336+0000","last_clean":"2024-03-21T08:30:35.315336+0000","last_became_active":"2024-03-21T08:29:07.032012+0000","last_became_peered":"2024-03-21T08:29:07.032012+0000","last_unstale":"2024-03-21T08:30:35.315336+0000","last_undegraded":"2024-03-21T08:30:35.315336+0000","last_fullsized":"2024-03-21T08:30:35.315336+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:44:47.009078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645774+0000","last_change":"2024-03-21T08:29:06.882449+0000","last_active":"2024-03-21T08:30:06.645774+0000","last_peered":"2024-03-21T08:30:06.645774+0000","last_clean":"2024-03-21T08:30:06.645774+0000","last_became_active":"2024-03-21T08:29:06.880682+0000","last_became_peered":"2024-03-21T08:29:06.880682+0000","last_unstale":"2024-03-21T08:30:06.645774+0000","last_undegraded":"2024-03-21T08:30:06.645774+0000","last_fullsized":"2024-03-21T08:30:06.645774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:09.464388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.661857+0000","last_change":"2024-03-21T08:30:06.659993+0000","last_active":"2024-03-21T08:30:06.661857+0000","last_peered":"2024-03-21T08:30:06.661857+0000","last_clean":"2024-03-21T08:30:06.661857+0000","last_became_active":"2024-03-21T08:30:06.653383+0000","last_became_peered":"2024-03-21T08:30:06.653383+0000","last_unstale":"2024-03-21T08:30:06.661857+0000","last_undegraded":"2024-03-21T08:30:06.661857+0000","last_fullsized":"2024-03-21T08:30:06.661857+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:43.836881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317829+0000","last_change":"2024-03-21T08:28:36.670176+0000","last_active":"2024-03-21T08:30:35.317829+0000","last_peered":"2024-03-21T08:30:35.317829+0000","last_clean":"2024-03-21T08:30:35.317829+0000","last_became_active":"2024-03-21T08:28:36.669477+0000","last_became_peered":"2024-03-21T08:28:36.669477+0000","last_unstale":"2024-03-21T08:30:35.317829+0000","last_undegraded":"2024-03-21T08:30:35.317829+0000","last_fullsized":"2024-03-21T08:30:35.317829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:10:21.289484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654777+0000","last_change":"2024-03-21T08:29:06.784996+0000","last_active":"2024-03-21T08:30:06.654777+0000","last_peered":"2024-03-21T08:30:06.654777+0000","last_clean":"2024-03-21T08:30:06.654777+0000","last_became_active":"2024-03-21T08:29:06.783403+0000","last_became_peered":"2024-03-21T08:29:06.783403+0000","last_unstale":"2024-03-21T08:30:06.654777+0000","last_undegraded":"2024-03-21T08:30:06.654777+0000","last_fullsized":"2024-03-21T08:30:06.654777+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:18:59.912956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641118+0000","last_change":"2024-03-21T08:28:35.667502+0000","last_active":"2024-03-21T08:30:06.641118+0000","last_peered":"2024-03-21T08:30:06.641118+0000","last_clean":"2024-03-21T08:30:06.641118+0000","last_became_active":"2024-03-21T08:28:35.666667+0000","last_became_peered":"2024-03-21T08:28:35.666667+0000","last_unstale":"2024-03-21T08:30:06.641118+0000","last_undegraded":"2024-03-21T08:30:06.641118+0000","last_fullsized":"2024-03-21T08:30:06.641118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:42.196361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.318414+0000","last_change":"2024-03-21T08:28:36.670855+0000","last_active":"2024-03-21T08:30:35.318414+0000","last_peered":"2024-03-21T08:30:35.318414+0000","last_clean":"2024-03-21T08:30:35.318414+0000","last_became_active":"2024-03-21T08:28:36.666012+0000","last_became_peered":"2024-03-21T08:28:36.666012+0000","last_unstale":"2024-03-21T08:30:35.318414+0000","last_undegraded":"2024-03-21T08:30:35.318414+0000","last_fullsized":"2024-03-21T08:30:35.318414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:19.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652911+0000","last_change":"2024-03-21T08:28:40.700037+0000","last_active":"2024-03-21T08:30:06.652911+0000","last_peered":"2024-03-21T08:30:06.652911+0000","last_clean":"2024-03-21T08:30:06.652911+0000","last_became_active":"2024-03-21T08:28:40.698127+0000","last_became_peered":"2024-03-21T08:28:40.698127+0000","last_unstale":"2024-03-21T08:30:06.652911+0000","last_undegraded":"2024-03-21T08:30:06.652911+0000","last_fullsized":"2024-03-21T08:30:06.652911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:48:57.135690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664533+0000","last_change":"2024-03-21T08:28:35.677071+0000","last_active":"2024-03-21T08:30:06.664533+0000","last_peered":"2024-03-21T08:30:06.664533+0000","last_clean":"2024-03-21T08:30:06.664533+0000","last_became_active":"2024-03-21T08:28:35.676652+0000","last_became_peered":"2024-03-21T08:28:35.676652+0000","last_unstale":"2024-03-21T08:30:06.664533+0000","last_undegraded":"2024-03-21T08:30:06.664533+0000","last_fullsized":"2024-03-21T08:30:06.664533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:24:54.097952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.321747+0000","last_change":"2024-03-21T08:28:36.677981+0000","last_active":"2024-03-21T08:30:35.321747+0000","last_peered":"2024-03-21T08:30:35.321747+0000","last_clean":"2024-03-21T08:30:35.321747+0000","last_became_active":"2024-03-21T08:28:36.677512+0000","last_became_peered":"2024-03-21T08:28:36.677512+0000","last_unstale":"2024-03-21T08:30:35.321747+0000","last_undegraded":"2024-03-21T08:30:35.321747+0000","last_fullsized":"2024-03-21T08:30:35.321747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:50.742928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646728+0000","last_change":"2024-03-21T08:28:40.692672+0000","last_active":"2024-03-21T08:30:06.646728+0000","last_peered":"2024-03-21T08:30:06.646728+0000","last_clean":"2024-03-21T08:30:06.646728+0000","last_became_active":"2024-03-21T08:28:40.688940+0000","last_became_peered":"2024-03-21T08:28:40.688940+0000","last_unstale":"2024-03-21T08:30:06.646728+0000","last_undegraded":"2024-03-21T08:30:06.646728+0000","last_fullsized":"2024-03-21T08:30:06.646728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:39.569709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:32.739007+0000","last_change":"2024-03-21T08:28:35.681970+0000","last_active":"2024-03-21T08:31:32.739007+0000","last_peered":"2024-03-21T08:31:32.739007+0000","last_clean":"2024-03-21T08:31:32.739007+0000","last_became_active":"2024-03-21T08:28:35.681252+0000","last_became_peered":"2024-03-21T08:28:35.681252+0000","last_unstale":"2024-03-21T08:31:32.739007+0000","last_undegraded":"2024-03-21T08:31:32.739007+0000","last_fullsized":"2024-03-21T08:31:32.739007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:46.345359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.334525+0000","last_change":"2024-03-21T08:28:36.675419+0000","last_active":"2024-03-21T08:30:35.334525+0000","last_peered":"2024-03-21T08:30:35.334525+0000","last_clean":"2024-03-21T08:30:35.334525+0000","last_became_active":"2024-03-21T08:28:36.674039+0000","last_became_peered":"2024-03-21T08:28:36.674039+0000","last_unstale":"2024-03-21T08:30:35.334525+0000","last_undegraded":"2024-03-21T08:30:35.334525+0000","last_fullsized":"2024-03-21T08:30:35.334525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:09.667634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648585+0000","last_change":"2024-03-21T08:28:40.690076+0000","last_active":"2024-03-21T08:30:06.648585+0000","last_peered":"2024-03-21T08:30:06.648585+0000","last_clean":"2024-03-21T08:30:06.648585+0000","last_became_active":"2024-03-21T08:28:40.683817+0000","last_became_peered":"2024-03-21T08:28:40.683817+0000","last_unstale":"2024-03-21T08:30:06.648585+0000","last_undegraded":"2024-03-21T08:30:06.648585+0000","last_fullsized":"2024-03-21T08:30:06.648585+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:33:00.948879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654217+0000","last_change":"2024-03-21T08:28:35.690693+0000","last_active":"2024-03-21T08:30:06.654217+0000","last_peered":"2024-03-21T08:30:06.654217+0000","last_clean":"2024-03-21T08:30:06.654217+0000","last_became_active":"2024-03-21T08:28:35.690250+0000","last_became_peered":"2024-03-21T08:28:35.690250+0000","last_unstale":"2024-03-21T08:30:06.654217+0000","last_undegraded":"2024-03-21T08:30:06.654217+0000","last_fullsized":"2024-03-21T08:30:06.654217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:44:21.307772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330814+0000","last_change":"2024-03-21T08:28:36.661872+0000","last_active":"2024-03-21T08:30:35.330814+0000","last_peered":"2024-03-21T08:30:35.330814+0000","last_clean":"2024-03-21T08:30:35.330814+0000","last_became_active":"2024-03-21T08:28:36.661483+0000","last_became_peered":"2024-03-21T08:28:36.661483+0000","last_unstale":"2024-03-21T08:30:35.330814+0000","last_undegraded":"2024-03-21T08:30:35.330814+0000","last_fullsized":"2024-03-21T08:30:35.330814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:03:35.630141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665016+0000","last_change":"2024-03-21T08:28:40.682753+0000","last_active":"2024-03-21T08:30:06.665016+0000","last_peered":"2024-03-21T08:30:06.665016+0000","last_clean":"2024-03-21T08:30:06.665016+0000","last_became_active":"2024-03-21T08:28:40.682278+0000","last_became_peered":"2024-03-21T08:28:40.682278+0000","last_unstale":"2024-03-21T08:30:06.665016+0000","last_undegraded":"2024-03-21T08:30:06.665016+0000","last_fullsized":"2024-03-21T08:30:06.665016+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:20.280442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666308+0000","last_change":"2024-03-21T08:28:35.672524+0000","last_active":"2024-03-21T08:30:06.666308+0000","last_peered":"2024-03-21T08:30:06.666308+0000","last_clean":"2024-03-21T08:30:06.666308+0000","last_became_active":"2024-03-21T08:28:35.672051+0000","last_became_peered":"2024-03-21T08:28:35.672051+0000","last_unstale":"2024-03-21T08:30:06.666308+0000","last_undegraded":"2024-03-21T08:30:06.666308+0000","last_fullsized":"2024-03-21T08:30:06.666308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:53.336320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329907+0000","last_change":"2024-03-21T08:29:06.532172+0000","last_active":"2024-03-21T08:30:35.329907+0000","last_peered":"2024-03-21T08:30:35.329907+0000","last_clean":"2024-03-21T08:30:35.329907+0000","last_became_active":"2024-03-21T08:29:06.531439+0000","last_became_peered":"2024-03-21T08:29:06.531439+0000","last_unstale":"2024-03-21T08:30:35.329907+0000","last_undegraded":"2024-03-21T08:30:35.329907+0000","last_fullsized":"2024-03-21T08:30:35.329907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:27.222359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648624+0000","last_change":"2024-03-21T08:28:40.682294+0000","last_active":"2024-03-21T08:30:06.648624+0000","last_peered":"2024-03-21T08:30:06.648624+0000","last_clean":"2024-03-21T08:30:06.648624+0000","last_became_active":"2024-03-21T08:28:40.681855+0000","last_became_peered":"2024-03-21T08:28:40.681855+0000","last_unstale":"2024-03-21T08:30:06.648624+0000","last_undegraded":"2024-03-21T08:30:06.648624+0000","last_fullsized":"2024-03-21T08:30:06.648624+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:49.756322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'40","reported_seq":374,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:33.736869+0000","last_change":"2024-03-21T08:28:35.682947+0000","last_active":"2024-03-21T08:31:33.736869+0000","last_peered":"2024-03-21T08:31:33.736869+0000","last_clean":"2024-03-21T08:31:33.736869+0000","last_became_active":"2024-03-21T08:28:35.682395+0000","last_became_peered":"2024-03-21T08:28:35.682395+0000","last_unstale":"2024-03-21T08:31:33.736869+0000","last_undegraded":"2024-03-21T08:31:33.736869+0000","last_fullsized":"2024-03-21T08:31:33.736869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:07:48.072988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67389,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":218,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.310604+0000","last_change":"2024-03-21T08:28:36.651140+0000","last_active":"2024-03-21T08:30:35.310604+0000","last_peered":"2024-03-21T08:30:35.310604+0000","last_clean":"2024-03-21T08:30:35.310604+0000","last_became_active":"2024-03-21T08:28:36.650551+0000","last_became_peered":"2024-03-21T08:28:36.650551+0000","last_unstale":"2024-03-21T08:30:35.310604+0000","last_undegraded":"2024-03-21T08:30:35.310604+0000","last_fullsized":"2024-03-21T08:30:35.310604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:53.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.667084+0000","last_change":"2024-03-21T08:28:40.691814+0000","last_active":"2024-03-21T08:30:06.667084+0000","last_peered":"2024-03-21T08:30:06.667084+0000","last_clean":"2024-03-21T08:30:06.667084+0000","last_became_active":"2024-03-21T08:28:40.691372+0000","last_became_peered":"2024-03-21T08:28:40.691372+0000","last_unstale":"2024-03-21T08:30:06.667084+0000","last_undegraded":"2024-03-21T08:30:06.667084+0000","last_fullsized":"2024-03-21T08:30:06.667084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:21:42.612037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645742+0000","last_change":"2024-03-21T08:28:35.687178+0000","last_active":"2024-03-21T08:30:06.645742+0000","last_peered":"2024-03-21T08:30:06.645742+0000","last_clean":"2024-03-21T08:30:06.645742+0000","last_became_active":"2024-03-21T08:28:35.686771+0000","last_became_peered":"2024-03-21T08:28:35.686771+0000","last_unstale":"2024-03-21T08:30:06.645742+0000","last_undegraded":"2024-03-21T08:30:06.645742+0000","last_fullsized":"2024-03-21T08:30:06.645742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:54.665866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317525+0000","last_change":"2024-03-21T08:28:36.662952+0000","last_active":"2024-03-21T08:30:35.317525+0000","last_peered":"2024-03-21T08:30:35.317525+0000","last_clean":"2024-03-21T08:30:35.317525+0000","last_became_active":"2024-03-21T08:28:36.662468+0000","last_became_peered":"2024-03-21T08:28:36.662468+0000","last_unstale":"2024-03-21T08:30:35.317525+0000","last_undegraded":"2024-03-21T08:30:35.317525+0000","last_fullsized":"2024-03-21T08:30:35.317525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:40.486998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656409+0000","last_change":"2024-03-21T08:28:40.693999+0000","last_active":"2024-03-21T08:30:06.656409+0000","last_peered":"2024-03-21T08:30:06.656409+0000","last_clean":"2024-03-21T08:30:06.656409+0000","last_became_active":"2024-03-21T08:28:40.693507+0000","last_became_peered":"2024-03-21T08:28:40.693507+0000","last_unstale":"2024-03-21T08:30:06.656409+0000","last_undegraded":"2024-03-21T08:30:06.656409+0000","last_fullsized":"2024-03-21T08:30:06.656409+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:14.159379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.668728+0000","last_change":"2024-03-21T08:28:35.677536+0000","last_active":"2024-03-21T08:30:06.668728+0000","last_peered":"2024-03-21T08:30:06.668728+0000","last_clean":"2024-03-21T08:30:06.668728+0000","last_became_active":"2024-03-21T08:28:35.677100+0000","last_became_peered":"2024-03-21T08:28:35.677100+0000","last_unstale":"2024-03-21T08:30:06.668728+0000","last_undegraded":"2024-03-21T08:30:06.668728+0000","last_fullsized":"2024-03-21T08:30:06.668728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:52:47.275869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.322854+0000","last_change":"2024-03-21T08:28:36.667190+0000","last_active":"2024-03-21T08:30:35.322854+0000","last_peered":"2024-03-21T08:30:35.322854+0000","last_clean":"2024-03-21T08:30:35.322854+0000","last_became_active":"2024-03-21T08:28:36.665336+0000","last_became_peered":"2024-03-21T08:28:36.665336+0000","last_unstale":"2024-03-21T08:30:35.322854+0000","last_undegraded":"2024-03-21T08:30:35.322854+0000","last_fullsized":"2024-03-21T08:30:35.322854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:21.888426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649761+0000","last_change":"2024-03-21T08:28:40.696582+0000","last_active":"2024-03-21T08:30:06.649761+0000","last_peered":"2024-03-21T08:30:06.649761+0000","last_clean":"2024-03-21T08:30:06.649761+0000","last_became_active":"2024-03-21T08:28:40.689583+0000","last_became_peered":"2024-03-21T08:28:40.689583+0000","last_unstale":"2024-03-21T08:30:06.649761+0000","last_undegraded":"2024-03-21T08:30:06.649761+0000","last_fullsized":"2024-03-21T08:30:06.649761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:57.329533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'4","reported_seq":185,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:33.736977+0000","last_change":"2024-03-21T08:28:35.675560+0000","last_active":"2024-03-21T08:31:33.736977+0000","last_peered":"2024-03-21T08:31:33.736977+0000","last_clean":"2024-03-21T08:31:33.736977+0000","last_became_active":"2024-03-21T08:28:35.673496+0000","last_became_peered":"2024-03-21T08:28:35.673496+0000","last_unstale":"2024-03-21T08:31:33.736977+0000","last_undegraded":"2024-03-21T08:31:33.736977+0000","last_fullsized":"2024-03-21T08:31:33.736977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:41:27.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.322318+0000","last_change":"2024-03-21T08:28:36.660869+0000","last_active":"2024-03-21T08:30:35.322318+0000","last_peered":"2024-03-21T08:30:35.322318+0000","last_clean":"2024-03-21T08:30:35.322318+0000","last_became_active":"2024-03-21T08:28:36.659291+0000","last_became_peered":"2024-03-21T08:28:36.659291+0000","last_unstale":"2024-03-21T08:30:35.322318+0000","last_undegraded":"2024-03-21T08:30:35.322318+0000","last_fullsized":"2024-03-21T08:30:35.322318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:06.534214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.643153+0000","last_change":"2024-03-21T08:28:40.701351+0000","last_active":"2024-03-21T08:30:06.643153+0000","last_peered":"2024-03-21T08:30:06.643153+0000","last_clean":"2024-03-21T08:30:06.643153+0000","last_became_active":"2024-03-21T08:28:40.700989+0000","last_became_peered":"2024-03-21T08:28:40.700989+0000","last_unstale":"2024-03-21T08:30:06.643153+0000","last_undegraded":"2024-03-21T08:30:06.643153+0000","last_fullsized":"2024-03-21T08:30:06.643153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:35.697013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650538+0000","last_change":"2024-03-21T08:28:35.685024+0000","last_active":"2024-03-21T08:30:06.650538+0000","last_peered":"2024-03-21T08:30:06.650538+0000","last_clean":"2024-03-21T08:30:06.650538+0000","last_became_active":"2024-03-21T08:28:35.684597+0000","last_became_peered":"2024-03-21T08:28:35.684597+0000","last_unstale":"2024-03-21T08:30:06.650538+0000","last_undegraded":"2024-03-21T08:30:06.650538+0000","last_fullsized":"2024-03-21T08:30:06.650538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:09:46.262782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332203+0000","last_change":"2024-03-21T08:28:36.664688+0000","last_active":"2024-03-21T08:30:35.332203+0000","last_peered":"2024-03-21T08:30:35.332203+0000","last_clean":"2024-03-21T08:30:35.332203+0000","last_became_active":"2024-03-21T08:28:36.663990+0000","last_became_peered":"2024-03-21T08:28:36.663990+0000","last_unstale":"2024-03-21T08:30:35.332203+0000","last_undegraded":"2024-03-21T08:30:35.332203+0000","last_fullsized":"2024-03-21T08:30:35.332203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:39.656290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664426+0000","last_change":"2024-03-21T08:28:40.695269+0000","last_active":"2024-03-21T08:30:06.664426+0000","last_peered":"2024-03-21T08:30:06.664426+0000","last_clean":"2024-03-21T08:30:06.664426+0000","last_became_active":"2024-03-21T08:28:40.694820+0000","last_became_peered":"2024-03-21T08:28:40.694820+0000","last_unstale":"2024-03-21T08:30:06.664426+0000","last_undegraded":"2024-03-21T08:30:06.664426+0000","last_fullsized":"2024-03-21T08:30:06.664426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:47:25.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.673554+0000","last_change":"2024-03-21T08:30:06.672662+0000","last_active":"2024-03-21T08:30:06.673554+0000","last_peered":"2024-03-21T08:30:06.673554+0000","last_clean":"2024-03-21T08:30:06.673554+0000","last_became_active":"2024-03-21T08:30:06.660161+0000","last_became_peered":"2024-03-21T08:30:06.660161+0000","last_unstale":"2024-03-21T08:30:06.673554+0000","last_undegraded":"2024-03-21T08:30:06.673554+0000","last_fullsized":"2024-03-21T08:30:06.673554+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:41.163709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,4],"acting":[8,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315069+0000","last_change":"2024-03-21T08:28:36.667605+0000","last_active":"2024-03-21T08:30:35.315069+0000","last_peered":"2024-03-21T08:30:35.315069+0000","last_clean":"2024-03-21T08:30:35.315069+0000","last_became_active":"2024-03-21T08:28:36.667300+0000","last_became_peered":"2024-03-21T08:28:36.667300+0000","last_unstale":"2024-03-21T08:30:35.315069+0000","last_undegraded":"2024-03-21T08:30:35.315069+0000","last_fullsized":"2024-03-21T08:30:35.315069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:49.268935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652659+0000","last_change":"2024-03-21T08:28:40.884964+0000","last_active":"2024-03-21T08:30:06.652659+0000","last_peered":"2024-03-21T08:30:06.652659+0000","last_clean":"2024-03-21T08:30:06.652659+0000","last_became_active":"2024-03-21T08:28:40.883445+0000","last_became_peered":"2024-03-21T08:28:40.883445+0000","last_unstale":"2024-03-21T08:30:06.652659+0000","last_undegraded":"2024-03-21T08:30:06.652659+0000","last_fullsized":"2024-03-21T08:30:06.652659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:36:54.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647966+0000","last_change":"2024-03-21T08:28:35.682581+0000","last_active":"2024-03-21T08:30:06.647966+0000","last_peered":"2024-03-21T08:30:06.647966+0000","last_clean":"2024-03-21T08:30:06.647966+0000","last_became_active":"2024-03-21T08:28:35.682150+0000","last_became_peered":"2024-03-21T08:28:35.682150+0000","last_unstale":"2024-03-21T08:30:06.647966+0000","last_undegraded":"2024-03-21T08:30:06.647966+0000","last_fullsized":"2024-03-21T08:30:06.647966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:08.729710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316730+0000","last_change":"2024-03-21T08:28:36.666632+0000","last_active":"2024-03-21T08:30:35.316730+0000","last_peered":"2024-03-21T08:30:35.316730+0000","last_clean":"2024-03-21T08:30:35.316730+0000","last_became_active":"2024-03-21T08:28:36.666305+0000","last_became_peered":"2024-03-21T08:28:36.666305+0000","last_unstale":"2024-03-21T08:30:35.316730+0000","last_undegraded":"2024-03-21T08:30:35.316730+0000","last_fullsized":"2024-03-21T08:30:35.316730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:15.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642891+0000","last_change":"2024-03-21T08:28:40.700382+0000","last_active":"2024-03-21T08:30:06.642891+0000","last_peered":"2024-03-21T08:30:06.642891+0000","last_clean":"2024-03-21T08:30:06.642891+0000","last_became_active":"2024-03-21T08:28:40.699762+0000","last_became_peered":"2024-03-21T08:28:40.699762+0000","last_unstale":"2024-03-21T08:30:06.642891+0000","last_undegraded":"2024-03-21T08:30:06.642891+0000","last_fullsized":"2024-03-21T08:30:06.642891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:53.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657107+0000","last_change":"2024-03-21T08:28:35.680310+0000","last_active":"2024-03-21T08:30:06.657107+0000","last_peered":"2024-03-21T08:30:06.657107+0000","last_clean":"2024-03-21T08:30:06.657107+0000","last_became_active":"2024-03-21T08:28:35.678634+0000","last_became_peered":"2024-03-21T08:28:35.678634+0000","last_unstale":"2024-03-21T08:30:06.657107+0000","last_undegraded":"2024-03-21T08:30:06.657107+0000","last_fullsized":"2024-03-21T08:30:06.657107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:56:42.872521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319411+0000","last_change":"2024-03-21T08:28:36.669527+0000","last_active":"2024-03-21T08:30:35.319411+0000","last_peered":"2024-03-21T08:30:35.319411+0000","last_clean":"2024-03-21T08:30:35.319411+0000","last_became_active":"2024-03-21T08:28:36.669196+0000","last_became_peered":"2024-03-21T08:28:36.669196+0000","last_unstale":"2024-03-21T08:30:35.319411+0000","last_undegraded":"2024-03-21T08:30:35.319411+0000","last_fullsized":"2024-03-21T08:30:35.319411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:07:34.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644070+0000","last_change":"2024-03-21T08:28:40.882949+0000","last_active":"2024-03-21T08:30:06.644070+0000","last_peered":"2024-03-21T08:30:06.644070+0000","last_clean":"2024-03-21T08:30:06.644070+0000","last_became_active":"2024-03-21T08:28:40.882432+0000","last_became_peered":"2024-03-21T08:28:40.882432+0000","last_unstale":"2024-03-21T08:30:06.644070+0000","last_undegraded":"2024-03-21T08:30:06.644070+0000","last_fullsized":"2024-03-21T08:30:06.644070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:51.171365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.321838+0000","last_change":"2024-03-21T08:28:36.665164+0000","last_active":"2024-03-21T08:30:35.321838+0000","last_peered":"2024-03-21T08:30:35.321838+0000","last_clean":"2024-03-21T08:30:35.321838+0000","last_became_active":"2024-03-21T08:28:36.664794+0000","last_became_peered":"2024-03-21T08:28:36.664794+0000","last_unstale":"2024-03-21T08:30:35.321838+0000","last_undegraded":"2024-03-21T08:30:35.321838+0000","last_fullsized":"2024-03-21T08:30:35.321838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:25:39.355633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'5","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:32.733229+0000","last_change":"2024-03-21T08:28:35.671754+0000","last_active":"2024-03-21T08:31:32.733229+0000","last_peered":"2024-03-21T08:31:32.733229+0000","last_clean":"2024-03-21T08:31:32.733229+0000","last_became_active":"2024-03-21T08:28:35.671363+0000","last_became_peered":"2024-03-21T08:28:35.671363+0000","last_unstale":"2024-03-21T08:31:32.733229+0000","last_undegraded":"2024-03-21T08:31:32.733229+0000","last_fullsized":"2024-03-21T08:31:32.733229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:22:11.147325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657862+0000","last_change":"2024-03-21T08:28:40.689038+0000","last_active":"2024-03-21T08:30:06.657862+0000","last_peered":"2024-03-21T08:30:06.657862+0000","last_clean":"2024-03-21T08:30:06.657862+0000","last_became_active":"2024-03-21T08:28:40.688538+0000","last_became_peered":"2024-03-21T08:28:40.688538+0000","last_unstale":"2024-03-21T08:30:06.657862+0000","last_undegraded":"2024-03-21T08:30:06.657862+0000","last_fullsized":"2024-03-21T08:30:06.657862+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:14.833594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331670+0000","last_change":"2024-03-21T08:28:36.667725+0000","last_active":"2024-03-21T08:30:35.331670+0000","last_peered":"2024-03-21T08:30:35.331670+0000","last_clean":"2024-03-21T08:30:35.331670+0000","last_became_active":"2024-03-21T08:28:36.667303+0000","last_became_peered":"2024-03-21T08:28:36.667303+0000","last_unstale":"2024-03-21T08:30:35.331670+0000","last_undegraded":"2024-03-21T08:30:35.331670+0000","last_fullsized":"2024-03-21T08:30:35.331670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:32:18.990943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664955+0000","last_change":"2024-03-21T08:28:35.673497+0000","last_active":"2024-03-21T08:30:06.664955+0000","last_peered":"2024-03-21T08:30:06.664955+0000","last_clean":"2024-03-21T08:30:06.664955+0000","last_became_active":"2024-03-21T08:28:35.673045+0000","last_became_peered":"2024-03-21T08:28:35.673045+0000","last_unstale":"2024-03-21T08:30:06.664955+0000","last_undegraded":"2024-03-21T08:30:06.664955+0000","last_fullsized":"2024-03-21T08:30:06.664955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:23.972672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":370,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653387+0000","last_change":"2024-03-21T08:26:58.139094+0000","last_active":"2024-03-21T08:30:06.653387+0000","last_peered":"2024-03-21T08:30:06.653387+0000","last_clean":"2024-03-21T08:30:06.653387+0000","last_became_active":"2024-03-21T08:26:58.137648+0000","last_became_peered":"2024-03-21T08:26:58.137648+0000","last_unstale":"2024-03-21T08:30:06.653387+0000","last_undegraded":"2024-03-21T08:30:06.653387+0000","last_fullsized":"2024-03-21T08:30:06.653387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_clean_scrub_stamp":"2024-03-21T08:22:17.977314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:57.950111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665774+0000","last_change":"2024-03-21T08:28:40.887865+0000","last_active":"2024-03-21T08:30:06.665774+0000","last_peered":"2024-03-21T08:30:06.665774+0000","last_clean":"2024-03-21T08:30:06.665774+0000","last_became_active":"2024-03-21T08:28:40.885538+0000","last_became_peered":"2024-03-21T08:28:40.885538+0000","last_unstale":"2024-03-21T08:30:06.665774+0000","last_undegraded":"2024-03-21T08:30:06.665774+0000","last_fullsized":"2024-03-21T08:30:06.665774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:26.532005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.309634+0000","last_change":"2024-03-21T08:28:36.668600+0000","last_active":"2024-03-21T08:30:35.309634+0000","last_peered":"2024-03-21T08:30:35.309634+0000","last_clean":"2024-03-21T08:30:35.309634+0000","last_became_active":"2024-03-21T08:28:36.668188+0000","last_became_peered":"2024-03-21T08:28:36.668188+0000","last_unstale":"2024-03-21T08:30:35.309634+0000","last_undegraded":"2024-03-21T08:30:35.309634+0000","last_fullsized":"2024-03-21T08:30:35.309634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:13.536003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.674753+0000","last_change":"2024-03-21T08:30:06.673425+0000","last_active":"2024-03-21T08:30:06.674753+0000","last_peered":"2024-03-21T08:30:06.674753+0000","last_clean":"2024-03-21T08:30:06.674753+0000","last_became_active":"2024-03-21T08:30:06.666858+0000","last_became_peered":"2024-03-21T08:30:06.666858+0000","last_unstale":"2024-03-21T08:30:06.674753+0000","last_undegraded":"2024-03-21T08:30:06.674753+0000","last_fullsized":"2024-03-21T08:30:06.674753+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:37.159978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655579+0000","last_change":"2024-03-21T08:29:07.032622+0000","last_active":"2024-03-21T08:30:06.655579+0000","last_peered":"2024-03-21T08:30:06.655579+0000","last_clean":"2024-03-21T08:30:06.655579+0000","last_became_active":"2024-03-21T08:29:07.030884+0000","last_became_peered":"2024-03-21T08:29:07.030884+0000","last_unstale":"2024-03-21T08:30:06.655579+0000","last_undegraded":"2024-03-21T08:30:06.655579+0000","last_fullsized":"2024-03-21T08:30:06.655579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:47.615776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654098+0000","last_change":"2024-03-21T08:28:35.672948+0000","last_active":"2024-03-21T08:30:06.654098+0000","last_peered":"2024-03-21T08:30:06.654098+0000","last_clean":"2024-03-21T08:30:06.654098+0000","last_became_active":"2024-03-21T08:28:35.671576+0000","last_became_peered":"2024-03-21T08:28:35.671576+0000","last_unstale":"2024-03-21T08:30:06.654098+0000","last_undegraded":"2024-03-21T08:30:06.654098+0000","last_fullsized":"2024-03-21T08:30:06.654098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:36:25.620392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.308174+0000","last_change":"2024-03-21T08:28:36.671173+0000","last_active":"2024-03-21T08:30:35.308174+0000","last_peered":"2024-03-21T08:30:35.308174+0000","last_clean":"2024-03-21T08:30:35.308174+0000","last_became_active":"2024-03-21T08:28:36.666264+0000","last_became_peered":"2024-03-21T08:28:36.666264+0000","last_unstale":"2024-03-21T08:30:35.308174+0000","last_undegraded":"2024-03-21T08:30:35.308174+0000","last_fullsized":"2024-03-21T08:30:35.308174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:35.081487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646351+0000","last_change":"2024-03-21T08:28:40.883801+0000","last_active":"2024-03-21T08:30:06.646351+0000","last_peered":"2024-03-21T08:30:06.646351+0000","last_clean":"2024-03-21T08:30:06.646351+0000","last_became_active":"2024-03-21T08:28:40.882556+0000","last_became_peered":"2024-03-21T08:28:40.882556+0000","last_unstale":"2024-03-21T08:30:06.646351+0000","last_undegraded":"2024-03-21T08:30:06.646351+0000","last_fullsized":"2024-03-21T08:30:06.646351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:24.211625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'10","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:49.542985+0000","last_change":"2024-03-21T08:28:35.668657+0000","last_active":"2024-03-21T08:30:49.542985+0000","last_peered":"2024-03-21T08:30:49.542985+0000","last_clean":"2024-03-21T08:30:49.542985+0000","last_became_active":"2024-03-21T08:28:35.668284+0000","last_became_peered":"2024-03-21T08:28:35.668284+0000","last_unstale":"2024-03-21T08:30:49.542985+0000","last_undegraded":"2024-03-21T08:30:49.542985+0000","last_fullsized":"2024-03-21T08:30:49.542985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:26:07.791388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316596+0000","last_change":"2024-03-21T08:28:36.652383+0000","last_active":"2024-03-21T08:30:35.316596+0000","last_peered":"2024-03-21T08:30:35.316596+0000","last_clean":"2024-03-21T08:30:35.316596+0000","last_became_active":"2024-03-21T08:28:36.652046+0000","last_became_peered":"2024-03-21T08:28:36.652046+0000","last_unstale":"2024-03-21T08:30:35.316596+0000","last_undegraded":"2024-03-21T08:30:35.316596+0000","last_fullsized":"2024-03-21T08:30:35.316596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:11:08.699391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644497+0000","last_change":"2024-03-21T08:28:40.686122+0000","last_active":"2024-03-21T08:30:06.644497+0000","last_peered":"2024-03-21T08:30:06.644497+0000","last_clean":"2024-03-21T08:30:06.644497+0000","last_became_active":"2024-03-21T08:28:40.685736+0000","last_became_peered":"2024-03-21T08:28:40.685736+0000","last_unstale":"2024-03-21T08:30:06.644497+0000","last_undegraded":"2024-03-21T08:30:06.644497+0000","last_fullsized":"2024-03-21T08:30:06.644497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:52.941896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665257+0000","last_change":"2024-03-21T08:28:35.675197+0000","last_active":"2024-03-21T08:30:06.665257+0000","last_peered":"2024-03-21T08:30:06.665257+0000","last_clean":"2024-03-21T08:30:06.665257+0000","last_became_active":"2024-03-21T08:28:35.674688+0000","last_became_peered":"2024-03-21T08:28:35.674688+0000","last_unstale":"2024-03-21T08:30:06.665257+0000","last_undegraded":"2024-03-21T08:30:06.665257+0000","last_fullsized":"2024-03-21T08:30:06.665257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:01:11.489294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328753+0000","last_change":"2024-03-21T08:28:36.668160+0000","last_active":"2024-03-21T08:30:35.328753+0000","last_peered":"2024-03-21T08:30:35.328753+0000","last_clean":"2024-03-21T08:30:35.328753+0000","last_became_active":"2024-03-21T08:28:36.665152+0000","last_became_peered":"2024-03-21T08:28:36.665152+0000","last_unstale":"2024-03-21T08:30:35.328753+0000","last_undegraded":"2024-03-21T08:30:35.328753+0000","last_fullsized":"2024-03-21T08:30:35.328753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:43:41.803822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663608+0000","last_change":"2024-03-21T08:28:40.884365+0000","last_active":"2024-03-21T08:30:06.663608+0000","last_peered":"2024-03-21T08:30:06.663608+0000","last_clean":"2024-03-21T08:30:06.663608+0000","last_became_active":"2024-03-21T08:28:40.883654+0000","last_became_peered":"2024-03-21T08:28:40.883654+0000","last_unstale":"2024-03-21T08:30:06.663608+0000","last_undegraded":"2024-03-21T08:30:06.663608+0000","last_fullsized":"2024-03-21T08:30:06.663608+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:46:43.888121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672790+0000","last_change":"2024-03-21T08:30:06.672111+0000","last_active":"2024-03-21T08:30:06.672790+0000","last_peered":"2024-03-21T08:30:06.672790+0000","last_clean":"2024-03-21T08:30:06.672790+0000","last_became_active":"2024-03-21T08:30:06.665627+0000","last_became_peered":"2024-03-21T08:30:06.665627+0000","last_unstale":"2024-03-21T08:30:06.672790+0000","last_undegraded":"2024-03-21T08:30:06.672790+0000","last_fullsized":"2024-03-21T08:30:06.672790+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:05.512276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.307537+0000","last_change":"2024-03-21T08:28:36.653797+0000","last_active":"2024-03-21T08:30:35.307537+0000","last_peered":"2024-03-21T08:30:35.307537+0000","last_clean":"2024-03-21T08:30:35.307537+0000","last_became_active":"2024-03-21T08:28:36.651872+0000","last_became_peered":"2024-03-21T08:28:36.651872+0000","last_unstale":"2024-03-21T08:30:35.307537+0000","last_undegraded":"2024-03-21T08:30:35.307537+0000","last_fullsized":"2024-03-21T08:30:35.307537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:09.467573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649747+0000","last_change":"2024-03-21T08:28:40.692996+0000","last_active":"2024-03-21T08:30:06.649747+0000","last_peered":"2024-03-21T08:30:06.649747+0000","last_clean":"2024-03-21T08:30:06.649747+0000","last_became_active":"2024-03-21T08:28:40.687598+0000","last_became_peered":"2024-03-21T08:28:40.687598+0000","last_unstale":"2024-03-21T08:30:06.649747+0000","last_undegraded":"2024-03-21T08:30:06.649747+0000","last_fullsized":"2024-03-21T08:30:06.649747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:09.557272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645955+0000","last_change":"2024-03-21T08:28:35.672567+0000","last_active":"2024-03-21T08:30:06.645955+0000","last_peered":"2024-03-21T08:30:06.645955+0000","last_clean":"2024-03-21T08:30:06.645955+0000","last_became_active":"2024-03-21T08:28:35.671849+0000","last_became_peered":"2024-03-21T08:28:35.671849+0000","last_unstale":"2024-03-21T08:30:06.645955+0000","last_undegraded":"2024-03-21T08:30:06.645955+0000","last_fullsized":"2024-03-21T08:30:06.645955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:13.050227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.320698+0000","last_change":"2024-03-21T08:28:36.664110+0000","last_active":"2024-03-21T08:30:35.320698+0000","last_peered":"2024-03-21T08:30:35.320698+0000","last_clean":"2024-03-21T08:30:35.320698+0000","last_became_active":"2024-03-21T08:28:36.663297+0000","last_became_peered":"2024-03-21T08:28:36.663297+0000","last_unstale":"2024-03-21T08:30:35.320698+0000","last_undegraded":"2024-03-21T08:30:35.320698+0000","last_fullsized":"2024-03-21T08:30:35.320698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:55.450399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663838+0000","last_change":"2024-03-21T08:28:40.682371+0000","last_active":"2024-03-21T08:30:06.663838+0000","last_peered":"2024-03-21T08:30:06.663838+0000","last_clean":"2024-03-21T08:30:06.663838+0000","last_became_active":"2024-03-21T08:28:40.681786+0000","last_became_peered":"2024-03-21T08:28:40.681786+0000","last_unstale":"2024-03-21T08:30:06.663838+0000","last_undegraded":"2024-03-21T08:30:06.663838+0000","last_fullsized":"2024-03-21T08:30:06.663838+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:15:38.130807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"96'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:46.232715+0000","last_change":"2024-03-21T08:28:35.685312+0000","last_active":"2024-03-21T08:30:46.232715+0000","last_peered":"2024-03-21T08:30:46.232715+0000","last_clean":"2024-03-21T08:30:46.232715+0000","last_became_active":"2024-03-21T08:28:35.684327+0000","last_became_peered":"2024-03-21T08:28:35.684327+0000","last_unstale":"2024-03-21T08:30:46.232715+0000","last_undegraded":"2024-03-21T08:30:46.232715+0000","last_fullsized":"2024-03-21T08:30:46.232715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:12.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330747+0000","last_change":"2024-03-21T08:28:36.671186+0000","last_active":"2024-03-21T08:30:35.330747+0000","last_peered":"2024-03-21T08:30:35.330747+0000","last_clean":"2024-03-21T08:30:35.330747+0000","last_became_active":"2024-03-21T08:28:36.662726+0000","last_became_peered":"2024-03-21T08:28:36.662726+0000","last_unstale":"2024-03-21T08:30:35.330747+0000","last_undegraded":"2024-03-21T08:30:35.330747+0000","last_fullsized":"2024-03-21T08:30:35.330747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:29.129014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663764+0000","last_change":"2024-03-21T08:28:40.885203+0000","last_active":"2024-03-21T08:30:06.663764+0000","last_peered":"2024-03-21T08:30:06.663764+0000","last_clean":"2024-03-21T08:30:06.663764+0000","last_became_active":"2024-03-21T08:28:40.883810+0000","last_became_peered":"2024-03-21T08:28:40.883810+0000","last_unstale":"2024-03-21T08:30:06.663764+0000","last_undegraded":"2024-03-21T08:30:06.663764+0000","last_fullsized":"2024-03-21T08:30:06.663764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:12:58.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651247+0000","last_change":"2024-03-21T08:28:35.690494+0000","last_active":"2024-03-21T08:30:06.651247+0000","last_peered":"2024-03-21T08:30:06.651247+0000","last_clean":"2024-03-21T08:30:06.651247+0000","last_became_active":"2024-03-21T08:28:35.690071+0000","last_became_peered":"2024-03-21T08:28:35.690071+0000","last_unstale":"2024-03-21T08:30:06.651247+0000","last_undegraded":"2024-03-21T08:30:06.651247+0000","last_fullsized":"2024-03-21T08:30:06.651247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:04:46.713006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333944+0000","last_change":"2024-03-21T08:28:36.665135+0000","last_active":"2024-03-21T08:30:35.333944+0000","last_peered":"2024-03-21T08:30:35.333944+0000","last_clean":"2024-03-21T08:30:35.333944+0000","last_became_active":"2024-03-21T08:28:36.663700+0000","last_became_peered":"2024-03-21T08:28:36.663700+0000","last_unstale":"2024-03-21T08:30:35.333944+0000","last_undegraded":"2024-03-21T08:30:35.333944+0000","last_fullsized":"2024-03-21T08:30:35.333944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:50:00.604958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649998+0000","last_change":"2024-03-21T08:28:40.696076+0000","last_active":"2024-03-21T08:30:06.649998+0000","last_peered":"2024-03-21T08:30:06.649998+0000","last_clean":"2024-03-21T08:30:06.649998+0000","last_became_active":"2024-03-21T08:28:40.695556+0000","last_became_peered":"2024-03-21T08:28:40.695556+0000","last_unstale":"2024-03-21T08:30:06.649998+0000","last_undegraded":"2024-03-21T08:30:06.649998+0000","last_fullsized":"2024-03-21T08:30:06.649998+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:42.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646454+0000","last_change":"2024-03-21T08:28:35.685545+0000","last_active":"2024-03-21T08:30:06.646454+0000","last_peered":"2024-03-21T08:30:06.646454+0000","last_clean":"2024-03-21T08:30:06.646454+0000","last_became_active":"2024-03-21T08:28:35.684795+0000","last_became_peered":"2024-03-21T08:28:35.684795+0000","last_unstale":"2024-03-21T08:30:06.646454+0000","last_undegraded":"2024-03-21T08:30:06.646454+0000","last_fullsized":"2024-03-21T08:30:06.646454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:40.308014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.334201+0000","last_change":"2024-03-21T08:28:36.656276+0000","last_active":"2024-03-21T08:30:35.334201+0000","last_peered":"2024-03-21T08:30:35.334201+0000","last_clean":"2024-03-21T08:30:35.334201+0000","last_became_active":"2024-03-21T08:28:36.655871+0000","last_became_peered":"2024-03-21T08:28:36.655871+0000","last_unstale":"2024-03-21T08:30:35.334201+0000","last_undegraded":"2024-03-21T08:30:35.334201+0000","last_fullsized":"2024-03-21T08:30:35.334201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:22.362165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651972+0000","last_change":"2024-03-21T08:29:06.783155+0000","last_active":"2024-03-21T08:30:06.651972+0000","last_peered":"2024-03-21T08:30:06.651972+0000","last_clean":"2024-03-21T08:30:06.651972+0000","last_became_active":"2024-03-21T08:29:06.782546+0000","last_became_peered":"2024-03-21T08:29:06.782546+0000","last_unstale":"2024-03-21T08:30:06.651972+0000","last_undegraded":"2024-03-21T08:30:06.651972+0000","last_fullsized":"2024-03-21T08:30:06.651972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:16.489888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666681+0000","last_change":"2024-03-21T08:28:35.672783+0000","last_active":"2024-03-21T08:30:06.666681+0000","last_peered":"2024-03-21T08:30:06.666681+0000","last_clean":"2024-03-21T08:30:06.666681+0000","last_became_active":"2024-03-21T08:28:35.672161+0000","last_became_peered":"2024-03-21T08:28:35.672161+0000","last_unstale":"2024-03-21T08:30:06.666681+0000","last_undegraded":"2024-03-21T08:30:06.666681+0000","last_fullsized":"2024-03-21T08:30:06.666681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:37.180625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315686+0000","last_change":"2024-03-21T08:28:36.659447+0000","last_active":"2024-03-21T08:30:35.315686+0000","last_peered":"2024-03-21T08:30:35.315686+0000","last_clean":"2024-03-21T08:30:35.315686+0000","last_became_active":"2024-03-21T08:28:36.657773+0000","last_became_peered":"2024-03-21T08:28:36.657773+0000","last_unstale":"2024-03-21T08:30:35.315686+0000","last_undegraded":"2024-03-21T08:30:35.315686+0000","last_fullsized":"2024-03-21T08:30:35.315686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:05.324498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664330+0000","last_change":"2024-03-21T08:28:40.679806+0000","last_active":"2024-03-21T08:30:06.664330+0000","last_peered":"2024-03-21T08:30:06.664330+0000","last_clean":"2024-03-21T08:30:06.664330+0000","last_became_active":"2024-03-21T08:28:40.679329+0000","last_became_peered":"2024-03-21T08:28:40.679329+0000","last_unstale":"2024-03-21T08:30:06.664330+0000","last_undegraded":"2024-03-21T08:30:06.664330+0000","last_fullsized":"2024-03-21T08:30:06.664330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:30:58.297581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.675639+0000","last_change":"2024-03-21T08:30:06.675069+0000","last_active":"2024-03-21T08:30:06.675639+0000","last_peered":"2024-03-21T08:30:06.675639+0000","last_clean":"2024-03-21T08:30:06.675639+0000","last_became_active":"2024-03-21T08:30:06.665699+0000","last_became_peered":"2024-03-21T08:30:06.665699+0000","last_unstale":"2024-03-21T08:30:06.675639+0000","last_undegraded":"2024-03-21T08:30:06.675639+0000","last_fullsized":"2024-03-21T08:30:06.675639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:30:07.336423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319014+0000","last_change":"2024-03-21T08:28:36.649194+0000","last_active":"2024-03-21T08:30:35.319014+0000","last_peered":"2024-03-21T08:30:35.319014+0000","last_clean":"2024-03-21T08:30:35.319014+0000","last_became_active":"2024-03-21T08:28:36.648663+0000","last_became_peered":"2024-03-21T08:28:36.648663+0000","last_unstale":"2024-03-21T08:30:35.319014+0000","last_undegraded":"2024-03-21T08:30:35.319014+0000","last_fullsized":"2024-03-21T08:30:35.319014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:16.296886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313728+0000","last_change":"2024-03-21T08:28:36.664473+0000","last_active":"2024-03-21T08:30:35.313728+0000","last_peered":"2024-03-21T08:30:35.313728+0000","last_clean":"2024-03-21T08:30:35.313728+0000","last_became_active":"2024-03-21T08:28:36.663785+0000","last_became_peered":"2024-03-21T08:28:36.663785+0000","last_unstale":"2024-03-21T08:30:35.313728+0000","last_undegraded":"2024-03-21T08:30:35.313728+0000","last_fullsized":"2024-03-21T08:30:35.313728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:00.409224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.309044+0000","last_change":"2024-03-21T08:28:36.668372+0000","last_active":"2024-03-21T08:30:35.309044+0000","last_peered":"2024-03-21T08:30:35.309044+0000","last_clean":"2024-03-21T08:30:35.309044+0000","last_became_active":"2024-03-21T08:28:36.666227+0000","last_became_peered":"2024-03-21T08:28:36.666227+0000","last_unstale":"2024-03-21T08:30:35.309044+0000","last_undegraded":"2024-03-21T08:30:35.309044+0000","last_fullsized":"2024-03-21T08:30:35.309044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:40:56.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317658+0000","last_change":"2024-03-21T08:28:36.670770+0000","last_active":"2024-03-21T08:30:35.317658+0000","last_peered":"2024-03-21T08:30:35.317658+0000","last_clean":"2024-03-21T08:30:35.317658+0000","last_became_active":"2024-03-21T08:28:36.670467+0000","last_became_peered":"2024-03-21T08:28:36.670467+0000","last_unstale":"2024-03-21T08:30:35.317658+0000","last_undegraded":"2024-03-21T08:30:35.317658+0000","last_fullsized":"2024-03-21T08:30:35.317658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:48.075189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329527+0000","last_change":"2024-03-21T08:28:36.669395+0000","last_active":"2024-03-21T08:30:35.329527+0000","last_peered":"2024-03-21T08:30:35.329527+0000","last_clean":"2024-03-21T08:30:35.329527+0000","last_became_active":"2024-03-21T08:28:36.668788+0000","last_became_peered":"2024-03-21T08:28:36.668788+0000","last_unstale":"2024-03-21T08:30:35.329527+0000","last_undegraded":"2024-03-21T08:30:35.329527+0000","last_fullsized":"2024-03-21T08:30:35.329527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:03.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.326209+0000","last_change":"2024-03-21T08:30:06.655160+0000","last_active":"2024-03-21T08:30:35.326209+0000","last_peered":"2024-03-21T08:30:35.326209+0000","last_clean":"2024-03-21T08:30:35.326209+0000","last_became_active":"2024-03-21T08:30:06.654744+0000","last_became_peered":"2024-03-21T08:30:06.654744+0000","last_unstale":"2024-03-21T08:30:35.326209+0000","last_undegraded":"2024-03-21T08:30:35.326209+0000","last_fullsized":"2024-03-21T08:30:35.326209+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:19:54.690562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.308832+0000","last_change":"2024-03-21T08:28:36.666836+0000","last_active":"2024-03-21T08:30:35.308832+0000","last_peered":"2024-03-21T08:30:35.308832+0000","last_clean":"2024-03-21T08:30:35.308832+0000","last_became_active":"2024-03-21T08:28:36.666538+0000","last_became_peered":"2024-03-21T08:28:36.666538+0000","last_unstale":"2024-03-21T08:30:35.308832+0000","last_undegraded":"2024-03-21T08:30:35.308832+0000","last_fullsized":"2024-03-21T08:30:35.308832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:43.353269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.325089+0000","last_change":"2024-03-21T08:28:36.667823+0000","last_active":"2024-03-21T08:30:35.325089+0000","last_peered":"2024-03-21T08:30:35.325089+0000","last_clean":"2024-03-21T08:30:35.325089+0000","last_became_active":"2024-03-21T08:28:36.666096+0000","last_became_peered":"2024-03-21T08:28:36.666096+0000","last_unstale":"2024-03-21T08:30:35.325089+0000","last_undegraded":"2024-03-21T08:30:35.325089+0000","last_fullsized":"2024-03-21T08:30:35.325089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:58.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312965+0000","last_change":"2024-03-21T08:28:36.672164+0000","last_active":"2024-03-21T08:30:35.312965+0000","last_peered":"2024-03-21T08:30:35.312965+0000","last_clean":"2024-03-21T08:30:35.312965+0000","last_became_active":"2024-03-21T08:28:36.671812+0000","last_became_peered":"2024-03-21T08:28:36.671812+0000","last_unstale":"2024-03-21T08:30:35.312965+0000","last_undegraded":"2024-03-21T08:30:35.312965+0000","last_fullsized":"2024-03-21T08:30:35.312965+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:24:19.687108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316596+0000","last_change":"2024-03-21T08:28:36.675056+0000","last_active":"2024-03-21T08:30:35.316596+0000","last_peered":"2024-03-21T08:30:35.316596+0000","last_clean":"2024-03-21T08:30:35.316596+0000","last_became_active":"2024-03-21T08:28:36.674586+0000","last_became_peered":"2024-03-21T08:28:36.674586+0000","last_unstale":"2024-03-21T08:30:35.316596+0000","last_undegraded":"2024-03-21T08:30:35.316596+0000","last_fullsized":"2024-03-21T08:30:35.316596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:04:51.207066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313977+0000","last_change":"2024-03-21T08:28:36.666370+0000","last_active":"2024-03-21T08:30:35.313977+0000","last_peered":"2024-03-21T08:30:35.313977+0000","last_clean":"2024-03-21T08:30:35.313977+0000","last_became_active":"2024-03-21T08:28:36.665950+0000","last_became_peered":"2024-03-21T08:28:36.665950+0000","last_unstale":"2024-03-21T08:30:35.313977+0000","last_undegraded":"2024-03-21T08:30:35.313977+0000","last_fullsized":"2024-03-21T08:30:35.313977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:49:31.945890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330531+0000","last_change":"2024-03-21T08:29:07.032862+0000","last_active":"2024-03-21T08:30:35.330531+0000","last_peered":"2024-03-21T08:30:35.330531+0000","last_clean":"2024-03-21T08:30:35.330531+0000","last_became_active":"2024-03-21T08:29:07.031784+0000","last_became_peered":"2024-03-21T08:29:07.031784+0000","last_unstale":"2024-03-21T08:30:35.330531+0000","last_undegraded":"2024-03-21T08:30:35.330531+0000","last_fullsized":"2024-03-21T08:30:35.330531+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:34.037495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330945+0000","last_change":"2024-03-21T08:28:36.660812+0000","last_active":"2024-03-21T08:30:35.330945+0000","last_peered":"2024-03-21T08:30:35.330945+0000","last_clean":"2024-03-21T08:30:35.330945+0000","last_became_active":"2024-03-21T08:28:36.660044+0000","last_became_peered":"2024-03-21T08:28:36.660044+0000","last_unstale":"2024-03-21T08:30:35.330945+0000","last_undegraded":"2024-03-21T08:30:35.330945+0000","last_fullsized":"2024-03-21T08:30:35.330945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:31.598678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316333+0000","last_change":"2024-03-21T08:28:36.669441+0000","last_active":"2024-03-21T08:30:35.316333+0000","last_peered":"2024-03-21T08:30:35.316333+0000","last_clean":"2024-03-21T08:30:35.316333+0000","last_became_active":"2024-03-21T08:28:36.668668+0000","last_became_peered":"2024-03-21T08:28:36.668668+0000","last_unstale":"2024-03-21T08:30:35.316333+0000","last_undegraded":"2024-03-21T08:30:35.316333+0000","last_fullsized":"2024-03-21T08:30:35.316333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:04.828297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72517,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":904,"num_read_kb":530,"num_write":181,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":217551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285621,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":534350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547258,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1189500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710786,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":538682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972425,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":533924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266769,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712316,"statfs":{"total":95995035648,"available":95961411584,"internally_reserved":0,"allocated":1699840,"data_stored":1131542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":529784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":597643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":531150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":595122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":529956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":534905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1126615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:31:36.489 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-21T08:31:36.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T08:31:36.616 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:36 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3493495700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:36.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:36 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3493495700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:36.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:36 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3493495700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:36.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 -- 172.21.15.3:0/3699662356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 msgr2=0x7efd30103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 --2- 172.21.15.3:0/3699662356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd30103be0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efd20009900 tx=0x7efd2002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:36.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 -- 172.21.15.3:0/3699662356 shutdown_connections 2024-03-21T08:31:36.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 --2- 172.21.15.3:0/3699662356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd30105370 0x7efd3010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 --2- 172.21.15.3:0/3699662356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd301049b0 0x7efd30104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 --2- 172.21.15.3:0/3699662356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd30103be0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 -- 172.21.15.3:0/3699662356 >> 172.21.15.3:0/3699662356 conn(0x7efd300fed50 msgr2=0x7efd30101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 -- 172.21.15.3:0/3699662356 shutdown_connections 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.757+0000 7efd34a00640 1 -- 172.21.15.3:0/3699662356 wait complete. 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 Processor -- start 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 -- start start 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd301999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd301049b0 0x7efd30199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd30105370 0x7efd301a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3010f310 con 0x7efd301037c0 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3010f190 con 0x7efd30105370 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.758+0000 7efd34a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3010f490 con 0x7efd301049b0 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd301999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd301999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34258/0 (socket says 172.21.15.3:34258) 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 -- 172.21.15.3:0/1016561090 learned_addr learned my addr 172.21.15.3:0/1016561090 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2f200640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd30105370 0x7efd301a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd27e00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd301049b0 0x7efd30199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 -- 172.21.15.3:0/1016561090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd301049b0 msgr2=0x7efd30199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd301049b0 0x7efd30199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 -- 172.21.15.3:0/1016561090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd30105370 msgr2=0x7efd301a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd30105370 0x7efd301a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2e800640 1 -- 172.21.15.3:0/1016561090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd301a1600 con 0x7efd301037c0 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.759+0000 7efd2f200640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd30105370 0x7efd301a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.760+0000 7efd2e800640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd301999e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efd20005ab0 tx=0x7efd200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:36.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.760+0000 7efd27400640 1 -- 172.21.15.3:0/1016561090 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20043020 con 0x7efd301037c0 2024-03-21T08:31:36.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.760+0000 7efd27400640 1 -- 172.21.15.3:0/1016561090 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd200054c0 con 0x7efd301037c0 2024-03-21T08:31:36.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.760+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd301a1890 con 0x7efd301037c0 2024-03-21T08:31:36.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.760+0000 7efd27400640 1 -- 172.21.15.3:0/1016561090 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20003d10 con 0x7efd301037c0 2024-03-21T08:31:36.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.760+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd301a1de0 con 0x7efd301037c0 2024-03-21T08:31:36.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.761+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3010ab60 con 0x7efd301037c0 2024-03-21T08:31:36.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.763+0000 7efd27400640 1 -- 172.21.15.3:0/1016561090 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7efd20046020 con 0x7efd301037c0 2024-03-21T08:31:36.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.765+0000 7efd27400640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efd140a2f10 0x7efd140a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:36.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.765+0000 7efd27400640 1 -- 172.21.15.3:0/1016561090 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7efd200f2060 con 0x7efd301037c0 2024-03-21T08:31:36.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.765+0000 7efd27e00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efd140a2f10 0x7efd140a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:36.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.766+0000 7efd27e00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efd140a2f10 0x7efd140a53d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efd18005fd0 tx=0x7efd18005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:36.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.767+0000 7efd27400640 1 -- 172.21.15.3:0/1016561090 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd2004d020 con 0x7efd301037c0 2024-03-21T08:31:36.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.968+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd30061ae0 con 0x7efd140a2f10 2024-03-21T08:31:36.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:36.996+0000 7efd27400640 1 -- 172.21.15.3:0/1016561090 <== mgr.14152 v2:172.21.15.3:6800/1520535032 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340532 (secure 0 0 0) 0x7efd30061ae0 con 0x7efd140a2f10 2024-03-21T08:31:37.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:37.006 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-03-21T08:31:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efd140a2f10 msgr2=0x7efd140a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efd140a2f10 0x7efd140a53d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efd18005fd0 tx=0x7efd18005850 comp rx=0 tx=0).stop 2024-03-21T08:31:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 msgr2=0x7efd301999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd301999e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efd20005ab0 tx=0x7efd200042f0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 shutdown_connections 2024-03-21T08:31:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd30105370 0x7efd301a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7efd140a2f10 0x7efd140a53d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd301049b0 0x7efd30199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 --2- 172.21.15.3:0/1016561090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd301037c0 0x7efd301999e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 >> 172.21.15.3:0/1016561090 conn(0x7efd300fed50 msgr2=0x7efd30107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 shutdown_connections 2024-03-21T08:31:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.005+0000 7efd34a00640 1 -- 172.21.15.3:0/1016561090 wait complete. 2024-03-21T08:31:37.030 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-03-21T08:31:35.414486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663119,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1040,"num_read_kb":649,"num_write":364,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372344,"kb_used_data":14872,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569480,"statfs":{"total":1151940427776,"available":1151559147520,"internally_reserved":0,"allocated":15228928,"data_stored":8373173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005448"},"pg_stats":[{"pgid":"3.3f","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332311+0000","last_change":"2024-03-21T08:28:36.667851+0000","last_active":"2024-03-21T08:30:35.332311+0000","last_peered":"2024-03-21T08:30:35.332311+0000","last_clean":"2024-03-21T08:30:35.332311+0000","last_became_active":"2024-03-21T08:28:36.665713+0000","last_became_peered":"2024-03-21T08:28:36.665713+0000","last_unstale":"2024-03-21T08:30:35.332311+0000","last_undegraded":"2024-03-21T08:30:35.332311+0000","last_fullsized":"2024-03-21T08:30:35.332311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:02.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329013+0000","last_change":"2024-03-21T08:28:36.650986+0000","last_active":"2024-03-21T08:30:35.329013+0000","last_peered":"2024-03-21T08:30:35.329013+0000","last_clean":"2024-03-21T08:30:35.329013+0000","last_became_active":"2024-03-21T08:28:36.650633+0000","last_became_peered":"2024-03-21T08:28:36.650633+0000","last_unstale":"2024-03-21T08:30:35.329013+0000","last_undegraded":"2024-03-21T08:30:35.329013+0000","last_fullsized":"2024-03-21T08:30:35.329013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:40:20.801814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.335139+0000","last_change":"2024-03-21T08:28:36.662984+0000","last_active":"2024-03-21T08:30:35.335139+0000","last_peered":"2024-03-21T08:30:35.335139+0000","last_clean":"2024-03-21T08:30:35.335139+0000","last_became_active":"2024-03-21T08:28:36.662657+0000","last_became_peered":"2024-03-21T08:28:36.662657+0000","last_unstale":"2024-03-21T08:30:35.335139+0000","last_undegraded":"2024-03-21T08:30:35.335139+0000","last_fullsized":"2024-03-21T08:30:35.335139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:41.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.299562+0000","last_change":"2024-03-21T08:28:36.665958+0000","last_active":"2024-03-21T08:30:35.299562+0000","last_peered":"2024-03-21T08:30:35.299562+0000","last_clean":"2024-03-21T08:30:35.299562+0000","last_became_active":"2024-03-21T08:28:36.664309+0000","last_became_peered":"2024-03-21T08:28:36.664309+0000","last_unstale":"2024-03-21T08:30:35.299562+0000","last_undegraded":"2024-03-21T08:30:35.299562+0000","last_fullsized":"2024-03-21T08:30:35.299562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:16.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315553+0000","last_change":"2024-03-21T08:28:36.657767+0000","last_active":"2024-03-21T08:30:35.315553+0000","last_peered":"2024-03-21T08:30:35.315553+0000","last_clean":"2024-03-21T08:30:35.315553+0000","last_became_active":"2024-03-21T08:28:36.657350+0000","last_became_peered":"2024-03-21T08:28:36.657350+0000","last_unstale":"2024-03-21T08:30:35.315553+0000","last_undegraded":"2024-03-21T08:30:35.315553+0000","last_fullsized":"2024-03-21T08:30:35.315553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:47.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312622+0000","last_change":"2024-03-21T08:29:06.916971+0000","last_active":"2024-03-21T08:30:35.312622+0000","last_peered":"2024-03-21T08:30:35.312622+0000","last_clean":"2024-03-21T08:30:35.312622+0000","last_became_active":"2024-03-21T08:29:06.915997+0000","last_became_peered":"2024-03-21T08:29:06.915997+0000","last_unstale":"2024-03-21T08:30:35.312622+0000","last_undegraded":"2024-03-21T08:30:35.312622+0000","last_fullsized":"2024-03-21T08:30:35.312622+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:49:02.525190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312579+0000","last_change":"2024-03-21T08:28:36.667787+0000","last_active":"2024-03-21T08:30:35.312579+0000","last_peered":"2024-03-21T08:30:35.312579+0000","last_clean":"2024-03-21T08:30:35.312579+0000","last_became_active":"2024-03-21T08:28:36.667451+0000","last_became_peered":"2024-03-21T08:28:36.667451+0000","last_unstale":"2024-03-21T08:30:35.312579+0000","last_undegraded":"2024-03-21T08:30:35.312579+0000","last_fullsized":"2024-03-21T08:30:35.312579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:01.874897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328289+0000","last_change":"2024-03-21T08:28:36.662445+0000","last_active":"2024-03-21T08:30:35.328289+0000","last_peered":"2024-03-21T08:30:35.328289+0000","last_clean":"2024-03-21T08:30:35.328289+0000","last_became_active":"2024-03-21T08:28:36.662113+0000","last_became_peered":"2024-03-21T08:28:36.662113+0000","last_unstale":"2024-03-21T08:30:35.328289+0000","last_undegraded":"2024-03-21T08:30:35.328289+0000","last_fullsized":"2024-03-21T08:30:35.328289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:21.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331556+0000","last_change":"2024-03-21T08:28:36.666084+0000","last_active":"2024-03-21T08:30:35.331556+0000","last_peered":"2024-03-21T08:30:35.331556+0000","last_clean":"2024-03-21T08:30:35.331556+0000","last_became_active":"2024-03-21T08:28:36.665354+0000","last_became_peered":"2024-03-21T08:28:36.665354+0000","last_unstale":"2024-03-21T08:30:35.331556+0000","last_undegraded":"2024-03-21T08:30:35.331556+0000","last_fullsized":"2024-03-21T08:30:35.331556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:02:50.160765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319234+0000","last_change":"2024-03-21T08:29:06.919231+0000","last_active":"2024-03-21T08:30:35.319234+0000","last_peered":"2024-03-21T08:30:35.319234+0000","last_clean":"2024-03-21T08:30:35.319234+0000","last_became_active":"2024-03-21T08:29:06.918199+0000","last_became_peered":"2024-03-21T08:29:06.918199+0000","last_unstale":"2024-03-21T08:30:35.319234+0000","last_undegraded":"2024-03-21T08:30:35.319234+0000","last_fullsized":"2024-03-21T08:30:35.319234+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:44.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333943+0000","last_change":"2024-03-21T08:28:36.656385+0000","last_active":"2024-03-21T08:30:35.333943+0000","last_peered":"2024-03-21T08:30:35.333943+0000","last_clean":"2024-03-21T08:30:35.333943+0000","last_became_active":"2024-03-21T08:28:36.656014+0000","last_became_peered":"2024-03-21T08:28:36.656014+0000","last_unstale":"2024-03-21T08:30:35.333943+0000","last_undegraded":"2024-03-21T08:30:35.333943+0000","last_fullsized":"2024-03-21T08:30:35.333943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:46.000495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319565+0000","last_change":"2024-03-21T08:28:36.666254+0000","last_active":"2024-03-21T08:30:35.319565+0000","last_peered":"2024-03-21T08:30:35.319565+0000","last_clean":"2024-03-21T08:30:35.319565+0000","last_became_active":"2024-03-21T08:28:36.664698+0000","last_became_peered":"2024-03-21T08:28:36.664698+0000","last_unstale":"2024-03-21T08:30:35.319565+0000","last_undegraded":"2024-03-21T08:30:35.319565+0000","last_fullsized":"2024-03-21T08:30:35.319565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:03:08.358159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.327841+0000","last_change":"2024-03-21T08:29:06.919930+0000","last_active":"2024-03-21T08:30:35.327841+0000","last_peered":"2024-03-21T08:30:35.327841+0000","last_clean":"2024-03-21T08:30:35.327841+0000","last_became_active":"2024-03-21T08:29:06.917852+0000","last_became_peered":"2024-03-21T08:29:06.917852+0000","last_unstale":"2024-03-21T08:30:35.327841+0000","last_undegraded":"2024-03-21T08:30:35.327841+0000","last_fullsized":"2024-03-21T08:30:35.327841+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:05:01.185388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328029+0000","last_change":"2024-03-21T08:28:36.655229+0000","last_active":"2024-03-21T08:30:35.328029+0000","last_peered":"2024-03-21T08:30:35.328029+0000","last_clean":"2024-03-21T08:30:35.328029+0000","last_became_active":"2024-03-21T08:28:36.653949+0000","last_became_peered":"2024-03-21T08:28:36.653949+0000","last_unstale":"2024-03-21T08:30:35.328029+0000","last_undegraded":"2024-03-21T08:30:35.328029+0000","last_fullsized":"2024-03-21T08:30:35.328029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:57:38.370897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328261+0000","last_change":"2024-03-21T08:28:36.664894+0000","last_active":"2024-03-21T08:30:35.328261+0000","last_peered":"2024-03-21T08:30:35.328261+0000","last_clean":"2024-03-21T08:30:35.328261+0000","last_became_active":"2024-03-21T08:28:36.664523+0000","last_became_peered":"2024-03-21T08:28:36.664523+0000","last_unstale":"2024-03-21T08:30:35.328261+0000","last_undegraded":"2024-03-21T08:30:35.328261+0000","last_fullsized":"2024-03-21T08:30:35.328261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:52:54.147459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333446+0000","last_change":"2024-03-21T08:28:36.662374+0000","last_active":"2024-03-21T08:30:35.333446+0000","last_peered":"2024-03-21T08:30:35.333446+0000","last_clean":"2024-03-21T08:30:35.333446+0000","last_became_active":"2024-03-21T08:28:36.662066+0000","last_became_peered":"2024-03-21T08:28:36.662066+0000","last_unstale":"2024-03-21T08:30:35.333446+0000","last_undegraded":"2024-03-21T08:30:35.333446+0000","last_fullsized":"2024-03-21T08:30:35.333446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:06.082925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332398+0000","last_change":"2024-03-21T08:28:36.672172+0000","last_active":"2024-03-21T08:30:35.332398+0000","last_peered":"2024-03-21T08:30:35.332398+0000","last_clean":"2024-03-21T08:30:35.332398+0000","last_became_active":"2024-03-21T08:28:36.671657+0000","last_became_peered":"2024-03-21T08:28:36.671657+0000","last_unstale":"2024-03-21T08:30:35.332398+0000","last_undegraded":"2024-03-21T08:30:35.332398+0000","last_fullsized":"2024-03-21T08:30:35.332398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:56.892818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332036+0000","last_change":"2024-03-21T08:28:36.666973+0000","last_active":"2024-03-21T08:30:35.332036+0000","last_peered":"2024-03-21T08:30:35.332036+0000","last_clean":"2024-03-21T08:30:35.332036+0000","last_became_active":"2024-03-21T08:28:36.663770+0000","last_became_peered":"2024-03-21T08:28:36.663770+0000","last_unstale":"2024-03-21T08:30:35.332036+0000","last_undegraded":"2024-03-21T08:30:35.332036+0000","last_fullsized":"2024-03-21T08:30:35.332036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:54:26.701009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.318007+0000","last_change":"2024-03-21T08:28:36.668793+0000","last_active":"2024-03-21T08:30:35.318007+0000","last_peered":"2024-03-21T08:30:35.318007+0000","last_clean":"2024-03-21T08:30:35.318007+0000","last_became_active":"2024-03-21T08:28:36.668452+0000","last_became_peered":"2024-03-21T08:28:36.668452+0000","last_unstale":"2024-03-21T08:30:35.318007+0000","last_undegraded":"2024-03-21T08:30:35.318007+0000","last_fullsized":"2024-03-21T08:30:35.318007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:45.191849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645032+0000","last_change":"2024-03-21T08:28:40.687618+0000","last_active":"2024-03-21T08:30:06.645032+0000","last_peered":"2024-03-21T08:30:06.645032+0000","last_clean":"2024-03-21T08:30:06.645032+0000","last_became_active":"2024-03-21T08:28:40.687155+0000","last_became_peered":"2024-03-21T08:28:40.687155+0000","last_unstale":"2024-03-21T08:30:06.645032+0000","last_undegraded":"2024-03-21T08:30:06.645032+0000","last_fullsized":"2024-03-21T08:30:06.645032+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:08.014790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'9","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.673458+0000","last_change":"2024-03-21T08:30:06.672911+0000","last_active":"2024-03-21T08:30:06.673458+0000","last_peered":"2024-03-21T08:30:06.673458+0000","last_clean":"2024-03-21T08:30:06.673458+0000","last_became_active":"2024-03-21T08:30:06.661974+0000","last_became_peered":"2024-03-21T08:30:06.661974+0000","last_unstale":"2024-03-21T08:30:06.673458+0000","last_undegraded":"2024-03-21T08:30:06.673458+0000","last_fullsized":"2024-03-21T08:30:06.673458+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:37:25.848279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313894+0000","last_change":"2024-03-21T08:28:36.670325+0000","last_active":"2024-03-21T08:30:35.313894+0000","last_peered":"2024-03-21T08:30:35.313894+0000","last_clean":"2024-03-21T08:30:35.313894+0000","last_became_active":"2024-03-21T08:28:36.669783+0000","last_became_peered":"2024-03-21T08:28:36.669783+0000","last_unstale":"2024-03-21T08:30:35.313894+0000","last_undegraded":"2024-03-21T08:30:35.313894+0000","last_fullsized":"2024-03-21T08:30:35.313894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:24.166567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663706+0000","last_change":"2024-03-21T08:28:40.689849+0000","last_active":"2024-03-21T08:30:06.663706+0000","last_peered":"2024-03-21T08:30:06.663706+0000","last_clean":"2024-03-21T08:30:06.663706+0000","last_became_active":"2024-03-21T08:28:40.689307+0000","last_became_peered":"2024-03-21T08:28:40.689307+0000","last_unstale":"2024-03-21T08:30:06.663706+0000","last_undegraded":"2024-03-21T08:30:06.663706+0000","last_fullsized":"2024-03-21T08:30:06.663706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:34:05.660317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647944+0000","last_change":"2024-03-21T08:28:35.674685+0000","last_active":"2024-03-21T08:30:06.647944+0000","last_peered":"2024-03-21T08:30:06.647944+0000","last_clean":"2024-03-21T08:30:06.647944+0000","last_became_active":"2024-03-21T08:28:35.674312+0000","last_became_peered":"2024-03-21T08:28:35.674312+0000","last_unstale":"2024-03-21T08:30:06.647944+0000","last_undegraded":"2024-03-21T08:30:06.647944+0000","last_fullsized":"2024-03-21T08:30:06.647944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:27.644111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.335342+0000","last_change":"2024-03-21T08:28:36.667744+0000","last_active":"2024-03-21T08:30:35.335342+0000","last_peered":"2024-03-21T08:30:35.335342+0000","last_clean":"2024-03-21T08:30:35.335342+0000","last_became_active":"2024-03-21T08:28:36.667279+0000","last_became_peered":"2024-03-21T08:28:36.667279+0000","last_unstale":"2024-03-21T08:30:35.335342+0000","last_undegraded":"2024-03-21T08:30:35.335342+0000","last_fullsized":"2024-03-21T08:30:35.335342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:14:24.974683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664879+0000","last_change":"2024-03-21T08:28:40.688864+0000","last_active":"2024-03-21T08:30:06.664879+0000","last_peered":"2024-03-21T08:30:06.664879+0000","last_clean":"2024-03-21T08:30:06.664879+0000","last_became_active":"2024-03-21T08:28:40.688029+0000","last_became_peered":"2024-03-21T08:28:40.688029+0000","last_unstale":"2024-03-21T08:30:06.664879+0000","last_undegraded":"2024-03-21T08:30:06.664879+0000","last_fullsized":"2024-03-21T08:30:06.664879+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:39.235807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.681016+0000","last_change":"2024-03-21T08:30:06.680224+0000","last_active":"2024-03-21T08:30:06.681016+0000","last_peered":"2024-03-21T08:30:06.681016+0000","last_clean":"2024-03-21T08:30:06.681016+0000","last_became_active":"2024-03-21T08:30:06.665898+0000","last_became_peered":"2024-03-21T08:30:06.665898+0000","last_unstale":"2024-03-21T08:30:06.681016+0000","last_undegraded":"2024-03-21T08:30:06.681016+0000","last_fullsized":"2024-03-21T08:30:06.681016+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:21.690576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333879+0000","last_change":"2024-03-21T08:28:36.667244+0000","last_active":"2024-03-21T08:30:35.333879+0000","last_peered":"2024-03-21T08:30:35.333879+0000","last_clean":"2024-03-21T08:30:35.333879+0000","last_became_active":"2024-03-21T08:28:36.666829+0000","last_became_peered":"2024-03-21T08:28:36.666829+0000","last_unstale":"2024-03-21T08:30:35.333879+0000","last_undegraded":"2024-03-21T08:30:35.333879+0000","last_fullsized":"2024-03-21T08:30:35.333879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:12:56.842084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657793+0000","last_change":"2024-03-21T08:28:40.692621+0000","last_active":"2024-03-21T08:30:06.657793+0000","last_peered":"2024-03-21T08:30:06.657793+0000","last_clean":"2024-03-21T08:30:06.657793+0000","last_became_active":"2024-03-21T08:28:40.691984+0000","last_became_peered":"2024-03-21T08:28:40.691984+0000","last_unstale":"2024-03-21T08:30:06.657793+0000","last_undegraded":"2024-03-21T08:30:06.657793+0000","last_fullsized":"2024-03-21T08:30:06.657793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:29.330702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651368+0000","last_change":"2024-03-21T08:28:35.676421+0000","last_active":"2024-03-21T08:30:06.651368+0000","last_peered":"2024-03-21T08:30:06.651368+0000","last_clean":"2024-03-21T08:30:06.651368+0000","last_became_active":"2024-03-21T08:28:35.674957+0000","last_became_peered":"2024-03-21T08:28:35.674957+0000","last_unstale":"2024-03-21T08:30:06.651368+0000","last_undegraded":"2024-03-21T08:30:06.651368+0000","last_fullsized":"2024-03-21T08:30:06.651368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:01:38.397757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331569+0000","last_change":"2024-03-21T08:28:36.664662+0000","last_active":"2024-03-21T08:30:35.331569+0000","last_peered":"2024-03-21T08:30:35.331569+0000","last_clean":"2024-03-21T08:30:35.331569+0000","last_became_active":"2024-03-21T08:28:36.664324+0000","last_became_peered":"2024-03-21T08:28:36.664324+0000","last_unstale":"2024-03-21T08:30:35.331569+0000","last_undegraded":"2024-03-21T08:30:35.331569+0000","last_fullsized":"2024-03-21T08:30:35.331569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:57.325869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645495+0000","last_change":"2024-03-21T08:28:40.676297+0000","last_active":"2024-03-21T08:30:06.645495+0000","last_peered":"2024-03-21T08:30:06.645495+0000","last_clean":"2024-03-21T08:30:06.645495+0000","last_became_active":"2024-03-21T08:28:40.675337+0000","last_became_peered":"2024-03-21T08:28:40.675337+0000","last_unstale":"2024-03-21T08:30:06.645495+0000","last_undegraded":"2024-03-21T08:30:06.645495+0000","last_fullsized":"2024-03-21T08:30:06.645495+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:05.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'8","reported_seq":189,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:31.737556+0000","last_change":"2024-03-21T08:28:35.687651+0000","last_active":"2024-03-21T08:31:31.737556+0000","last_peered":"2024-03-21T08:31:31.737556+0000","last_clean":"2024-03-21T08:31:31.737556+0000","last_became_active":"2024-03-21T08:28:35.686284+0000","last_became_peered":"2024-03-21T08:28:35.686284+0000","last_unstale":"2024-03-21T08:31:31.737556+0000","last_undegraded":"2024-03-21T08:31:31.737556+0000","last_fullsized":"2024-03-21T08:31:31.737556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:56.121798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315336+0000","last_change":"2024-03-21T08:29:07.033436+0000","last_active":"2024-03-21T08:30:35.315336+0000","last_peered":"2024-03-21T08:30:35.315336+0000","last_clean":"2024-03-21T08:30:35.315336+0000","last_became_active":"2024-03-21T08:29:07.032012+0000","last_became_peered":"2024-03-21T08:29:07.032012+0000","last_unstale":"2024-03-21T08:30:35.315336+0000","last_undegraded":"2024-03-21T08:30:35.315336+0000","last_fullsized":"2024-03-21T08:30:35.315336+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:44:47.009078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645774+0000","last_change":"2024-03-21T08:29:06.882449+0000","last_active":"2024-03-21T08:30:06.645774+0000","last_peered":"2024-03-21T08:30:06.645774+0000","last_clean":"2024-03-21T08:30:06.645774+0000","last_became_active":"2024-03-21T08:29:06.880682+0000","last_became_peered":"2024-03-21T08:29:06.880682+0000","last_unstale":"2024-03-21T08:30:06.645774+0000","last_undegraded":"2024-03-21T08:30:06.645774+0000","last_fullsized":"2024-03-21T08:30:06.645774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:09.464388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.661857+0000","last_change":"2024-03-21T08:30:06.659993+0000","last_active":"2024-03-21T08:30:06.661857+0000","last_peered":"2024-03-21T08:30:06.661857+0000","last_clean":"2024-03-21T08:30:06.661857+0000","last_became_active":"2024-03-21T08:30:06.653383+0000","last_became_peered":"2024-03-21T08:30:06.653383+0000","last_unstale":"2024-03-21T08:30:06.661857+0000","last_undegraded":"2024-03-21T08:30:06.661857+0000","last_fullsized":"2024-03-21T08:30:06.661857+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:43.836881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317829+0000","last_change":"2024-03-21T08:28:36.670176+0000","last_active":"2024-03-21T08:30:35.317829+0000","last_peered":"2024-03-21T08:30:35.317829+0000","last_clean":"2024-03-21T08:30:35.317829+0000","last_became_active":"2024-03-21T08:28:36.669477+0000","last_became_peered":"2024-03-21T08:28:36.669477+0000","last_unstale":"2024-03-21T08:30:35.317829+0000","last_undegraded":"2024-03-21T08:30:35.317829+0000","last_fullsized":"2024-03-21T08:30:35.317829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:10:21.289484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654777+0000","last_change":"2024-03-21T08:29:06.784996+0000","last_active":"2024-03-21T08:30:06.654777+0000","last_peered":"2024-03-21T08:30:06.654777+0000","last_clean":"2024-03-21T08:30:06.654777+0000","last_became_active":"2024-03-21T08:29:06.783403+0000","last_became_peered":"2024-03-21T08:29:06.783403+0000","last_unstale":"2024-03-21T08:30:06.654777+0000","last_undegraded":"2024-03-21T08:30:06.654777+0000","last_fullsized":"2024-03-21T08:30:06.654777+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:18:59.912956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.641118+0000","last_change":"2024-03-21T08:28:35.667502+0000","last_active":"2024-03-21T08:30:06.641118+0000","last_peered":"2024-03-21T08:30:06.641118+0000","last_clean":"2024-03-21T08:30:06.641118+0000","last_became_active":"2024-03-21T08:28:35.666667+0000","last_became_peered":"2024-03-21T08:28:35.666667+0000","last_unstale":"2024-03-21T08:30:06.641118+0000","last_undegraded":"2024-03-21T08:30:06.641118+0000","last_fullsized":"2024-03-21T08:30:06.641118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:42.196361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.318414+0000","last_change":"2024-03-21T08:28:36.670855+0000","last_active":"2024-03-21T08:30:35.318414+0000","last_peered":"2024-03-21T08:30:35.318414+0000","last_clean":"2024-03-21T08:30:35.318414+0000","last_became_active":"2024-03-21T08:28:36.666012+0000","last_became_peered":"2024-03-21T08:28:36.666012+0000","last_unstale":"2024-03-21T08:30:35.318414+0000","last_undegraded":"2024-03-21T08:30:35.318414+0000","last_fullsized":"2024-03-21T08:30:35.318414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:19.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652911+0000","last_change":"2024-03-21T08:28:40.700037+0000","last_active":"2024-03-21T08:30:06.652911+0000","last_peered":"2024-03-21T08:30:06.652911+0000","last_clean":"2024-03-21T08:30:06.652911+0000","last_became_active":"2024-03-21T08:28:40.698127+0000","last_became_peered":"2024-03-21T08:28:40.698127+0000","last_unstale":"2024-03-21T08:30:06.652911+0000","last_undegraded":"2024-03-21T08:30:06.652911+0000","last_fullsized":"2024-03-21T08:30:06.652911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:48:57.135690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664533+0000","last_change":"2024-03-21T08:28:35.677071+0000","last_active":"2024-03-21T08:30:06.664533+0000","last_peered":"2024-03-21T08:30:06.664533+0000","last_clean":"2024-03-21T08:30:06.664533+0000","last_became_active":"2024-03-21T08:28:35.676652+0000","last_became_peered":"2024-03-21T08:28:35.676652+0000","last_unstale":"2024-03-21T08:30:06.664533+0000","last_undegraded":"2024-03-21T08:30:06.664533+0000","last_fullsized":"2024-03-21T08:30:06.664533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:24:54.097952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.321747+0000","last_change":"2024-03-21T08:28:36.677981+0000","last_active":"2024-03-21T08:30:35.321747+0000","last_peered":"2024-03-21T08:30:35.321747+0000","last_clean":"2024-03-21T08:30:35.321747+0000","last_became_active":"2024-03-21T08:28:36.677512+0000","last_became_peered":"2024-03-21T08:28:36.677512+0000","last_unstale":"2024-03-21T08:30:35.321747+0000","last_undegraded":"2024-03-21T08:30:35.321747+0000","last_fullsized":"2024-03-21T08:30:35.321747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:50.742928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646728+0000","last_change":"2024-03-21T08:28:40.692672+0000","last_active":"2024-03-21T08:30:06.646728+0000","last_peered":"2024-03-21T08:30:06.646728+0000","last_clean":"2024-03-21T08:30:06.646728+0000","last_became_active":"2024-03-21T08:28:40.688940+0000","last_became_peered":"2024-03-21T08:28:40.688940+0000","last_unstale":"2024-03-21T08:30:06.646728+0000","last_undegraded":"2024-03-21T08:30:06.646728+0000","last_fullsized":"2024-03-21T08:30:06.646728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:39.569709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:32.739007+0000","last_change":"2024-03-21T08:28:35.681970+0000","last_active":"2024-03-21T08:31:32.739007+0000","last_peered":"2024-03-21T08:31:32.739007+0000","last_clean":"2024-03-21T08:31:32.739007+0000","last_became_active":"2024-03-21T08:28:35.681252+0000","last_became_peered":"2024-03-21T08:28:35.681252+0000","last_unstale":"2024-03-21T08:31:32.739007+0000","last_undegraded":"2024-03-21T08:31:32.739007+0000","last_fullsized":"2024-03-21T08:31:32.739007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:46.345359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.334525+0000","last_change":"2024-03-21T08:28:36.675419+0000","last_active":"2024-03-21T08:30:35.334525+0000","last_peered":"2024-03-21T08:30:35.334525+0000","last_clean":"2024-03-21T08:30:35.334525+0000","last_became_active":"2024-03-21T08:28:36.674039+0000","last_became_peered":"2024-03-21T08:28:36.674039+0000","last_unstale":"2024-03-21T08:30:35.334525+0000","last_undegraded":"2024-03-21T08:30:35.334525+0000","last_fullsized":"2024-03-21T08:30:35.334525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:09.667634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648585+0000","last_change":"2024-03-21T08:28:40.690076+0000","last_active":"2024-03-21T08:30:06.648585+0000","last_peered":"2024-03-21T08:30:06.648585+0000","last_clean":"2024-03-21T08:30:06.648585+0000","last_became_active":"2024-03-21T08:28:40.683817+0000","last_became_peered":"2024-03-21T08:28:40.683817+0000","last_unstale":"2024-03-21T08:30:06.648585+0000","last_undegraded":"2024-03-21T08:30:06.648585+0000","last_fullsized":"2024-03-21T08:30:06.648585+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:33:00.948879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654217+0000","last_change":"2024-03-21T08:28:35.690693+0000","last_active":"2024-03-21T08:30:06.654217+0000","last_peered":"2024-03-21T08:30:06.654217+0000","last_clean":"2024-03-21T08:30:06.654217+0000","last_became_active":"2024-03-21T08:28:35.690250+0000","last_became_peered":"2024-03-21T08:28:35.690250+0000","last_unstale":"2024-03-21T08:30:06.654217+0000","last_undegraded":"2024-03-21T08:30:06.654217+0000","last_fullsized":"2024-03-21T08:30:06.654217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:44:21.307772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330814+0000","last_change":"2024-03-21T08:28:36.661872+0000","last_active":"2024-03-21T08:30:35.330814+0000","last_peered":"2024-03-21T08:30:35.330814+0000","last_clean":"2024-03-21T08:30:35.330814+0000","last_became_active":"2024-03-21T08:28:36.661483+0000","last_became_peered":"2024-03-21T08:28:36.661483+0000","last_unstale":"2024-03-21T08:30:35.330814+0000","last_undegraded":"2024-03-21T08:30:35.330814+0000","last_fullsized":"2024-03-21T08:30:35.330814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:03:35.630141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665016+0000","last_change":"2024-03-21T08:28:40.682753+0000","last_active":"2024-03-21T08:30:06.665016+0000","last_peered":"2024-03-21T08:30:06.665016+0000","last_clean":"2024-03-21T08:30:06.665016+0000","last_became_active":"2024-03-21T08:28:40.682278+0000","last_became_peered":"2024-03-21T08:28:40.682278+0000","last_unstale":"2024-03-21T08:30:06.665016+0000","last_undegraded":"2024-03-21T08:30:06.665016+0000","last_fullsized":"2024-03-21T08:30:06.665016+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:20.280442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666308+0000","last_change":"2024-03-21T08:28:35.672524+0000","last_active":"2024-03-21T08:30:06.666308+0000","last_peered":"2024-03-21T08:30:06.666308+0000","last_clean":"2024-03-21T08:30:06.666308+0000","last_became_active":"2024-03-21T08:28:35.672051+0000","last_became_peered":"2024-03-21T08:28:35.672051+0000","last_unstale":"2024-03-21T08:30:06.666308+0000","last_undegraded":"2024-03-21T08:30:06.666308+0000","last_fullsized":"2024-03-21T08:30:06.666308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:53.336320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329907+0000","last_change":"2024-03-21T08:29:06.532172+0000","last_active":"2024-03-21T08:30:35.329907+0000","last_peered":"2024-03-21T08:30:35.329907+0000","last_clean":"2024-03-21T08:30:35.329907+0000","last_became_active":"2024-03-21T08:29:06.531439+0000","last_became_peered":"2024-03-21T08:29:06.531439+0000","last_unstale":"2024-03-21T08:30:35.329907+0000","last_undegraded":"2024-03-21T08:30:35.329907+0000","last_fullsized":"2024-03-21T08:30:35.329907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:27.222359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.648624+0000","last_change":"2024-03-21T08:28:40.682294+0000","last_active":"2024-03-21T08:30:06.648624+0000","last_peered":"2024-03-21T08:30:06.648624+0000","last_clean":"2024-03-21T08:30:06.648624+0000","last_became_active":"2024-03-21T08:28:40.681855+0000","last_became_peered":"2024-03-21T08:28:40.681855+0000","last_unstale":"2024-03-21T08:30:06.648624+0000","last_undegraded":"2024-03-21T08:30:06.648624+0000","last_fullsized":"2024-03-21T08:30:06.648624+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:49.756322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"96'40","reported_seq":374,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:33.736869+0000","last_change":"2024-03-21T08:28:35.682947+0000","last_active":"2024-03-21T08:31:33.736869+0000","last_peered":"2024-03-21T08:31:33.736869+0000","last_clean":"2024-03-21T08:31:33.736869+0000","last_became_active":"2024-03-21T08:28:35.682395+0000","last_became_peered":"2024-03-21T08:28:35.682395+0000","last_unstale":"2024-03-21T08:31:33.736869+0000","last_undegraded":"2024-03-21T08:31:33.736869+0000","last_fullsized":"2024-03-21T08:31:33.736869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:07:48.072988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67389,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":218,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.310604+0000","last_change":"2024-03-21T08:28:36.651140+0000","last_active":"2024-03-21T08:30:35.310604+0000","last_peered":"2024-03-21T08:30:35.310604+0000","last_clean":"2024-03-21T08:30:35.310604+0000","last_became_active":"2024-03-21T08:28:36.650551+0000","last_became_peered":"2024-03-21T08:28:36.650551+0000","last_unstale":"2024-03-21T08:30:35.310604+0000","last_undegraded":"2024-03-21T08:30:35.310604+0000","last_fullsized":"2024-03-21T08:30:35.310604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:53.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.667084+0000","last_change":"2024-03-21T08:28:40.691814+0000","last_active":"2024-03-21T08:30:06.667084+0000","last_peered":"2024-03-21T08:30:06.667084+0000","last_clean":"2024-03-21T08:30:06.667084+0000","last_became_active":"2024-03-21T08:28:40.691372+0000","last_became_peered":"2024-03-21T08:28:40.691372+0000","last_unstale":"2024-03-21T08:30:06.667084+0000","last_undegraded":"2024-03-21T08:30:06.667084+0000","last_fullsized":"2024-03-21T08:30:06.667084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:21:42.612037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645742+0000","last_change":"2024-03-21T08:28:35.687178+0000","last_active":"2024-03-21T08:30:06.645742+0000","last_peered":"2024-03-21T08:30:06.645742+0000","last_clean":"2024-03-21T08:30:06.645742+0000","last_became_active":"2024-03-21T08:28:35.686771+0000","last_became_peered":"2024-03-21T08:28:35.686771+0000","last_unstale":"2024-03-21T08:30:06.645742+0000","last_undegraded":"2024-03-21T08:30:06.645742+0000","last_fullsized":"2024-03-21T08:30:06.645742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:54.665866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317525+0000","last_change":"2024-03-21T08:28:36.662952+0000","last_active":"2024-03-21T08:30:35.317525+0000","last_peered":"2024-03-21T08:30:35.317525+0000","last_clean":"2024-03-21T08:30:35.317525+0000","last_became_active":"2024-03-21T08:28:36.662468+0000","last_became_peered":"2024-03-21T08:28:36.662468+0000","last_unstale":"2024-03-21T08:30:35.317525+0000","last_undegraded":"2024-03-21T08:30:35.317525+0000","last_fullsized":"2024-03-21T08:30:35.317525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:40.486998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.656409+0000","last_change":"2024-03-21T08:28:40.693999+0000","last_active":"2024-03-21T08:30:06.656409+0000","last_peered":"2024-03-21T08:30:06.656409+0000","last_clean":"2024-03-21T08:30:06.656409+0000","last_became_active":"2024-03-21T08:28:40.693507+0000","last_became_peered":"2024-03-21T08:28:40.693507+0000","last_unstale":"2024-03-21T08:30:06.656409+0000","last_undegraded":"2024-03-21T08:30:06.656409+0000","last_fullsized":"2024-03-21T08:30:06.656409+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:14.159379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.668728+0000","last_change":"2024-03-21T08:28:35.677536+0000","last_active":"2024-03-21T08:30:06.668728+0000","last_peered":"2024-03-21T08:30:06.668728+0000","last_clean":"2024-03-21T08:30:06.668728+0000","last_became_active":"2024-03-21T08:28:35.677100+0000","last_became_peered":"2024-03-21T08:28:35.677100+0000","last_unstale":"2024-03-21T08:30:06.668728+0000","last_undegraded":"2024-03-21T08:30:06.668728+0000","last_fullsized":"2024-03-21T08:30:06.668728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:52:47.275869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.322854+0000","last_change":"2024-03-21T08:28:36.667190+0000","last_active":"2024-03-21T08:30:35.322854+0000","last_peered":"2024-03-21T08:30:35.322854+0000","last_clean":"2024-03-21T08:30:35.322854+0000","last_became_active":"2024-03-21T08:28:36.665336+0000","last_became_peered":"2024-03-21T08:28:36.665336+0000","last_unstale":"2024-03-21T08:30:35.322854+0000","last_undegraded":"2024-03-21T08:30:35.322854+0000","last_fullsized":"2024-03-21T08:30:35.322854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:21.888426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649761+0000","last_change":"2024-03-21T08:28:40.696582+0000","last_active":"2024-03-21T08:30:06.649761+0000","last_peered":"2024-03-21T08:30:06.649761+0000","last_clean":"2024-03-21T08:30:06.649761+0000","last_became_active":"2024-03-21T08:28:40.689583+0000","last_became_peered":"2024-03-21T08:28:40.689583+0000","last_unstale":"2024-03-21T08:30:06.649761+0000","last_undegraded":"2024-03-21T08:30:06.649761+0000","last_fullsized":"2024-03-21T08:30:06.649761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:57.329533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"96'4","reported_seq":185,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:33.736977+0000","last_change":"2024-03-21T08:28:35.675560+0000","last_active":"2024-03-21T08:31:33.736977+0000","last_peered":"2024-03-21T08:31:33.736977+0000","last_clean":"2024-03-21T08:31:33.736977+0000","last_became_active":"2024-03-21T08:28:35.673496+0000","last_became_peered":"2024-03-21T08:28:35.673496+0000","last_unstale":"2024-03-21T08:31:33.736977+0000","last_undegraded":"2024-03-21T08:31:33.736977+0000","last_fullsized":"2024-03-21T08:31:33.736977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:41:27.996458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.322318+0000","last_change":"2024-03-21T08:28:36.660869+0000","last_active":"2024-03-21T08:30:35.322318+0000","last_peered":"2024-03-21T08:30:35.322318+0000","last_clean":"2024-03-21T08:30:35.322318+0000","last_became_active":"2024-03-21T08:28:36.659291+0000","last_became_peered":"2024-03-21T08:28:36.659291+0000","last_unstale":"2024-03-21T08:30:35.322318+0000","last_undegraded":"2024-03-21T08:30:35.322318+0000","last_fullsized":"2024-03-21T08:30:35.322318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:06.534214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.643153+0000","last_change":"2024-03-21T08:28:40.701351+0000","last_active":"2024-03-21T08:30:06.643153+0000","last_peered":"2024-03-21T08:30:06.643153+0000","last_clean":"2024-03-21T08:30:06.643153+0000","last_became_active":"2024-03-21T08:28:40.700989+0000","last_became_peered":"2024-03-21T08:28:40.700989+0000","last_unstale":"2024-03-21T08:30:06.643153+0000","last_undegraded":"2024-03-21T08:30:06.643153+0000","last_fullsized":"2024-03-21T08:30:06.643153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:35.697013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.650538+0000","last_change":"2024-03-21T08:28:35.685024+0000","last_active":"2024-03-21T08:30:06.650538+0000","last_peered":"2024-03-21T08:30:06.650538+0000","last_clean":"2024-03-21T08:30:06.650538+0000","last_became_active":"2024-03-21T08:28:35.684597+0000","last_became_peered":"2024-03-21T08:28:35.684597+0000","last_unstale":"2024-03-21T08:30:06.650538+0000","last_undegraded":"2024-03-21T08:30:06.650538+0000","last_fullsized":"2024-03-21T08:30:06.650538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:09:46.262782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.332203+0000","last_change":"2024-03-21T08:28:36.664688+0000","last_active":"2024-03-21T08:30:35.332203+0000","last_peered":"2024-03-21T08:30:35.332203+0000","last_clean":"2024-03-21T08:30:35.332203+0000","last_became_active":"2024-03-21T08:28:36.663990+0000","last_became_peered":"2024-03-21T08:28:36.663990+0000","last_unstale":"2024-03-21T08:30:35.332203+0000","last_undegraded":"2024-03-21T08:30:35.332203+0000","last_fullsized":"2024-03-21T08:30:35.332203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:39.656290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664426+0000","last_change":"2024-03-21T08:28:40.695269+0000","last_active":"2024-03-21T08:30:06.664426+0000","last_peered":"2024-03-21T08:30:06.664426+0000","last_clean":"2024-03-21T08:30:06.664426+0000","last_became_active":"2024-03-21T08:28:40.694820+0000","last_became_peered":"2024-03-21T08:28:40.694820+0000","last_unstale":"2024-03-21T08:30:06.664426+0000","last_undegraded":"2024-03-21T08:30:06.664426+0000","last_fullsized":"2024-03-21T08:30:06.664426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:47:25.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.673554+0000","last_change":"2024-03-21T08:30:06.672662+0000","last_active":"2024-03-21T08:30:06.673554+0000","last_peered":"2024-03-21T08:30:06.673554+0000","last_clean":"2024-03-21T08:30:06.673554+0000","last_became_active":"2024-03-21T08:30:06.660161+0000","last_became_peered":"2024-03-21T08:30:06.660161+0000","last_unstale":"2024-03-21T08:30:06.673554+0000","last_undegraded":"2024-03-21T08:30:06.673554+0000","last_fullsized":"2024-03-21T08:30:06.673554+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:41.163709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,4],"acting":[8,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315069+0000","last_change":"2024-03-21T08:28:36.667605+0000","last_active":"2024-03-21T08:30:35.315069+0000","last_peered":"2024-03-21T08:30:35.315069+0000","last_clean":"2024-03-21T08:30:35.315069+0000","last_became_active":"2024-03-21T08:28:36.667300+0000","last_became_peered":"2024-03-21T08:28:36.667300+0000","last_unstale":"2024-03-21T08:30:35.315069+0000","last_undegraded":"2024-03-21T08:30:35.315069+0000","last_fullsized":"2024-03-21T08:30:35.315069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:40:49.268935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.652659+0000","last_change":"2024-03-21T08:28:40.884964+0000","last_active":"2024-03-21T08:30:06.652659+0000","last_peered":"2024-03-21T08:30:06.652659+0000","last_clean":"2024-03-21T08:30:06.652659+0000","last_became_active":"2024-03-21T08:28:40.883445+0000","last_became_peered":"2024-03-21T08:28:40.883445+0000","last_unstale":"2024-03-21T08:30:06.652659+0000","last_undegraded":"2024-03-21T08:30:06.652659+0000","last_fullsized":"2024-03-21T08:30:06.652659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:36:54.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.647966+0000","last_change":"2024-03-21T08:28:35.682581+0000","last_active":"2024-03-21T08:30:06.647966+0000","last_peered":"2024-03-21T08:30:06.647966+0000","last_clean":"2024-03-21T08:30:06.647966+0000","last_became_active":"2024-03-21T08:28:35.682150+0000","last_became_peered":"2024-03-21T08:28:35.682150+0000","last_unstale":"2024-03-21T08:30:06.647966+0000","last_undegraded":"2024-03-21T08:30:06.647966+0000","last_fullsized":"2024-03-21T08:30:06.647966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:08.729710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316730+0000","last_change":"2024-03-21T08:28:36.666632+0000","last_active":"2024-03-21T08:30:35.316730+0000","last_peered":"2024-03-21T08:30:35.316730+0000","last_clean":"2024-03-21T08:30:35.316730+0000","last_became_active":"2024-03-21T08:28:36.666305+0000","last_became_peered":"2024-03-21T08:28:36.666305+0000","last_unstale":"2024-03-21T08:30:35.316730+0000","last_undegraded":"2024-03-21T08:30:35.316730+0000","last_fullsized":"2024-03-21T08:30:35.316730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:15.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.642891+0000","last_change":"2024-03-21T08:28:40.700382+0000","last_active":"2024-03-21T08:30:06.642891+0000","last_peered":"2024-03-21T08:30:06.642891+0000","last_clean":"2024-03-21T08:30:06.642891+0000","last_became_active":"2024-03-21T08:28:40.699762+0000","last_became_peered":"2024-03-21T08:28:40.699762+0000","last_unstale":"2024-03-21T08:30:06.642891+0000","last_undegraded":"2024-03-21T08:30:06.642891+0000","last_fullsized":"2024-03-21T08:30:06.642891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:53.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657107+0000","last_change":"2024-03-21T08:28:35.680310+0000","last_active":"2024-03-21T08:30:06.657107+0000","last_peered":"2024-03-21T08:30:06.657107+0000","last_clean":"2024-03-21T08:30:06.657107+0000","last_became_active":"2024-03-21T08:28:35.678634+0000","last_became_peered":"2024-03-21T08:28:35.678634+0000","last_unstale":"2024-03-21T08:30:06.657107+0000","last_undegraded":"2024-03-21T08:30:06.657107+0000","last_fullsized":"2024-03-21T08:30:06.657107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:56:42.872521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319411+0000","last_change":"2024-03-21T08:28:36.669527+0000","last_active":"2024-03-21T08:30:35.319411+0000","last_peered":"2024-03-21T08:30:35.319411+0000","last_clean":"2024-03-21T08:30:35.319411+0000","last_became_active":"2024-03-21T08:28:36.669196+0000","last_became_peered":"2024-03-21T08:28:36.669196+0000","last_unstale":"2024-03-21T08:30:35.319411+0000","last_undegraded":"2024-03-21T08:30:35.319411+0000","last_fullsized":"2024-03-21T08:30:35.319411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:07:34.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644070+0000","last_change":"2024-03-21T08:28:40.882949+0000","last_active":"2024-03-21T08:30:06.644070+0000","last_peered":"2024-03-21T08:30:06.644070+0000","last_clean":"2024-03-21T08:30:06.644070+0000","last_became_active":"2024-03-21T08:28:40.882432+0000","last_became_peered":"2024-03-21T08:28:40.882432+0000","last_unstale":"2024-03-21T08:30:06.644070+0000","last_undegraded":"2024-03-21T08:30:06.644070+0000","last_fullsized":"2024-03-21T08:30:06.644070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:26:51.171365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.321838+0000","last_change":"2024-03-21T08:28:36.665164+0000","last_active":"2024-03-21T08:30:35.321838+0000","last_peered":"2024-03-21T08:30:35.321838+0000","last_clean":"2024-03-21T08:30:35.321838+0000","last_became_active":"2024-03-21T08:28:36.664794+0000","last_became_peered":"2024-03-21T08:28:36.664794+0000","last_unstale":"2024-03-21T08:30:35.321838+0000","last_undegraded":"2024-03-21T08:30:35.321838+0000","last_fullsized":"2024-03-21T08:30:35.321838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:25:39.355633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'5","reported_seq":182,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:31:32.733229+0000","last_change":"2024-03-21T08:28:35.671754+0000","last_active":"2024-03-21T08:31:32.733229+0000","last_peered":"2024-03-21T08:31:32.733229+0000","last_clean":"2024-03-21T08:31:32.733229+0000","last_became_active":"2024-03-21T08:28:35.671363+0000","last_became_peered":"2024-03-21T08:28:35.671363+0000","last_unstale":"2024-03-21T08:31:32.733229+0000","last_undegraded":"2024-03-21T08:31:32.733229+0000","last_fullsized":"2024-03-21T08:31:32.733229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:22:11.147325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.657862+0000","last_change":"2024-03-21T08:28:40.689038+0000","last_active":"2024-03-21T08:30:06.657862+0000","last_peered":"2024-03-21T08:30:06.657862+0000","last_clean":"2024-03-21T08:30:06.657862+0000","last_became_active":"2024-03-21T08:28:40.688538+0000","last_became_peered":"2024-03-21T08:28:40.688538+0000","last_unstale":"2024-03-21T08:30:06.657862+0000","last_undegraded":"2024-03-21T08:30:06.657862+0000","last_fullsized":"2024-03-21T08:30:06.657862+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:14.833594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.331670+0000","last_change":"2024-03-21T08:28:36.667725+0000","last_active":"2024-03-21T08:30:35.331670+0000","last_peered":"2024-03-21T08:30:35.331670+0000","last_clean":"2024-03-21T08:30:35.331670+0000","last_became_active":"2024-03-21T08:28:36.667303+0000","last_became_peered":"2024-03-21T08:28:36.667303+0000","last_unstale":"2024-03-21T08:30:35.331670+0000","last_undegraded":"2024-03-21T08:30:35.331670+0000","last_fullsized":"2024-03-21T08:30:35.331670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:32:18.990943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664955+0000","last_change":"2024-03-21T08:28:35.673497+0000","last_active":"2024-03-21T08:30:06.664955+0000","last_peered":"2024-03-21T08:30:06.664955+0000","last_clean":"2024-03-21T08:30:06.664955+0000","last_became_active":"2024-03-21T08:28:35.673045+0000","last_became_peered":"2024-03-21T08:28:35.673045+0000","last_unstale":"2024-03-21T08:30:06.664955+0000","last_undegraded":"2024-03-21T08:30:06.664955+0000","last_fullsized":"2024-03-21T08:30:06.664955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:23.972672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":370,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.653387+0000","last_change":"2024-03-21T08:26:58.139094+0000","last_active":"2024-03-21T08:30:06.653387+0000","last_peered":"2024-03-21T08:30:06.653387+0000","last_clean":"2024-03-21T08:30:06.653387+0000","last_became_active":"2024-03-21T08:26:58.137648+0000","last_became_peered":"2024-03-21T08:26:58.137648+0000","last_unstale":"2024-03-21T08:30:06.653387+0000","last_undegraded":"2024-03-21T08:30:06.653387+0000","last_fullsized":"2024-03-21T08:30:06.653387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:22:17.977314+0000","last_clean_scrub_stamp":"2024-03-21T08:22:17.977314+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:57.950111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665774+0000","last_change":"2024-03-21T08:28:40.887865+0000","last_active":"2024-03-21T08:30:06.665774+0000","last_peered":"2024-03-21T08:30:06.665774+0000","last_clean":"2024-03-21T08:30:06.665774+0000","last_became_active":"2024-03-21T08:28:40.885538+0000","last_became_peered":"2024-03-21T08:28:40.885538+0000","last_unstale":"2024-03-21T08:30:06.665774+0000","last_undegraded":"2024-03-21T08:30:06.665774+0000","last_fullsized":"2024-03-21T08:30:06.665774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:26.532005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.309634+0000","last_change":"2024-03-21T08:28:36.668600+0000","last_active":"2024-03-21T08:30:35.309634+0000","last_peered":"2024-03-21T08:30:35.309634+0000","last_clean":"2024-03-21T08:30:35.309634+0000","last_became_active":"2024-03-21T08:28:36.668188+0000","last_became_peered":"2024-03-21T08:28:36.668188+0000","last_unstale":"2024-03-21T08:30:35.309634+0000","last_undegraded":"2024-03-21T08:30:35.309634+0000","last_fullsized":"2024-03-21T08:30:35.309634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:13.536003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.674753+0000","last_change":"2024-03-21T08:30:06.673425+0000","last_active":"2024-03-21T08:30:06.674753+0000","last_peered":"2024-03-21T08:30:06.674753+0000","last_clean":"2024-03-21T08:30:06.674753+0000","last_became_active":"2024-03-21T08:30:06.666858+0000","last_became_peered":"2024-03-21T08:30:06.666858+0000","last_unstale":"2024-03-21T08:30:06.674753+0000","last_undegraded":"2024-03-21T08:30:06.674753+0000","last_fullsized":"2024-03-21T08:30:06.674753+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:37.159978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.655579+0000","last_change":"2024-03-21T08:29:07.032622+0000","last_active":"2024-03-21T08:30:06.655579+0000","last_peered":"2024-03-21T08:30:06.655579+0000","last_clean":"2024-03-21T08:30:06.655579+0000","last_became_active":"2024-03-21T08:29:07.030884+0000","last_became_peered":"2024-03-21T08:29:07.030884+0000","last_unstale":"2024-03-21T08:30:06.655579+0000","last_undegraded":"2024-03-21T08:30:06.655579+0000","last_fullsized":"2024-03-21T08:30:06.655579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:47.615776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.654098+0000","last_change":"2024-03-21T08:28:35.672948+0000","last_active":"2024-03-21T08:30:06.654098+0000","last_peered":"2024-03-21T08:30:06.654098+0000","last_clean":"2024-03-21T08:30:06.654098+0000","last_became_active":"2024-03-21T08:28:35.671576+0000","last_became_peered":"2024-03-21T08:28:35.671576+0000","last_unstale":"2024-03-21T08:30:06.654098+0000","last_undegraded":"2024-03-21T08:30:06.654098+0000","last_fullsized":"2024-03-21T08:30:06.654098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:36:25.620392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.308174+0000","last_change":"2024-03-21T08:28:36.671173+0000","last_active":"2024-03-21T08:30:35.308174+0000","last_peered":"2024-03-21T08:30:35.308174+0000","last_clean":"2024-03-21T08:30:35.308174+0000","last_became_active":"2024-03-21T08:28:36.666264+0000","last_became_peered":"2024-03-21T08:28:36.666264+0000","last_unstale":"2024-03-21T08:30:35.308174+0000","last_undegraded":"2024-03-21T08:30:35.308174+0000","last_fullsized":"2024-03-21T08:30:35.308174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:35.081487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646351+0000","last_change":"2024-03-21T08:28:40.883801+0000","last_active":"2024-03-21T08:30:06.646351+0000","last_peered":"2024-03-21T08:30:06.646351+0000","last_clean":"2024-03-21T08:30:06.646351+0000","last_became_active":"2024-03-21T08:28:40.882556+0000","last_became_peered":"2024-03-21T08:28:40.882556+0000","last_unstale":"2024-03-21T08:30:06.646351+0000","last_undegraded":"2024-03-21T08:30:06.646351+0000","last_fullsized":"2024-03-21T08:30:06.646351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:24.211625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'10","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:49.542985+0000","last_change":"2024-03-21T08:28:35.668657+0000","last_active":"2024-03-21T08:30:49.542985+0000","last_peered":"2024-03-21T08:30:49.542985+0000","last_clean":"2024-03-21T08:30:49.542985+0000","last_became_active":"2024-03-21T08:28:35.668284+0000","last_became_peered":"2024-03-21T08:28:35.668284+0000","last_unstale":"2024-03-21T08:30:49.542985+0000","last_undegraded":"2024-03-21T08:30:49.542985+0000","last_fullsized":"2024-03-21T08:30:49.542985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:26:07.791388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316596+0000","last_change":"2024-03-21T08:28:36.652383+0000","last_active":"2024-03-21T08:30:35.316596+0000","last_peered":"2024-03-21T08:30:35.316596+0000","last_clean":"2024-03-21T08:30:35.316596+0000","last_became_active":"2024-03-21T08:28:36.652046+0000","last_became_peered":"2024-03-21T08:28:36.652046+0000","last_unstale":"2024-03-21T08:30:35.316596+0000","last_undegraded":"2024-03-21T08:30:35.316596+0000","last_fullsized":"2024-03-21T08:30:35.316596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:11:08.699391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.644497+0000","last_change":"2024-03-21T08:28:40.686122+0000","last_active":"2024-03-21T08:30:06.644497+0000","last_peered":"2024-03-21T08:30:06.644497+0000","last_clean":"2024-03-21T08:30:06.644497+0000","last_became_active":"2024-03-21T08:28:40.685736+0000","last_became_peered":"2024-03-21T08:28:40.685736+0000","last_unstale":"2024-03-21T08:30:06.644497+0000","last_undegraded":"2024-03-21T08:30:06.644497+0000","last_fullsized":"2024-03-21T08:30:06.644497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:52.941896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.665257+0000","last_change":"2024-03-21T08:28:35.675197+0000","last_active":"2024-03-21T08:30:06.665257+0000","last_peered":"2024-03-21T08:30:06.665257+0000","last_clean":"2024-03-21T08:30:06.665257+0000","last_became_active":"2024-03-21T08:28:35.674688+0000","last_became_peered":"2024-03-21T08:28:35.674688+0000","last_unstale":"2024-03-21T08:30:06.665257+0000","last_undegraded":"2024-03-21T08:30:06.665257+0000","last_fullsized":"2024-03-21T08:30:06.665257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:01:11.489294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.328753+0000","last_change":"2024-03-21T08:28:36.668160+0000","last_active":"2024-03-21T08:30:35.328753+0000","last_peered":"2024-03-21T08:30:35.328753+0000","last_clean":"2024-03-21T08:30:35.328753+0000","last_became_active":"2024-03-21T08:28:36.665152+0000","last_became_peered":"2024-03-21T08:28:36.665152+0000","last_unstale":"2024-03-21T08:30:35.328753+0000","last_undegraded":"2024-03-21T08:30:35.328753+0000","last_fullsized":"2024-03-21T08:30:35.328753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:43:41.803822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663608+0000","last_change":"2024-03-21T08:28:40.884365+0000","last_active":"2024-03-21T08:30:06.663608+0000","last_peered":"2024-03-21T08:30:06.663608+0000","last_clean":"2024-03-21T08:30:06.663608+0000","last_became_active":"2024-03-21T08:28:40.883654+0000","last_became_peered":"2024-03-21T08:28:40.883654+0000","last_unstale":"2024-03-21T08:30:06.663608+0000","last_undegraded":"2024-03-21T08:30:06.663608+0000","last_fullsized":"2024-03-21T08:30:06.663608+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:46:43.888121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.672790+0000","last_change":"2024-03-21T08:30:06.672111+0000","last_active":"2024-03-21T08:30:06.672790+0000","last_peered":"2024-03-21T08:30:06.672790+0000","last_clean":"2024-03-21T08:30:06.672790+0000","last_became_active":"2024-03-21T08:30:06.665627+0000","last_became_peered":"2024-03-21T08:30:06.665627+0000","last_unstale":"2024-03-21T08:30:06.672790+0000","last_undegraded":"2024-03-21T08:30:06.672790+0000","last_fullsized":"2024-03-21T08:30:06.672790+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:05.512276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.307537+0000","last_change":"2024-03-21T08:28:36.653797+0000","last_active":"2024-03-21T08:30:35.307537+0000","last_peered":"2024-03-21T08:30:35.307537+0000","last_clean":"2024-03-21T08:30:35.307537+0000","last_became_active":"2024-03-21T08:28:36.651872+0000","last_became_peered":"2024-03-21T08:28:36.651872+0000","last_unstale":"2024-03-21T08:30:35.307537+0000","last_undegraded":"2024-03-21T08:30:35.307537+0000","last_fullsized":"2024-03-21T08:30:35.307537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:09.467573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649747+0000","last_change":"2024-03-21T08:28:40.692996+0000","last_active":"2024-03-21T08:30:06.649747+0000","last_peered":"2024-03-21T08:30:06.649747+0000","last_clean":"2024-03-21T08:30:06.649747+0000","last_became_active":"2024-03-21T08:28:40.687598+0000","last_became_peered":"2024-03-21T08:28:40.687598+0000","last_unstale":"2024-03-21T08:30:06.649747+0000","last_undegraded":"2024-03-21T08:30:06.649747+0000","last_fullsized":"2024-03-21T08:30:06.649747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:09.557272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.645955+0000","last_change":"2024-03-21T08:28:35.672567+0000","last_active":"2024-03-21T08:30:06.645955+0000","last_peered":"2024-03-21T08:30:06.645955+0000","last_clean":"2024-03-21T08:30:06.645955+0000","last_became_active":"2024-03-21T08:28:35.671849+0000","last_became_peered":"2024-03-21T08:28:35.671849+0000","last_unstale":"2024-03-21T08:30:06.645955+0000","last_undegraded":"2024-03-21T08:30:06.645955+0000","last_fullsized":"2024-03-21T08:30:06.645955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:13.050227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.320698+0000","last_change":"2024-03-21T08:28:36.664110+0000","last_active":"2024-03-21T08:30:35.320698+0000","last_peered":"2024-03-21T08:30:35.320698+0000","last_clean":"2024-03-21T08:30:35.320698+0000","last_became_active":"2024-03-21T08:28:36.663297+0000","last_became_peered":"2024-03-21T08:28:36.663297+0000","last_unstale":"2024-03-21T08:30:35.320698+0000","last_undegraded":"2024-03-21T08:30:35.320698+0000","last_fullsized":"2024-03-21T08:30:35.320698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:55.450399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663838+0000","last_change":"2024-03-21T08:28:40.682371+0000","last_active":"2024-03-21T08:30:06.663838+0000","last_peered":"2024-03-21T08:30:06.663838+0000","last_clean":"2024-03-21T08:30:06.663838+0000","last_became_active":"2024-03-21T08:28:40.681786+0000","last_became_peered":"2024-03-21T08:28:40.681786+0000","last_unstale":"2024-03-21T08:30:06.663838+0000","last_undegraded":"2024-03-21T08:30:06.663838+0000","last_fullsized":"2024-03-21T08:30:06.663838+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:15:38.130807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"96'2","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:46.232715+0000","last_change":"2024-03-21T08:28:35.685312+0000","last_active":"2024-03-21T08:30:46.232715+0000","last_peered":"2024-03-21T08:30:46.232715+0000","last_clean":"2024-03-21T08:30:46.232715+0000","last_became_active":"2024-03-21T08:28:35.684327+0000","last_became_peered":"2024-03-21T08:28:35.684327+0000","last_unstale":"2024-03-21T08:30:46.232715+0000","last_undegraded":"2024-03-21T08:30:46.232715+0000","last_fullsized":"2024-03-21T08:30:46.232715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:12.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330747+0000","last_change":"2024-03-21T08:28:36.671186+0000","last_active":"2024-03-21T08:30:35.330747+0000","last_peered":"2024-03-21T08:30:35.330747+0000","last_clean":"2024-03-21T08:30:35.330747+0000","last_became_active":"2024-03-21T08:28:36.662726+0000","last_became_peered":"2024-03-21T08:28:36.662726+0000","last_unstale":"2024-03-21T08:30:35.330747+0000","last_undegraded":"2024-03-21T08:30:35.330747+0000","last_fullsized":"2024-03-21T08:30:35.330747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:29.129014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.663764+0000","last_change":"2024-03-21T08:28:40.885203+0000","last_active":"2024-03-21T08:30:06.663764+0000","last_peered":"2024-03-21T08:30:06.663764+0000","last_clean":"2024-03-21T08:30:06.663764+0000","last_became_active":"2024-03-21T08:28:40.883810+0000","last_became_peered":"2024-03-21T08:28:40.883810+0000","last_unstale":"2024-03-21T08:30:06.663764+0000","last_undegraded":"2024-03-21T08:30:06.663764+0000","last_fullsized":"2024-03-21T08:30:06.663764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:12:58.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651247+0000","last_change":"2024-03-21T08:28:35.690494+0000","last_active":"2024-03-21T08:30:06.651247+0000","last_peered":"2024-03-21T08:30:06.651247+0000","last_clean":"2024-03-21T08:30:06.651247+0000","last_became_active":"2024-03-21T08:28:35.690071+0000","last_became_peered":"2024-03-21T08:28:35.690071+0000","last_unstale":"2024-03-21T08:30:06.651247+0000","last_undegraded":"2024-03-21T08:30:06.651247+0000","last_fullsized":"2024-03-21T08:30:06.651247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:04:46.713006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.333944+0000","last_change":"2024-03-21T08:28:36.665135+0000","last_active":"2024-03-21T08:30:35.333944+0000","last_peered":"2024-03-21T08:30:35.333944+0000","last_clean":"2024-03-21T08:30:35.333944+0000","last_became_active":"2024-03-21T08:28:36.663700+0000","last_became_peered":"2024-03-21T08:28:36.663700+0000","last_unstale":"2024-03-21T08:30:35.333944+0000","last_undegraded":"2024-03-21T08:30:35.333944+0000","last_fullsized":"2024-03-21T08:30:35.333944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:50:00.604958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.649998+0000","last_change":"2024-03-21T08:28:40.696076+0000","last_active":"2024-03-21T08:30:06.649998+0000","last_peered":"2024-03-21T08:30:06.649998+0000","last_clean":"2024-03-21T08:30:06.649998+0000","last_became_active":"2024-03-21T08:28:40.695556+0000","last_became_peered":"2024-03-21T08:28:40.695556+0000","last_unstale":"2024-03-21T08:30:06.649998+0000","last_undegraded":"2024-03-21T08:30:06.649998+0000","last_fullsized":"2024-03-21T08:30:06.649998+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:42.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.646454+0000","last_change":"2024-03-21T08:28:35.685545+0000","last_active":"2024-03-21T08:30:06.646454+0000","last_peered":"2024-03-21T08:30:06.646454+0000","last_clean":"2024-03-21T08:30:06.646454+0000","last_became_active":"2024-03-21T08:28:35.684795+0000","last_became_peered":"2024-03-21T08:28:35.684795+0000","last_unstale":"2024-03-21T08:30:06.646454+0000","last_undegraded":"2024-03-21T08:30:06.646454+0000","last_fullsized":"2024-03-21T08:30:06.646454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:40.308014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.334201+0000","last_change":"2024-03-21T08:28:36.656276+0000","last_active":"2024-03-21T08:30:35.334201+0000","last_peered":"2024-03-21T08:30:35.334201+0000","last_clean":"2024-03-21T08:30:35.334201+0000","last_became_active":"2024-03-21T08:28:36.655871+0000","last_became_peered":"2024-03-21T08:28:36.655871+0000","last_unstale":"2024-03-21T08:30:35.334201+0000","last_undegraded":"2024-03-21T08:30:35.334201+0000","last_fullsized":"2024-03-21T08:30:35.334201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:22.362165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.651972+0000","last_change":"2024-03-21T08:29:06.783155+0000","last_active":"2024-03-21T08:30:06.651972+0000","last_peered":"2024-03-21T08:30:06.651972+0000","last_clean":"2024-03-21T08:30:06.651972+0000","last_became_active":"2024-03-21T08:29:06.782546+0000","last_became_peered":"2024-03-21T08:29:06.782546+0000","last_unstale":"2024-03-21T08:30:06.651972+0000","last_undegraded":"2024-03-21T08:30:06.651972+0000","last_fullsized":"2024-03-21T08:30:06.651972+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:16.489888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.666681+0000","last_change":"2024-03-21T08:28:35.672783+0000","last_active":"2024-03-21T08:30:06.666681+0000","last_peered":"2024-03-21T08:30:06.666681+0000","last_clean":"2024-03-21T08:30:06.666681+0000","last_became_active":"2024-03-21T08:28:35.672161+0000","last_became_peered":"2024-03-21T08:28:35.672161+0000","last_unstale":"2024-03-21T08:30:06.666681+0000","last_undegraded":"2024-03-21T08:30:06.666681+0000","last_fullsized":"2024-03-21T08:30:06.666681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:37.180625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.315686+0000","last_change":"2024-03-21T08:28:36.659447+0000","last_active":"2024-03-21T08:30:35.315686+0000","last_peered":"2024-03-21T08:30:35.315686+0000","last_clean":"2024-03-21T08:30:35.315686+0000","last_became_active":"2024-03-21T08:28:36.657773+0000","last_became_peered":"2024-03-21T08:28:36.657773+0000","last_unstale":"2024-03-21T08:30:35.315686+0000","last_undegraded":"2024-03-21T08:30:35.315686+0000","last_fullsized":"2024-03-21T08:30:35.315686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:05.324498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.664330+0000","last_change":"2024-03-21T08:28:40.679806+0000","last_active":"2024-03-21T08:30:06.664330+0000","last_peered":"2024-03-21T08:30:06.664330+0000","last_clean":"2024-03-21T08:30:06.664330+0000","last_became_active":"2024-03-21T08:28:40.679329+0000","last_became_peered":"2024-03-21T08:28:40.679329+0000","last_unstale":"2024-03-21T08:30:06.664330+0000","last_undegraded":"2024-03-21T08:30:06.664330+0000","last_fullsized":"2024-03-21T08:30:06.664330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:39.645693+0000","last_clean_scrub_stamp":"2024-03-21T08:28:39.645693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:30:58.297581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:06.675639+0000","last_change":"2024-03-21T08:30:06.675069+0000","last_active":"2024-03-21T08:30:06.675639+0000","last_peered":"2024-03-21T08:30:06.675639+0000","last_clean":"2024-03-21T08:30:06.675639+0000","last_became_active":"2024-03-21T08:30:06.665699+0000","last_became_peered":"2024-03-21T08:30:06.665699+0000","last_unstale":"2024-03-21T08:30:06.675639+0000","last_undegraded":"2024-03-21T08:30:06.675639+0000","last_fullsized":"2024-03-21T08:30:06.675639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:34.591968+0000","last_clean_scrub_stamp":"2024-03-21T08:28:34.591968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:30:07.336423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.319014+0000","last_change":"2024-03-21T08:28:36.649194+0000","last_active":"2024-03-21T08:30:35.319014+0000","last_peered":"2024-03-21T08:30:35.319014+0000","last_clean":"2024-03-21T08:30:35.319014+0000","last_became_active":"2024-03-21T08:28:36.648663+0000","last_became_peered":"2024-03-21T08:28:36.648663+0000","last_unstale":"2024-03-21T08:30:35.319014+0000","last_undegraded":"2024-03-21T08:30:35.319014+0000","last_fullsized":"2024-03-21T08:30:35.319014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:16.296886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313728+0000","last_change":"2024-03-21T08:28:36.664473+0000","last_active":"2024-03-21T08:30:35.313728+0000","last_peered":"2024-03-21T08:30:35.313728+0000","last_clean":"2024-03-21T08:30:35.313728+0000","last_became_active":"2024-03-21T08:28:36.663785+0000","last_became_peered":"2024-03-21T08:28:36.663785+0000","last_unstale":"2024-03-21T08:30:35.313728+0000","last_undegraded":"2024-03-21T08:30:35.313728+0000","last_fullsized":"2024-03-21T08:30:35.313728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:00.409224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.309044+0000","last_change":"2024-03-21T08:28:36.668372+0000","last_active":"2024-03-21T08:30:35.309044+0000","last_peered":"2024-03-21T08:30:35.309044+0000","last_clean":"2024-03-21T08:30:35.309044+0000","last_became_active":"2024-03-21T08:28:36.666227+0000","last_became_peered":"2024-03-21T08:28:36.666227+0000","last_unstale":"2024-03-21T08:30:35.309044+0000","last_undegraded":"2024-03-21T08:30:35.309044+0000","last_fullsized":"2024-03-21T08:30:35.309044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:40:56.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.317658+0000","last_change":"2024-03-21T08:28:36.670770+0000","last_active":"2024-03-21T08:30:35.317658+0000","last_peered":"2024-03-21T08:30:35.317658+0000","last_clean":"2024-03-21T08:30:35.317658+0000","last_became_active":"2024-03-21T08:28:36.670467+0000","last_became_peered":"2024-03-21T08:28:36.670467+0000","last_unstale":"2024-03-21T08:30:35.317658+0000","last_undegraded":"2024-03-21T08:30:35.317658+0000","last_fullsized":"2024-03-21T08:30:35.317658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:48.075189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.329527+0000","last_change":"2024-03-21T08:28:36.669395+0000","last_active":"2024-03-21T08:30:35.329527+0000","last_peered":"2024-03-21T08:30:35.329527+0000","last_clean":"2024-03-21T08:30:35.329527+0000","last_became_active":"2024-03-21T08:28:36.668788+0000","last_became_peered":"2024-03-21T08:28:36.668788+0000","last_unstale":"2024-03-21T08:30:35.329527+0000","last_undegraded":"2024-03-21T08:30:35.329527+0000","last_fullsized":"2024-03-21T08:30:35.329527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:03.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.326209+0000","last_change":"2024-03-21T08:30:06.655160+0000","last_active":"2024-03-21T08:30:35.326209+0000","last_peered":"2024-03-21T08:30:35.326209+0000","last_clean":"2024-03-21T08:30:35.326209+0000","last_became_active":"2024-03-21T08:30:06.654744+0000","last_became_peered":"2024-03-21T08:30:06.654744+0000","last_unstale":"2024-03-21T08:30:35.326209+0000","last_undegraded":"2024-03-21T08:30:35.326209+0000","last_fullsized":"2024-03-21T08:30:35.326209+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:19:54.690562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.308832+0000","last_change":"2024-03-21T08:28:36.666836+0000","last_active":"2024-03-21T08:30:35.308832+0000","last_peered":"2024-03-21T08:30:35.308832+0000","last_clean":"2024-03-21T08:30:35.308832+0000","last_became_active":"2024-03-21T08:28:36.666538+0000","last_became_peered":"2024-03-21T08:28:36.666538+0000","last_unstale":"2024-03-21T08:30:35.308832+0000","last_undegraded":"2024-03-21T08:30:35.308832+0000","last_fullsized":"2024-03-21T08:30:35.308832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:43.353269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.325089+0000","last_change":"2024-03-21T08:28:36.667823+0000","last_active":"2024-03-21T08:30:35.325089+0000","last_peered":"2024-03-21T08:30:35.325089+0000","last_clean":"2024-03-21T08:30:35.325089+0000","last_became_active":"2024-03-21T08:28:36.666096+0000","last_became_peered":"2024-03-21T08:28:36.666096+0000","last_unstale":"2024-03-21T08:30:35.325089+0000","last_undegraded":"2024-03-21T08:30:35.325089+0000","last_fullsized":"2024-03-21T08:30:35.325089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:14:58.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.312965+0000","last_change":"2024-03-21T08:28:36.672164+0000","last_active":"2024-03-21T08:30:35.312965+0000","last_peered":"2024-03-21T08:30:35.312965+0000","last_clean":"2024-03-21T08:30:35.312965+0000","last_became_active":"2024-03-21T08:28:36.671812+0000","last_became_peered":"2024-03-21T08:28:36.671812+0000","last_unstale":"2024-03-21T08:30:35.312965+0000","last_undegraded":"2024-03-21T08:30:35.312965+0000","last_fullsized":"2024-03-21T08:30:35.312965+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:24:19.687108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316596+0000","last_change":"2024-03-21T08:28:36.675056+0000","last_active":"2024-03-21T08:30:35.316596+0000","last_peered":"2024-03-21T08:30:35.316596+0000","last_clean":"2024-03-21T08:30:35.316596+0000","last_became_active":"2024-03-21T08:28:36.674586+0000","last_became_peered":"2024-03-21T08:28:36.674586+0000","last_unstale":"2024-03-21T08:30:35.316596+0000","last_undegraded":"2024-03-21T08:30:35.316596+0000","last_fullsized":"2024-03-21T08:30:35.316596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:04:51.207066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.313977+0000","last_change":"2024-03-21T08:28:36.666370+0000","last_active":"2024-03-21T08:30:35.313977+0000","last_peered":"2024-03-21T08:30:35.313977+0000","last_clean":"2024-03-21T08:30:35.313977+0000","last_became_active":"2024-03-21T08:28:36.665950+0000","last_became_peered":"2024-03-21T08:28:36.665950+0000","last_unstale":"2024-03-21T08:30:35.313977+0000","last_undegraded":"2024-03-21T08:30:35.313977+0000","last_fullsized":"2024-03-21T08:30:35.313977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:49:31.945890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330531+0000","last_change":"2024-03-21T08:29:07.032862+0000","last_active":"2024-03-21T08:30:35.330531+0000","last_peered":"2024-03-21T08:30:35.330531+0000","last_clean":"2024-03-21T08:30:35.330531+0000","last_became_active":"2024-03-21T08:29:07.031784+0000","last_became_peered":"2024-03-21T08:29:07.031784+0000","last_unstale":"2024-03-21T08:30:35.330531+0000","last_undegraded":"2024-03-21T08:30:35.330531+0000","last_fullsized":"2024-03-21T08:30:35.330531+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:34.037495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.330945+0000","last_change":"2024-03-21T08:28:36.660812+0000","last_active":"2024-03-21T08:30:35.330945+0000","last_peered":"2024-03-21T08:30:35.330945+0000","last_clean":"2024-03-21T08:30:35.330945+0000","last_became_active":"2024-03-21T08:28:36.660044+0000","last_became_peered":"2024-03-21T08:28:36.660044+0000","last_unstale":"2024-03-21T08:30:35.330945+0000","last_undegraded":"2024-03-21T08:30:35.330945+0000","last_fullsized":"2024-03-21T08:30:35.330945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:31.598678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-21T08:30:35.316333+0000","last_change":"2024-03-21T08:28:36.669441+0000","last_active":"2024-03-21T08:30:35.316333+0000","last_peered":"2024-03-21T08:30:35.316333+0000","last_clean":"2024-03-21T08:30:35.316333+0000","last_became_active":"2024-03-21T08:28:36.668668+0000","last_became_peered":"2024-03-21T08:28:36.668668+0000","last_unstale":"2024-03-21T08:30:35.316333+0000","last_undegraded":"2024-03-21T08:30:35.316333+0000","last_fullsized":"2024-03-21T08:30:35.316333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:28:35.623032+0000","last_clean_scrub_stamp":"2024-03-21T08:28:35.623032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:04.828297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72517,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":904,"num_read_kb":530,"num_write":181,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":217551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285621,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":534350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547258,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1189500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710786,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":538682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972425,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":533924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266769,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712316,"statfs":{"total":95995035648,"available":95961411584,"internally_reserved":0,"allocated":1699840,"data_stored":1131542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":529784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":597643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":531150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":595122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":529956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":534905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1126615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:31:37.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:31:37.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 -- 172.21.15.3:0/698877349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc105370 msgr2=0x7f08cc10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 --2- 172.21.15.3:0/698877349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc105370 0x7f08cc10bc00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009ca0 tx=0x7f08c402c100 comp rx=0 tx=0).stop 2024-03-21T08:31:37.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 -- 172.21.15.3:0/698877349 shutdown_connections 2024-03-21T08:31:37.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 --2- 172.21.15.3:0/698877349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc105370 0x7f08cc10bc00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 --2- 172.21.15.3:0/698877349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08cc1049b0 0x7f08cc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 --2- 172.21.15.3:0/698877349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08cc1037c0 0x7f08cc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 -- 172.21.15.3:0/698877349 >> 172.21.15.3:0/698877349 conn(0x7f08cc0fed50 msgr2=0x7f08cc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 -- 172.21.15.3:0/698877349 shutdown_connections 2024-03-21T08:31:37.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.307+0000 7f08d2a00640 1 -- 172.21.15.3:0/698877349 wait complete. 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 Processor -- start 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 -- start start 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08cc1037c0 0x7f08cc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc1049b0 0x7f08cc199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08cc105370 0x7f08cc1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc10f2f0 con 0x7f08cc1049b0 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc10f170 con 0x7f08cc1037c0 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d2a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc10f470 con 0x7f08cc105370 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.308+0000 7f08d0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc1049b0 0x7f08cc199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc1049b0 0x7f08cc199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34266/0 (socket says 172.21.15.3:34266) 2024-03-21T08:31:37.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 -- 172.21.15.3:0/283201232 learned_addr learned my addr 172.21.15.3:0/283201232 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d1600640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08cc1037c0 0x7f08cc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d2000640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08cc105370 0x7f08cc1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 -- 172.21.15.3:0/283201232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08cc105370 msgr2=0x7f08cc1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08cc105370 0x7f08cc1a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 -- 172.21.15.3:0/283201232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08cc1037c0 msgr2=0x7f08cc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08cc1037c0 0x7f08cc199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 -- 172.21.15.3:0/283201232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc1a1560 con 0x7f08cc1049b0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d1600640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08cc1037c0 0x7f08cc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08d0c00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc1049b0 0x7f08cc199e80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d770 tx=0x7f08b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.309+0000 7f08ca000640 1 -- 172.21.15.3:0/283201232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8017070 con 0x7f08cc1049b0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.310+0000 7f08ca000640 1 -- 172.21.15.3:0/283201232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08b80054d0 con 0x7f08cc1049b0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.310+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc1a1850 con 0x7f08cc1049b0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.310+0000 7f08ca000640 1 -- 172.21.15.3:0/283201232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004cf0 con 0x7f08cc1049b0 2024-03-21T08:31:37.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.310+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc1a1da0 con 0x7f08cc1049b0 2024-03-21T08:31:37.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.311+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc10aae0 con 0x7f08cc1049b0 2024-03-21T08:31:37.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.314+0000 7f08ca000640 1 -- 172.21.15.3:0/283201232 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f08b8004e90 con 0x7f08cc1049b0 2024-03-21T08:31:37.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.316+0000 7f08ca000640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f08b40a2f90 0x7f08b40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.316+0000 7f08ca000640 1 -- 172.21.15.3:0/283201232 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f08b80cf220 con 0x7f08cc1049b0 2024-03-21T08:31:37.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.317+0000 7f08d1600640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f08b40a2f90 0x7f08b40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.317+0000 7f08d1600640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f08b40a2f90 0x7f08b40a5450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005d00 tx=0x7f08c0005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.321+0000 7f08ca000640 1 -- 172.21.15.3:0/283201232 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08b809bc10 con 0x7f08cc1049b0 2024-03-21T08:31:37.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:37 smithi003 ceph-mon[31675]: pgmap v461: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-21T08:31:37.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:37 smithi003 ceph-mon[31675]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:37.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.529+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08cc0622a0 con 0x7f08cc1049b0 2024-03-21T08:31:37.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.533+0000 7f08ca000640 1 -- 172.21.15.3:0/283201232 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7f08b809b510 con 0x7f08cc1049b0 2024-03-21T08:31:37.535 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:37.536 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f08b40a2f90 msgr2=0x7f08b40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f08b40a2f90 0x7f08b40a5450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005d00 tx=0x7f08c0005c90 comp rx=0 tx=0).stop 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc1049b0 msgr2=0x7f08cc199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc1049b0 0x7f08cc199e80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d770 tx=0x7f08b800dc30 comp rx=0 tx=0).stop 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 shutdown_connections 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f08b40a2f90 0x7f08b40a5450 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08cc105370 0x7f08cc1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08cc1049b0 0x7f08cc199e80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 --2- 172.21.15.3:0/283201232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08cc1037c0 0x7f08cc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 >> 172.21.15.3:0/283201232 conn(0x7f08cc0fed50 msgr2=0x7f08cc107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:37.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 shutdown_connections 2024-03-21T08:31:37.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.537+0000 7f08d2a00640 1 -- 172.21.15.3:0/283201232 wait complete. 2024-03-21T08:31:37.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:31:37.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:31:37.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:37 smithi084 ceph-mon[35477]: pgmap v461: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-21T08:31:37.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:37 smithi084 ceph-mon[35477]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:37.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:37 smithi171 ceph-mon[35388]: pgmap v461: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-21T08:31:37.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:37 smithi171 ceph-mon[35388]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.790+0000 7fabfd800640 1 -- 172.21.15.3:0/3573296257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8077c40 msgr2=0x7fabf80784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.790+0000 7fabfd800640 1 --2- 172.21.15.3:0/3573296257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8077c40 0x7fabf80784e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009900 tx=0x7fabf002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.791+0000 7fabfd800640 1 -- 172.21.15.3:0/3573296257 shutdown_connections 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.791+0000 7fabfd800640 1 --2- 172.21.15.3:0/3573296257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8077c40 0x7fabf80784e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.791+0000 7fabfd800640 1 --2- 172.21.15.3:0/3573296257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf80772a0 0x7fabf8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.791+0000 7fabfd800640 1 --2- 172.21.15.3:0/3573296257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf8076060 0x7fabf8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.791+0000 7fabfd800640 1 -- 172.21.15.3:0/3573296257 >> 172.21.15.3:0/3573296257 conn(0x7fabf80fee50 msgr2=0x7fabf8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.791+0000 7fabfd800640 1 -- 172.21.15.3:0/3573296257 shutdown_connections 2024-03-21T08:31:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.791+0000 7fabfd800640 1 -- 172.21.15.3:0/3573296257 wait complete. 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 Processor -- start 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 -- start start 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8076060 0x7fabf819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf80772a0 0x7fabf819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf8077c40 0x7fabf81a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8113820 con 0x7fabf8076060 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf81136a0 con 0x7fabf80772a0 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabfd800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf81139a0 con 0x7fabf8077c40 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.792+0000 7fabf6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8076060 0x7fabf819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8076060 0x7fabf819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34284/0 (socket says 172.21.15.3:34284) 2024-03-21T08:31:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 -- 172.21.15.3:0/1125171222 learned_addr learned my addr 172.21.15.3:0/1125171222 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6000640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf80772a0 0x7fabf819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf7400640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf8077c40 0x7fabf81a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 -- 172.21.15.3:0/1125171222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf8077c40 msgr2=0x7fabf81a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf8077c40 0x7fabf81a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 -- 172.21.15.3:0/1125171222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf80772a0 msgr2=0x7fabf819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf80772a0 0x7fabf819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 -- 172.21.15.3:0/1125171222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf81a5ad0 con 0x7fabf8076060 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.793+0000 7fabf6a00640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8076060 0x7fabf819deb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabe800eab0 tx=0x7fabe800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.794+0000 7fabef400640 1 -- 172.21.15.3:0/1125171222 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800ce40 con 0x7fabf8076060 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.794+0000 7fabef400640 1 -- 172.21.15.3:0/1125171222 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fabe8011920 con 0x7fabf8076060 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.794+0000 7fabef400640 1 -- 172.21.15.3:0/1125171222 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8010a00 con 0x7fabf8076060 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.794+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf81a5dc0 con 0x7fabf8076060 2024-03-21T08:31:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.794+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf81a6230 con 0x7fabf8076060 2024-03-21T08:31:37.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.795+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf8110450 con 0x7fabf8076060 2024-03-21T08:31:37.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.797+0000 7fabef400640 1 -- 172.21.15.3:0/1125171222 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fabe8011ad0 con 0x7fabf8076060 2024-03-21T08:31:37.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.798+0000 7fabef400640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fabdc0a2f20 0x7fabdc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:37.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.798+0000 7fabef400640 1 -- 172.21.15.3:0/1125171222 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fabe80cfea0 con 0x7fabf8076060 2024-03-21T08:31:37.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.799+0000 7fabf6000640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fabdc0a2f20 0x7fabdc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:37.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.799+0000 7fabf6000640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fabdc0a2f20 0x7fabdc0a53e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fabf819f3c0 tx=0x7fabe400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:37.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:37.804+0000 7fabef400640 1 -- 172.21.15.3:0/1125171222 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fabe8016030 con 0x7fabf8076060 2024-03-21T08:31:38.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.005+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabf8061ae0 con 0x7fabf8076060 2024-03-21T08:31:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.006+0000 7fabef400640 1 -- 172.21.15.3:0/1125171222 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabe809c830 con 0x7fabf8076060 2024-03-21T08:31:38.008 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fabdc0a2f20 msgr2=0x7fabdc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fabdc0a2f20 0x7fabdc0a53e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fabf819f3c0 tx=0x7fabe400b390 comp rx=0 tx=0).stop 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8076060 msgr2=0x7fabf819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8076060 0x7fabf819deb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabe800eab0 tx=0x7fabe800ef70 comp rx=0 tx=0).stop 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 shutdown_connections 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fabdc0a2f20 0x7fabdc0a53e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf8077c40 0x7fabf81a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf80772a0 0x7fabf819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 --2- 172.21.15.3:0/1125171222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabf8076060 0x7fabf819deb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 >> 172.21.15.3:0/1125171222 conn(0x7fabf80fee50 msgr2=0x7fabf81005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:38.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 shutdown_connections 2024-03-21T08:31:38.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.009+0000 7fabfd800640 1 -- 172.21.15.3:0/1125171222 wait complete. 2024-03-21T08:31:38.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:31:38.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.262+0000 7fb810a00640 1 -- 172.21.15.3:0/4196777622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c100730 msgr2=0x7fb80c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.262+0000 7fb810a00640 1 --2- 172.21.15.3:0/4196777622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c100730 0x7fb80c100b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009900 tx=0x7fb7f802bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.262+0000 7fb810a00640 1 -- 172.21.15.3:0/4196777622 shutdown_connections 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.262+0000 7fb810a00640 1 --2- 172.21.15.3:0/4196777622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80c1010d0 0x7fb80c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.262+0000 7fb810a00640 1 --2- 172.21.15.3:0/4196777622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c100730 0x7fb80c100b90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.262+0000 7fb810a00640 1 --2- 172.21.15.3:0/4196777622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb80c0ffdd0 0x7fb80c1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.262+0000 7fb810a00640 1 -- 172.21.15.3:0/4196777622 >> 172.21.15.3:0/4196777622 conn(0x7fb80c0fb990 msgr2=0x7fb80c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.263+0000 7fb810a00640 1 -- 172.21.15.3:0/4196777622 shutdown_connections 2024-03-21T08:31:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.263+0000 7fb810a00640 1 -- 172.21.15.3:0/4196777622 wait complete. 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 Processor -- start 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 -- start start 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80c0ffdd0 0x7fb80c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb80c100730 0x7fb80c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1010d0 0x7fb80c1a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c10f240 con 0x7fb80c1010d0 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c10f0c0 con 0x7fb80c0ffdd0 2024-03-21T08:31:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.264+0000 7fb810a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c10f3c0 con 0x7fb80c100730 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1010d0 0x7fb80c1a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1010d0 0x7fb80c1a0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34314/0 (socket says 172.21.15.3:34314) 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 -- 172.21.15.3:0/4215463407 learned_addr learned my addr 172.21.15.3:0/4215463407 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80a800640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80c0ffdd0 0x7fb80c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb809e00640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb80c100730 0x7fb80c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 -- 172.21.15.3:0/4215463407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb80c100730 msgr2=0x7fb80c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb80c100730 0x7fb80c199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 -- 172.21.15.3:0/4215463407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80c0ffdd0 msgr2=0x7fb80c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80c0ffdd0 0x7fb80c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.265+0000 7fb80b200640 1 -- 172.21.15.3:0/4215463407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c1a1520 con 0x7fb80c1010d0 2024-03-21T08:31:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.266+0000 7fb80b200640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1010d0 0x7fb80c1a0eb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb80400eb10 tx=0x7fb80400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.267+0000 7fb803400640 1 -- 172.21.15.3:0/4215463407 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804004b20 con 0x7fb80c1010d0 2024-03-21T08:31:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.267+0000 7fb803400640 1 -- 172.21.15.3:0/4215463407 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb804004cc0 con 0x7fb80c1010d0 2024-03-21T08:31:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.267+0000 7fb803400640 1 -- 172.21.15.3:0/4215463407 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804010690 con 0x7fb80c1010d0 2024-03-21T08:31:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.267+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c1a1810 con 0x7fb80c1010d0 2024-03-21T08:31:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.267+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c1a1ce0 con 0x7fb80c1010d0 2024-03-21T08:31:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.269+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c10ab20 con 0x7fb80c1010d0 2024-03-21T08:31:38.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.271+0000 7fb803400640 1 -- 172.21.15.3:0/4215463407 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fb804005020 con 0x7fb80c1010d0 2024-03-21T08:31:38.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.273+0000 7fb803400640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb7f00a2f20 0x7fb7f00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.273+0000 7fb803400640 1 -- 172.21.15.3:0/4215463407 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fb8040cfd10 con 0x7fb80c1010d0 2024-03-21T08:31:38.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.273+0000 7fb80a800640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb7f00a2f20 0x7fb7f00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.274+0000 7fb80a800640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb7f00a2f20 0x7fb7f00a53e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc005d00 tx=0x7fb7fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:38.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.278+0000 7fb803400640 1 -- 172.21.15.3:0/4215463407 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb80409c700 con 0x7fb80c1010d0 2024-03-21T08:31:38.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:38 smithi003 ceph-mon[31675]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:38.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:38 smithi003 ceph-mon[31675]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:38.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:38 smithi003 ceph-mon[31675]: pgmap v462: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-21T08:31:38.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:38 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/283201232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:38.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:38 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1125171222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:38.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.484+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb80c0622a0 con 0x7fb80c1010d0 2024-03-21T08:31:38.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.485+0000 7fb803400640 1 -- 172.21.15.3:0/4215463407 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb80409c000 con 0x7fb80c1010d0 2024-03-21T08:31:38.487 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb7f00a2f20 msgr2=0x7fb7f00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb7f00a2f20 0x7fb7f00a53e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc005d00 tx=0x7fb7fc00b410 comp rx=0 tx=0).stop 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1010d0 msgr2=0x7fb80c1a0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1010d0 0x7fb80c1a0eb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb80400eb10 tx=0x7fb80400c2d0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 shutdown_connections 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb80c1010d0 0x7fb80c1a0eb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fb7f00a2f20 0x7fb7f00a53e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb80c100730 0x7fb80c199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 --2- 172.21.15.3:0/4215463407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb80c0ffdd0 0x7fb80c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 >> 172.21.15.3:0/4215463407 conn(0x7fb80c0fb990 msgr2=0x7fb80c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:38.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 shutdown_connections 2024-03-21T08:31:38.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.488+0000 7fb810a00640 1 -- 172.21.15.3:0/4215463407 wait complete. 2024-03-21T08:31:38.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:31:38.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:38 smithi084 ceph-mon[35477]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:38.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:38 smithi084 ceph-mon[35477]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:38.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:38 smithi084 ceph-mon[35477]: pgmap v462: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-21T08:31:38.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:38 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/283201232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:38.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:38 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1125171222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:38.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:38 smithi171 ceph-mon[35388]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:38.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:38 smithi171 ceph-mon[35388]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:31:38.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:38 smithi171 ceph-mon[35388]: pgmap v462: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-21T08:31:38.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:38 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/283201232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:38.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:38 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1125171222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:31:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.737+0000 7fe4f7200640 1 -- 172.21.15.3:0/4093136101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f8076060 msgr2=0x7fe4f8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.737+0000 7fe4f7200640 1 --2- 172.21.15.3:0/4093136101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f8076060 0x7fe4f8076480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009900 tx=0x7fe4e802bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.737+0000 7fe4f7200640 1 -- 172.21.15.3:0/4093136101 shutdown_connections 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.737+0000 7fe4f7200640 1 --2- 172.21.15.3:0/4093136101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8077c40 0x7fe4f80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.737+0000 7fe4f7200640 1 --2- 172.21.15.3:0/4093136101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4f80772a0 0x7fe4f8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.737+0000 7fe4f7200640 1 --2- 172.21.15.3:0/4093136101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f8076060 0x7fe4f8076480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.737+0000 7fe4f7200640 1 -- 172.21.15.3:0/4093136101 >> 172.21.15.3:0/4093136101 conn(0x7fe4f80fec70 msgr2=0x7fe4f81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.738+0000 7fe4f7200640 1 -- 172.21.15.3:0/4093136101 shutdown_connections 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.738+0000 7fe4f7200640 1 -- 172.21.15.3:0/4093136101 wait complete. 2024-03-21T08:31:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 Processor -- start 2024-03-21T08:31:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 -- start start 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8076060 0x7fe4f819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f80772a0 0x7fe4f819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4f8077c40 0x7fe4f81a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f8113640 con 0x7fe4f8076060 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f81134c0 con 0x7fe4f80772a0 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.739+0000 7fe4f7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f81137c0 con 0x7fe4f8077c40 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8076060 0x7fe4f819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8076060 0x7fe4f819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34332/0 (socket says 172.21.15.3:34332) 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 -- 172.21.15.3:0/2399622065 learned_addr learned my addr 172.21.15.3:0/2399622065 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5400640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f80772a0 0x7fe4f819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f6800640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4f8077c40 0x7fe4f81a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 -- 172.21.15.3:0/2399622065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4f8077c40 msgr2=0x7fe4f81a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4f8077c40 0x7fe4f81a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 -- 172.21.15.3:0/2399622065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f80772a0 msgr2=0x7fe4f819e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f80772a0 0x7fe4f819e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.740+0000 7fe4f5e00640 1 -- 172.21.15.3:0/2399622065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f81a59c0 con 0x7fe4f8076060 2024-03-21T08:31:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.741+0000 7fe4f5e00640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8076060 0x7fe4f819ddb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8006010 tx=0x7fe4e8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.741+0000 7fe4eea00640 1 -- 172.21.15.3:0/2399622065 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8043020 con 0x7fe4f8076060 2024-03-21T08:31:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.741+0000 7fe4eea00640 1 -- 172.21.15.3:0/2399622065 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4e8004440 con 0x7fe4f8076060 2024-03-21T08:31:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.741+0000 7fe4eea00640 1 -- 172.21.15.3:0/2399622065 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8003bf0 con 0x7fe4f8076060 2024-03-21T08:31:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.741+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f81a5c50 con 0x7fe4f8076060 2024-03-21T08:31:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.741+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f81a6150 con 0x7fe4f8076060 2024-03-21T08:31:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.743+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f804f400 con 0x7fe4f8076060 2024-03-21T08:31:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.745+0000 7fe4eea00640 1 -- 172.21.15.3:0/2399622065 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fe4e8046020 con 0x7fe4f8076060 2024-03-21T08:31:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.746+0000 7fe4eea00640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.746+0000 7fe4eea00640 1 -- 172.21.15.3:0/2399622065 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe4e80f1d00 con 0x7fe4f8076060 2024-03-21T08:31:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.746+0000 7fe4f5400640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.747+0000 7fe4f5400640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400bbe0 tx=0x7fe4e400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.750+0000 7fe4eea00640 1 -- 172.21.15.3:0/2399622065 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe4e804d020 con 0x7fe4f8076060 2024-03-21T08:31:38.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.956+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4f80622a0 con 0x7fe4f8076060 2024-03-21T08:31:38.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.957+0000 7fe4eea00640 1 -- 172.21.15.3:0/2399622065 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4e80be690 con 0x7fe4f8076060 2024-03-21T08:31:38.959 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe4dc0a2f20 msgr2=0x7fe4dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400bbe0 tx=0x7fe4e400b390 comp rx=0 tx=0).stop 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8076060 msgr2=0x7fe4f819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8076060 0x7fe4f819ddb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8006010 tx=0x7fe4e8004040 comp rx=0 tx=0).stop 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 shutdown_connections 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4f8077c40 0x7fe4f81a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f80772a0 0x7fe4f819e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 --2- 172.21.15.3:0/2399622065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4f8076060 0x7fe4f819ddb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 >> 172.21.15.3:0/2399622065 conn(0x7fe4f80fec70 msgr2=0x7fe4f8100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 shutdown_connections 2024-03-21T08:31:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:38.960+0000 7fe4f7200640 1 -- 172.21.15.3:0/2399622065 wait complete. 2024-03-21T08:31:38.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:31:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 -- 172.21.15.3:0/3267309791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 msgr2=0x7f2d00073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 --2- 172.21.15.3:0/3267309791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00073ad0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009be0 tx=0x7f2cf002bf80 comp rx=0 tx=0).stop 2024-03-21T08:31:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 -- 172.21.15.3:0/3267309791 shutdown_connections 2024-03-21T08:31:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 --2- 172.21.15.3:0/3267309791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d00109370 0x7f2d0010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 --2- 172.21.15.3:0/3267309791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d000740a0 0x7f2d00074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 --2- 172.21.15.3:0/3267309791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00073ad0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 -- 172.21.15.3:0/3267309791 >> 172.21.15.3:0/3267309791 conn(0x7f2d000fca50 msgr2=0x7f2d000fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 -- 172.21.15.3:0/3267309791 shutdown_connections 2024-03-21T08:31:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.211+0000 7f2cffe00640 1 -- 172.21.15.3:0/3267309791 wait complete. 2024-03-21T08:31:39.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.212+0000 7f2cffe00640 1 Processor -- start 2024-03-21T08:31:39.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.212+0000 7f2cffe00640 1 -- start start 2024-03-21T08:31:39.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d000740a0 0x7f2d00199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d00109370 0x7f2d001a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cffe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0010f250 con 0x7f2d00075680 2024-03-21T08:31:39.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cffe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0010f0d0 con 0x7f2d00109370 2024-03-21T08:31:39.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cffe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0010f3d0 con 0x7f2d000740a0 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34352/0 (socket says 172.21.15.3:34352) 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 -- 172.21.15.3:0/2207362220 learned_addr learned my addr 172.21.15.3:0/2207362220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cff400640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d00109370 0x7f2d001a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfea00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d000740a0 0x7f2d00199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 -- 172.21.15.3:0/2207362220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d000740a0 msgr2=0x7f2d00199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d000740a0 0x7f2d00199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 -- 172.21.15.3:0/2207362220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d00109370 msgr2=0x7f2d001a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d00109370 0x7f2d001a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.213+0000 7f2cfe000640 1 -- 172.21.15.3:0/2207362220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d001a1570 con 0x7f2d00075680 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.214+0000 7f2cfea00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d000740a0 0x7f2d00199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.214+0000 7f2cfe000640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00199e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e9c0 tx=0x7f2ce800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.214+0000 7f2cf7400640 1 -- 172.21.15.3:0/2207362220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800cde0 con 0x7f2d00075680 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.214+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d001a1860 con 0x7f2d00075680 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.214+0000 7f2cf7400640 1 -- 172.21.15.3:0/2207362220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ce80035f0 con 0x7f2d00075680 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.214+0000 7f2cf7400640 1 -- 172.21.15.3:0/2207362220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8018630 con 0x7f2d00075680 2024-03-21T08:31:39.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.214+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d001a1e00 con 0x7f2d00075680 2024-03-21T08:31:39.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.215+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d00108130 con 0x7f2d00075680 2024-03-21T08:31:39.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.217+0000 7f2cf7400640 1 -- 172.21.15.3:0/2207362220 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f2ce80037a0 con 0x7f2d00075680 2024-03-21T08:31:39.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.219+0000 7f2cf7400640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2ce40a2f20 0x7f2ce40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.219+0000 7f2cf7400640 1 -- 172.21.15.3:0/2207362220 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f2ce80cfbd0 con 0x7f2d00075680 2024-03-21T08:31:39.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.219+0000 7f2cfea00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2ce40a2f20 0x7f2ce40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.220+0000 7f2cfea00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2ce40a2f20 0x7f2ce40a53e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00099a0 tx=0x7f2cf0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:39.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.223+0000 7f2cf7400640 1 -- 172.21.15.3:0/2207362220 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2ce809c560 con 0x7f2d00075680 2024-03-21T08:31:39.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.427+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2d000622a0 con 0x7f2d00075680 2024-03-21T08:31:39.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.429+0000 7f2cf7400640 1 -- 172.21.15.3:0/2207362220 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2ce8009510 con 0x7f2d00075680 2024-03-21T08:31:39.431 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2ce40a2f20 msgr2=0x7f2ce40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2ce40a2f20 0x7f2ce40a53e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00099a0 tx=0x7f2cf0000f40 comp rx=0 tx=0).stop 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 msgr2=0x7f2d00199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00199e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e9c0 tx=0x7f2ce800ee80 comp rx=0 tx=0).stop 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 shutdown_connections 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d00109370 0x7f2d001a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d00075680 0x7f2d00199e90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f2ce40a2f20 0x7f2ce40a53e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 --2- 172.21.15.3:0/2207362220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d000740a0 0x7f2d00199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.431+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 >> 172.21.15.3:0/2207362220 conn(0x7f2d000fca50 msgr2=0x7f2d00105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.432+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 shutdown_connections 2024-03-21T08:31:39.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.432+0000 7f2cffe00640 1 -- 172.21.15.3:0/2207362220 wait complete. 2024-03-21T08:31:39.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:31:39.538 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:39 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4215463407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:39.539 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:39 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2399622065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:39.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:39 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4215463407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:39.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:39 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2399622065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:39.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:39 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4215463407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:31:39.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:39 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2399622065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:31:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 -- 172.21.15.3:0/2808555906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c102030 msgr2=0x7f320c104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 --2- 172.21.15.3:0/2808555906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c102030 0x7f320c104420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3200009900 tx=0x7f320002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 -- 172.21.15.3:0/2808555906 shutdown_connections 2024-03-21T08:31:39.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 --2- 172.21.15.3:0/2808555906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f320c107290 0x7f320c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 --2- 172.21.15.3:0/2808555906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f320c104960 0x7f320c106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 --2- 172.21.15.3:0/2808555906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c102030 0x7f320c104420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 -- 172.21.15.3:0/2808555906 >> 172.21.15.3:0/2808555906 conn(0x7f320c0fba20 msgr2=0x7f320c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 -- 172.21.15.3:0/2808555906 shutdown_connections 2024-03-21T08:31:39.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.679+0000 7f3212200640 1 -- 172.21.15.3:0/2808555906 wait complete. 2024-03-21T08:31:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.680+0000 7f3212200640 1 Processor -- start 2024-03-21T08:31:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.680+0000 7f3212200640 1 -- start start 2024-03-21T08:31:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f3212200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f320c102030 0x7f320c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f3212200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c104960 0x7f320c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f3212200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f320c107290 0x7f320c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f3212200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c10d0f0 con 0x7f320c104960 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f3212200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c10cf70 con 0x7f320c107290 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f3212200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c10d270 con 0x7f320c102030 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f320be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c104960 0x7f320c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f320be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c104960 0x7f320c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34362/0 (socket says 172.21.15.3:34362) 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f320be00640 1 -- 172.21.15.3:0/1979579684 learned_addr learned my addr 172.21.15.3:0/1979579684 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:39.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.681+0000 7f3211800640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f320c107290 0x7f320c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f3210e00640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f320c102030 0x7f320c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f320be00640 1 -- 172.21.15.3:0/1979579684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f320c102030 msgr2=0x7f320c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f320be00640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f320c102030 0x7f320c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f320be00640 1 -- 172.21.15.3:0/1979579684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f320c107290 msgr2=0x7f320c1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f320be00640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f320c107290 0x7f320c1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f320be00640 1 -- 172.21.15.3:0/1979579684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c1a16a0 con 0x7f320c104960 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f320be00640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c104960 0x7f320c199fc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f31f800e9c0 tx=0x7f31f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.682+0000 7f3209600640 1 -- 172.21.15.3:0/1979579684 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800cd80 con 0x7f320c104960 2024-03-21T08:31:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.683+0000 7f3209600640 1 -- 172.21.15.3:0/1979579684 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f31f80054d0 con 0x7f320c104960 2024-03-21T08:31:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.683+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c1a1990 con 0x7f320c104960 2024-03-21T08:31:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.683+0000 7f3209600640 1 -- 172.21.15.3:0/1979579684 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8010430 con 0x7f320c104960 2024-03-21T08:31:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.683+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1a1f30 con 0x7f320c104960 2024-03-21T08:31:39.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.684+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c108de0 con 0x7f320c104960 2024-03-21T08:31:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.686+0000 7f3209600640 1 -- 172.21.15.3:0/1979579684 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f31f8004fd0 con 0x7f320c104960 2024-03-21T08:31:39.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.690+0000 7f3209600640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f31f40a2f90 0x7f31f40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:39.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.690+0000 7f3209600640 1 -- 172.21.15.3:0/1979579684 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f31f80cfc60 con 0x7f320c104960 2024-03-21T08:31:39.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.690+0000 7f3210e00640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f31f40a2f90 0x7f31f40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.691+0000 7f3210e00640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f31f40a2f90 0x7f31f40a5450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32000096c0 tx=0x7f3200000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:39.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.694+0000 7f3209600640 1 -- 172.21.15.3:0/1979579684 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f31f809c5f0 con 0x7f320c104960 2024-03-21T08:31:39.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.945+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f320c02d070 con 0x7f320c104960 2024-03-21T08:31:39.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.946+0000 7f3209600640 1 -- 172.21.15.3:0/1979579684 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f31f809bef0 con 0x7f320c104960 2024-03-21T08:31:39.949 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:39.949 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:31:29.351404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34472,34409]},"id":1}]} 2024-03-21T08:31:39.950 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-03-21T08:31:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f31f40a2f90 msgr2=0x7f31f40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f31f40a2f90 0x7f31f40a5450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32000096c0 tx=0x7f3200000bc0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c104960 msgr2=0x7f320c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c104960 0x7f320c199fc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f31f800e9c0 tx=0x7f31f800ee80 comp rx=0 tx=0).stop 2024-03-21T08:31:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 shutdown_connections 2024-03-21T08:31:39.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f320c107290 0x7f320c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f320c104960 0x7f320c199fc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f31f40a2f90 0x7f31f40a5450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 --2- 172.21.15.3:0/1979579684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f320c102030 0x7f320c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.949+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 >> 172.21.15.3:0/1979579684 conn(0x7f320c0fba20 msgr2=0x7f320c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.950+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 shutdown_connections 2024-03-21T08:31:39.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:39.950+0000 7f3212200640 1 -- 172.21.15.3:0/1979579684 wait complete. 2024-03-21T08:31:39.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:31:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 -- 172.21.15.3:0/3955768730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0102150 msgr2=0x7f86a0104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 --2- 172.21.15.3:0/3955768730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0102150 0x7f86a0104540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8694009900 tx=0x7f869402bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 -- 172.21.15.3:0/3955768730 shutdown_connections 2024-03-21T08:31:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 --2- 172.21.15.3:0/3955768730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a01073b0 0x7f86a0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 --2- 172.21.15.3:0/3955768730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a0104a80 0x7f86a0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 --2- 172.21.15.3:0/3955768730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0102150 0x7f86a0104540 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 -- 172.21.15.3:0/3955768730 >> 172.21.15.3:0/3955768730 conn(0x7f86a00fbb40 msgr2=0x7f86a00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 -- 172.21.15.3:0/3955768730 shutdown_connections 2024-03-21T08:31:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.166+0000 7f86a8200640 1 -- 172.21.15.3:0/3955768730 wait complete. 2024-03-21T08:31:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.167+0000 7f86a8200640 1 Processor -- start 2024-03-21T08:31:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a8200640 1 -- start start 2024-03-21T08:31:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a8200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a0102150 0x7f86a0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a8200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0104a80 0x7f86a019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a8200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a01073b0 0x7f86a01a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a8200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a010d280 con 0x7f86a0104a80 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a8200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a010d100 con 0x7f86a01073b0 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a8200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a010d400 con 0x7f86a0102150 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0104a80 0x7f86a019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0104a80 0x7f86a019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53250/0 (socket says 172.21.15.3:53250) 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a4e00640 1 -- 172.21.15.3:0/535747257 learned_addr learned my addr 172.21.15.3:0/535747257 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a6200640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a01073b0 0x7f86a01a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.168+0000 7f86a5800640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a0102150 0x7f86a0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a4e00640 1 -- 172.21.15.3:0/535747257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a0102150 msgr2=0x7f86a0199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a4e00640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a0102150 0x7f86a0199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a4e00640 1 -- 172.21.15.3:0/535747257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a01073b0 msgr2=0x7f86a01a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a4e00640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a01073b0 0x7f86a01a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a4e00640 1 -- 172.21.15.3:0/535747257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a01a1770 con 0x7f86a0104a80 2024-03-21T08:31:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a5800640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a0102150 0x7f86a0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a4e00640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0104a80 0x7f86a019a090 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f868c00d740 tx=0x7f868c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f869e000640 1 -- 172.21.15.3:0/535747257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c017070 con 0x7f86a0104a80 2024-03-21T08:31:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f869e000640 1 -- 172.21.15.3:0/535747257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f868c00de50 con 0x7f86a0104a80 2024-03-21T08:31:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a01a1a60 con 0x7f86a0104a80 2024-03-21T08:31:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f869e000640 1 -- 172.21.15.3:0/535747257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c004d60 con 0x7f86a0104a80 2024-03-21T08:31:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.169+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a01a1f60 con 0x7f86a0104a80 2024-03-21T08:31:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.171+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a0108f00 con 0x7f86a0104a80 2024-03-21T08:31:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.174+0000 7f869e000640 1 -- 172.21.15.3:0/535747257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f868c004f00 con 0x7f86a0104a80 2024-03-21T08:31:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.175+0000 7f869e000640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f86880a2f20 0x7f86880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.175+0000 7f869e000640 1 -- 172.21.15.3:0/535747257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f868c0cf6c0 con 0x7f86a0104a80 2024-03-21T08:31:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.176+0000 7f86a5800640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f86880a2f20 0x7f86880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.176+0000 7f86a5800640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f86880a2f20 0x7f86880a53e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8694036040 tx=0x7f8694005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.178+0000 7f869e000640 1 -- 172.21.15.3:0/535747257 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f868c09c0b0 con 0x7f86a0104a80 2024-03-21T08:31:40.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.387+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86a002d270 con 0x7f86a0104a80 2024-03-21T08:31:40.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.390+0000 7f869e000640 1 -- 172.21.15.3:0/535747257 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24880 (secure 0 0 0) 0x7f868c09b9b0 con 0x7f86a0104a80 2024-03-21T08:31:40.393 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:31:40.393 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:30:06.635311+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f86880a2f20 msgr2=0x7f86880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f86880a2f20 0x7f86880a53e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8694036040 tx=0x7f8694005fd0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0104a80 msgr2=0x7f86a019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0104a80 0x7f86a019a090 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f868c00d740 tx=0x7f868c00dc00 comp rx=0 tx=0).stop 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 shutdown_connections 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a01073b0 0x7f86a01a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86a0104a80 0x7f86a019a090 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.392+0000 7f86a8200640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f86880a2f20 0x7f86880a53e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.393+0000 7f86a8200640 1 --2- 172.21.15.3:0/535747257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a0102150 0x7f86a0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.393+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 >> 172.21.15.3:0/535747257 conn(0x7f86a00fbb40 msgr2=0x7f86a0101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.393+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 shutdown_connections 2024-03-21T08:31:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.393+0000 7f86a8200640 1 -- 172.21.15.3:0/535747257 wait complete. 2024-03-21T08:31:40.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:40 smithi003 ceph-mon[31675]: pgmap v463: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-21T08:31:40.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:40 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2207362220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:40.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:40 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1979579684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:40.414 DEBUG:tasks.fwd_scrub:Yielding 2024-03-21T08:31:40.414 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T08:31:40.425 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T08:31:40.425 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:31:40.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-21T08:31:40.426 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-21T08:31:40.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T08:31:40.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:31:40.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T08:31:40.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.617+0000 7f355a200640 1 -- 172.21.15.3:0/3451303380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 msgr2=0x7f35541021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.617+0000 7f355a200640 1 --2- 172.21.15.3:0/3451303380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f35541021b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3548009900 tx=0x7f354802bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:40.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.618+0000 7f355a200640 1 -- 172.21.15.3:0/3451303380 shutdown_connections 2024-03-21T08:31:40.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.618+0000 7f355a200640 1 --2- 172.21.15.3:0/3451303380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35541050a0 0x7f3554105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.618+0000 7f355a200640 1 --2- 172.21.15.3:0/3451303380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541026f0 0x7f3554104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.618+0000 7f355a200640 1 --2- 172.21.15.3:0/3451303380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f35541021b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.618+0000 7f355a200640 1 -- 172.21.15.3:0/3451303380 >> 172.21.15.3:0/3451303380 conn(0x7f35540fb9a0 msgr2=0x7f35540fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.618+0000 7f355a200640 1 -- 172.21.15.3:0/3451303380 shutdown_connections 2024-03-21T08:31:40.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.618+0000 7f355a200640 1 -- 172.21.15.3:0/3451303380 wait complete. 2024-03-21T08:31:40.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 Processor -- start 2024-03-21T08:31:40.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 -- start start 2024-03-21T08:31:40.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f355419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35541026f0 0x7f355419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541050a0 0x7f35541a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3554113660 con 0x7f35540ffd80 2024-03-21T08:31:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35541134e0 con 0x7f35541050a0 2024-03-21T08:31:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.619+0000 7f355a200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35541137e0 con 0x7f35541026f0 2024-03-21T08:31:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f355419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f355419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53262/0 (socket says 172.21.15.3:53262) 2024-03-21T08:31:40.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 -- 172.21.15.3:0/366986128 learned_addr learned my addr 172.21.15.3:0/366986128 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3559800640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541050a0 0x7f35541a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3553e00640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35541026f0 0x7f355419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 -- 172.21.15.3:0/366986128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35541026f0 msgr2=0x7f355419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35541026f0 0x7f355419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 -- 172.21.15.3:0/366986128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541050a0 msgr2=0x7f35541a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541050a0 0x7f35541a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.620+0000 7f3558e00640 1 -- 172.21.15.3:0/366986128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35541a5960 con 0x7f35540ffd80 2024-03-21T08:31:40.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.621+0000 7f3558e00640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f355419dda0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3548000c00 tx=0x7f35480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.621+0000 7f3551600640 1 -- 172.21.15.3:0/366986128 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548043020 con 0x7f35540ffd80 2024-03-21T08:31:40.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.621+0000 7f3551600640 1 -- 172.21.15.3:0/366986128 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3548004aa0 con 0x7f35540ffd80 2024-03-21T08:31:40.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.621+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35541a5bf0 con 0x7f35540ffd80 2024-03-21T08:31:40.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.621+0000 7f3551600640 1 -- 172.21.15.3:0/366986128 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548003d70 con 0x7f35540ffd80 2024-03-21T08:31:40.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.621+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35541a6140 con 0x7f35540ffd80 2024-03-21T08:31:40.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:40 smithi084 ceph-mon[35477]: pgmap v463: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-21T08:31:40.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:40 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2207362220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:40.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:40 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1979579684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:40.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.623+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35540fd490 con 0x7f35540ffd80 2024-03-21T08:31:40.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.624+0000 7f3551600640 1 -- 172.21.15.3:0/366986128 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f3548046020 con 0x7f35540ffd80 2024-03-21T08:31:40.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.627+0000 7f3551600640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f353c0a2f00 0x7f353c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.627+0000 7f3551600640 1 -- 172.21.15.3:0/366986128 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f35480f1d30 con 0x7f35540ffd80 2024-03-21T08:31:40.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.627+0000 7f3553e00640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f353c0a2f00 0x7f353c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.628+0000 7f3553e00640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f353c0a2f00 0x7f353c0a53c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3540009b60 tx=0x7f3540005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.631+0000 7f3551600640 1 -- 172.21.15.3:0/366986128 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f354804d020 con 0x7f35540ffd80 2024-03-21T08:31:40.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:40 smithi171 ceph-mon[35388]: pgmap v463: 129 pgs: 129 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-21T08:31:40.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:40 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2207362220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:31:40.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:40 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1979579684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:31:40.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.632+0000 7fbf26400640 1 -- 172.21.15.3:0/877829445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20104ad0 msgr2=0x7fbf20104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.632+0000 7fbf26400640 1 --2- 172.21.15.3:0/877829445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20104ad0 0x7fbf20104f50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009900 tx=0x7fbf0c02bc50 comp rx=0 tx=0).stop 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.633+0000 7fbf26400640 1 -- 172.21.15.3:0/877829445 shutdown_connections 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.633+0000 7fbf26400640 1 --2- 172.21.15.3:0/877829445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf20105490 0x7fbf2010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.633+0000 7fbf26400640 1 --2- 172.21.15.3:0/877829445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20104ad0 0x7fbf20104f50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.633+0000 7fbf26400640 1 --2- 172.21.15.3:0/877829445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf201038e0 0x7fbf20103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.633+0000 7fbf26400640 1 -- 172.21.15.3:0/877829445 >> 172.21.15.3:0/877829445 conn(0x7fbf200fee50 msgr2=0x7fbf20101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.633+0000 7fbf26400640 1 -- 172.21.15.3:0/877829445 shutdown_connections 2024-03-21T08:31:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.633+0000 7fbf26400640 1 -- 172.21.15.3:0/877829445 wait complete. 2024-03-21T08:31:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 Processor -- start 2024-03-21T08:31:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 -- start start 2024-03-21T08:31:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf201038e0 0x7fbf20199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf20104ad0 0x7fbf2019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20105490 0x7fbf201a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2010f440 con 0x7fbf20105490 2024-03-21T08:31:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2010f2c0 con 0x7fbf20104ad0 2024-03-21T08:31:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf26400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2010f5c0 con 0x7fbf201038e0 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.634+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20105490 0x7fbf201a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20105490 0x7fbf201a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53282/0 (socket says 172.21.15.3:53282) 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 -- 172.21.15.3:0/236870795 learned_addr learned my addr 172.21.15.3:0/236870795 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1f400640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf201038e0 0x7fbf20199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf20104ad0 0x7fbf2019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 -- 172.21.15.3:0/236870795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf201038e0 msgr2=0x7fbf20199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf201038e0 0x7fbf20199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 -- 172.21.15.3:0/236870795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf20104ad0 msgr2=0x7fbf2019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf20104ad0 0x7fbf2019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 -- 172.21.15.3:0/236870795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf201a1760 con 0x7fbf20105490 2024-03-21T08:31:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.635+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20105490 0x7fbf201a10f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009fd0 tx=0x7fbf1800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.636+0000 7fbf17e00640 1 -- 172.21.15.3:0/236870795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18017070 con 0x7fbf20105490 2024-03-21T08:31:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.636+0000 7fbf17e00640 1 -- 172.21.15.3:0/236870795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf18003e20 con 0x7fbf20105490 2024-03-21T08:31:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.636+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf201a1a50 con 0x7fbf20105490 2024-03-21T08:31:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.636+0000 7fbf17e00640 1 -- 172.21.15.3:0/236870795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18010460 con 0x7fbf20105490 2024-03-21T08:31:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.636+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf201a1f20 con 0x7fbf20105490 2024-03-21T08:31:40.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.639+0000 7fbf17e00640 1 -- 172.21.15.3:0/236870795 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7fbf18003970 con 0x7fbf20105490 2024-03-21T08:31:40.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.639+0000 7fbf291d0740 1 Processor -- start 2024-03-21T08:31:40.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf291d0740 1 -- start start 2024-03-21T08:31:40.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31a8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf291d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b3108eb0 con 0x5577b31a3a90 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf291d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b30b1a60 con 0x5577b31a30a0 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf291d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b3104f50 con 0x5577b31a8c00 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53298/0 (socket says 172.21.15.3:53298) 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf1f400640 1 -- 172.21.15.3:0/1090601467 learned_addr learned my addr 172.21.15.3:0/1090601467 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.640+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1f400640 1 -- 172.21.15.3:0/1090601467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 msgr2=0x5577b31ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1f400640 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1f400640 1 -- 172.21.15.3:0/1090601467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 msgr2=0x5577b31a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1f400640 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1f400640 1 -- 172.21.15.3:0/1090601467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577b2ee57e0 con 0x5577b31a3a90 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf17e00640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf040a2f20 0x7fbf040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf17e00640 1 -- 172.21.15.3:0/236870795 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fbf180cf700 con 0x7fbf20105490 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1f400640 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31a8590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000bd10 tx=0x7fbf1000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6040e7a109754bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.641+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf040a2f20 0x7fbf040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.642+0000 7fbf14c00640 1 -- 172.21.15.3:0/1090601467 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10015070 con 0x5577b31a3a90 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.642+0000 7fbf14c00640 1 -- 172.21.15.3:0/1090601467 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf1001a910 con 0x5577b31a3a90 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.642+0000 7fbf14c00640 1 -- 172.21.15.3:0/1090601467 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10011860 con 0x5577b31a3a90 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 -- 172.21.15.3:0/1090601467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 msgr2=0x5577b31a8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf040a2f20 0x7fbf040a53e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c006010 tx=0x7fbf0c02c5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31a8590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000bd10 tx=0x7fbf1000f730 comp rx=0 tx=0).stop 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 -- 172.21.15.3:0/1090601467 shutdown_connections 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31a8590 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1090601467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 -- 172.21.15.3:0/1090601467 >> 172.21.15.3:0/1090601467 conn(0x5577b319e610 msgr2=0x5577b319ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 -- 172.21.15.3:0/1090601467 shutdown_connections 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.643+0000 7fbf291d0740 1 -- 172.21.15.3:0/1090601467 wait complete. 2024-03-21T08:31:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.644+0000 7fbf291d0740 1 -- start start 2024-03-21T08:31:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.644+0000 7fbf291d0740 1 Processor -- start 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.644+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf291d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b31abbb0 con 0x5577b31a3a90 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf291d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b30c4eb0 con 0x5577b31a30a0 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf291d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b31abd30 con 0x5577b31a8c00 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf291d0740 10 client.0 _ll_register_callbacks cb 0x5577b30b9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53308/0 (socket says 172.21.15.3:53308) 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 -- 172.21.15.3:0/698531258 learned_addr learned my addr 172.21.15.3:0/698531258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1f400640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 -- 172.21.15.3:0/698531258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 msgr2=0x5577b31b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 -- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 msgr2=0x5577b31ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1ea00640 1 -- 172.21.15.3:0/698531258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577b2f45f70 con 0x5577b31a3a90 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.645+0000 7fbf1f400640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.646+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31ad7c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c02ce30 tx=0x7fbf0c02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80616d8e773ba446 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.646+0000 7fbf09600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.646+0000 7fbf09600640 1 -- 172.21.15.3:0/698531258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c051020 con 0x5577b31a3a90 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.646+0000 7fbf09600640 1 -- 172.21.15.3:0/698531258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fbf0c005050 con 0x5577b31a3a90 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.646+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf1001a910 con 0x5577b31a3a90 2024-03-21T08:31:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.646+0000 7fbf09600640 1 -- 172.21.15.3:0/698531258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf0c05c050 con 0x5577b31a3a90 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 10 client.15621 fetch_fsmap learned FSMap version 23 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5577b2ddd170 con 0x5577b31a3a90 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf09600640 1 -- 172.21.15.3:0/698531258 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf1001a910 con 0x5577b31a3a90 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf09600640 1 -- 172.21.15.3:0/698531258 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbf0c040040 con 0x5577b31a3a90 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 10 client.15621 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 20 client.15621 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b31b9130 0x5577b31bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 4 client.15621 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:31:40.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.647+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577b31163f0 con 0x5577b31b9130 2024-03-21T08:31:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.648+0000 7fbf1f400640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b31b9130 0x5577b31bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.649+0000 7fbf1f400640 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b31b9130 0x5577b31bb5f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb035a856650ae0 server_cookie=1864d82927c4132c in_seq=0 out_seq=0 2024-03-21T08:31:40.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.649+0000 7fbf09600640 10 client.15621 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf09600640 1 -- 172.21.15.3:0/698531258 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5577b31163f0 con 0x5577b31b9130 2024-03-21T08:31:40.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf09600640 10 client.15621 handle_command_reply: tid=0 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 client.15621 shutdown 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 msgr2=0x5577b31ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31ad7c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c02ce30 tx=0x7fbf0c02bc50 comp rx=0 tx=0).stop 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 shutdown_connections 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b31b9130 0x5577b31bb5f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31ad7c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 --2- 172.21.15.3:0/698531258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.651+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 >> 172.21.15.3:0/698531258 conn(0x5577b319e610 msgr2=0x5577b319d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.652+0000 7fbf09600640 0 client.15621 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.652+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 shutdown_connections 2024-03-21T08:31:40.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.652+0000 7fbf291d0740 1 -- 172.21.15.3:0/698531258 wait complete. 2024-03-21T08:31:40.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.652+0000 7fbf291d0740 20 client.15621 trim_cache size 0 max 16384 2024-03-21T08:31:40.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 Processor -- start 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 -- start start 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b319d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b319dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201aea70 con 0x5577b31a3a90 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201ac570 con 0x5577b31a30a0 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.672+0000 7fbf291d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201aa610 con 0x5577b31a8c00 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.673+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b319d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.673+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b319d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:38972/0 (socket says 172.21.15.3:38972) 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.673+0000 7fbf1f400640 1 -- 172.21.15.3:0/1092610826 learned_addr learned my addr 172.21.15.3:0/1092610826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.673+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b319dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.673+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.674+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1092610826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 msgr2=0x5577b31b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.674+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.674+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1092610826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 msgr2=0x5577b319d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.674+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b319d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.674+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1092610826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577b319f380 con 0x5577b31a3a90 2024-03-21T08:31:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.674+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b319dbf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c004eb0 tx=0x7fbf0c003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d381c8c8907bf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.674+0000 7fbf0be00640 1 -- 172.21.15.3:0/1092610826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c065070 con 0x5577b31a3a90 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.675+0000 7fbf0be00640 1 -- 172.21.15.3:0/1092610826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf0c048070 con 0x5577b31a3a90 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.675+0000 7fbf0be00640 1 -- 172.21.15.3:0/1092610826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c051050 con 0x5577b31a3a90 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 -- 172.21.15.3:0/1092610826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 msgr2=0x5577b319dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b319dbf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c004eb0 tx=0x7fbf0c003a60 comp rx=0 tx=0).stop 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 -- 172.21.15.3:0/1092610826 shutdown_connections 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b319dbf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1092610826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b319d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 -- 172.21.15.3:0/1092610826 >> 172.21.15.3:0/1092610826 conn(0x5577b319e610 msgr2=0x5577b319cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 -- 172.21.15.3:0/1092610826 shutdown_connections 2024-03-21T08:31:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 -- 172.21.15.3:0/1092610826 wait complete. 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.676+0000 7fbf291d0740 1 -- start start 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 1 Processor -- start 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201aac10 con 0x5577b31a3a90 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201abf10 con 0x5577b31a30a0 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201ab250 con 0x5577b31a8c00 2024-03-21T08:31:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.677+0000 7fbf291d0740 10 client.0 _ll_register_callbacks cb 0x7fbf0c02c900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:40.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53328/0 (socket says 172.21.15.3:53328) 2024-03-21T08:31:40.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1411880661 learned_addr learned my addr 172.21.15.3:0/1411880661 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1411880661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 msgr2=0x5577b31b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1411880661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 msgr2=0x5577b31b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.678+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1411880661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577b31b3540 con 0x5577b31a3a90 2024-03-21T08:31:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.679+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31b2ac0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c043460 tx=0x7fbf0c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32906d1c05959542 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.679+0000 7fbf09600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.679+0000 7fbf09600640 1 -- 172.21.15.3:0/1411880661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c065030 con 0x5577b31a3a90 2024-03-21T08:31:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.679+0000 7fbf09600640 1 -- 172.21.15.3:0/1411880661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fbf0c005540 con 0x5577b31a3a90 2024-03-21T08:31:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.679+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf0c048070 con 0x5577b31a3a90 2024-03-21T08:31:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.679+0000 7fbf09600640 1 -- 172.21.15.3:0/1411880661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf0c051040 con 0x5577b31a3a90 2024-03-21T08:31:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 10 client.15627 fetch_fsmap learned FSMap version 23 2024-03-21T08:31:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5577b3215700 con 0x5577b31a3a90 2024-03-21T08:31:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf09600640 1 -- 172.21.15.3:0/1411880661 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf0c048070 con 0x5577b31a3a90 2024-03-21T08:31:40.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf09600640 1 -- 172.21.15.3:0/1411880661 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbf0c043900 con 0x5577b31a3a90 2024-03-21T08:31:40.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 10 client.15627 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 20 client.15627 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b3215fd0 0x5577b3218430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 4 client.15627 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.680+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5577b3110740 con 0x5577b3215fd0 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.681+0000 7fbf1f400640 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b3215fd0 0x5577b3218430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.682+0000 7fbf1f400640 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b3215fd0 0x5577b3218430 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78190fb40a496977 server_cookie=872c16d3226ee830 in_seq=0 out_seq=0 2024-03-21T08:31:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.682+0000 7fbf09600640 10 client.15627 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.683+0000 7fbf09600640 1 -- 172.21.15.3:0/1411880661 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5577b3110740 con 0x5577b3215fd0 2024-03-21T08:31:40.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.683+0000 7fbf09600640 10 client.15627 handle_command_reply: tid=0 2024-03-21T08:31:40.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.683+0000 7fbf291d0740 1 client.15627 shutdown 2024-03-21T08:31:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 msgr2=0x5577b31b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31b2ac0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c043460 tx=0x7fbf0c005c90 comp rx=0 tx=0).stop 2024-03-21T08:31:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 shutdown_connections 2024-03-21T08:31:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5577b3215fd0 0x5577b3218430 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577b31a8c00 0x5577b31b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577b31a3a90 0x5577b31b2ac0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 --2- 172.21.15.3:0/1411880661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b31a30a0 0x5577b31b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 >> 172.21.15.3:0/1411880661 conn(0x5577b319e610 msgr2=0x5577b319d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf09600640 0 client.15627 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 shutdown_connections 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.684+0000 7fbf291d0740 1 -- 172.21.15.3:0/1411880661 wait complete. 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf291d0740 20 client.15627 trim_cache size 0 max 16384 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-03-21T08:31:40.687 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "19dec71f-54bc-4e43-b5b4-e5e8e7d58ad4", 2024-03-21T08:31:40.688 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-03-21T08:31:40.688 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:31:40.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf040a2f20 msgr2=0x7fbf040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf040a2f20 0x7fbf040a53e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c006010 tx=0x7fbf0c02c5f0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20105490 msgr2=0x7fbf201a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20105490 0x7fbf201a10f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009fd0 tx=0x7fbf1800b260 comp rx=0 tx=0).stop 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 shutdown_connections 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf20105490 0x7fbf201a10f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf20104ad0 0x7fbf2019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7fbf040a2f20 0x7fbf040a53e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 --2- 172.21.15.3:0/236870795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf201038e0 0x7fbf20199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.685+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 >> 172.21.15.3:0/236870795 conn(0x7fbf200fee50 msgr2=0x7fbf20107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.686+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 shutdown_connections 2024-03-21T08:31:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.686+0000 7fbf26400640 1 -- 172.21.15.3:0/236870795 wait complete. 2024-03-21T08:31:40.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:31:40.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2801261655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6410dac0 msgr2=0x7f7d6410feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2801261655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6410dac0 0x7f7d6410feb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009cd0 tx=0x7f7d5c02c0e0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2801261655 shutdown_connections 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2801261655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6410dac0 0x7f7d6410feb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2801261655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64074100 0x7f7d64074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2801261655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d640756e0 0x7f7d64073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2801261655 >> 172.21.15.3:0/2801261655 conn(0x7f7d640fca60 msgr2=0x7f7d640feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2801261655 shutdown_connections 2024-03-21T08:31:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.900+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2801261655 wait complete. 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 Processor -- start 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 -- start start 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64074100 0x7f7d6419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d640756e0 0x7f7d6419e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d6410dac0 0x7f7d641a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64113610 con 0x7f7d640756e0 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64113490 con 0x7f7d64074100 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.901+0000 7f7d6aa00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64113790 con 0x7f7d6410dac0 2024-03-21T08:31:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d640756e0 0x7f7d6419e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d640756e0 0x7f7d6419e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53356/0 (socket says 172.21.15.3:53356) 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 -- 172.21.15.3:0/2291812741 learned_addr learned my addr 172.21.15.3:0/2291812741 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d69600640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64074100 0x7f7d6419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d6a000640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d6410dac0 0x7f7d641a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 -- 172.21.15.3:0/2291812741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d6410dac0 msgr2=0x7f7d641a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d6410dac0 0x7f7d641a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 -- 172.21.15.3:0/2291812741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64074100 msgr2=0x7f7d6419dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64074100 0x7f7d6419dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.902+0000 7f7d68c00640 1 -- 172.21.15.3:0/2291812741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d641a58f0 con 0x7f7d640756e0 2024-03-21T08:31:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.903+0000 7f7d68c00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d640756e0 0x7f7d6419e240 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000d950 tx=0x7f7d5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.903+0000 7f7d62000640 1 -- 172.21.15.3:0/2291812741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50017030 con 0x7f7d640756e0 2024-03-21T08:31:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.903+0000 7f7d62000640 1 -- 172.21.15.3:0/2291812741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d50004020 con 0x7f7d640756e0 2024-03-21T08:31:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.903+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d641a5be0 con 0x7f7d640756e0 2024-03-21T08:31:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.903+0000 7f7d62000640 1 -- 172.21.15.3:0/2291812741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500047a0 con 0x7f7d640756e0 2024-03-21T08:31:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.903+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d641a6130 con 0x7f7d640756e0 2024-03-21T08:31:40.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.906+0000 7f7d6ee14740 1 Processor -- start 2024-03-21T08:31:40.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.906+0000 7f7d6ee14740 1 -- start start 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.906+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81a050 0x558b4d81a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81aa40 0x558b4d81f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81fbb0 0x558b4d822040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4d77f250 con 0x558b4d81fbb0 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4d727a30 con 0x558b4d81aa40 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4d77b2f0 con 0x558b4d81a050 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d6a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81a050 0x558b4d81a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d69600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81aa40 0x558b4d81f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d69600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81aa40 0x558b4d81f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:39008/0 (socket says 172.21.15.3:39008) 2024-03-21T08:31:40.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d69600640 1 -- 172.21.15.3:0/3035242103 learned_addr learned my addr 172.21.15.3:0/3035242103 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.907+0000 7f7d6a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81a050 0x558b4d81a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.3:39076/0 (socket says 172.21.15.3:39076) 2024-03-21T08:31:40.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.908+0000 7f7d62000640 1 -- 172.21.15.3:0/2291812741 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 137853+0+0 (secure 0 0 0) 0x7f7d500041d0 con 0x7f7d640756e0 2024-03-21T08:31:40.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.908+0000 7f7d6a000640 1 -- 172.21.15.3:0/3035242103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81aa40 msgr2=0x558b4d81f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.908+0000 7f7d6a000640 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81aa40 0x558b4d81f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.908+0000 7f7d6a000640 1 -- 172.21.15.3:0/3035242103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81fbb0 msgr2=0x558b4d822040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.908+0000 7f7d6a000640 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81fbb0 0x558b4d822040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.908+0000 7f7d6a000640 1 -- 172.21.15.3:0/3035242103 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b4d712690 con 0x558b4d81a050 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.908+0000 7f7d69600640 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81aa40 0x558b4d81f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.909+0000 7f7d6a000640 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81a050 0x558b4d81a470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009bf0 tx=0x7f7d5c005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c748346d8f1f5d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.909+0000 7f7d62000640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7d4c0a2f20 0x7f7d4c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.909+0000 7f7d56600640 1 -- 172.21.15.3:0/3035242103 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c043020 con 0x558b4d81a050 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.909+0000 7f7d62000640 1 -- 172.21.15.3:0/2291812741 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7d500cee10 con 0x7f7d640756e0 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.909+0000 7f7d56600640 1 -- 172.21.15.3:0/3035242103 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d5c032df0 con 0x558b4d81a050 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d56600640 1 -- 172.21.15.3:0/3035242103 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c039040 con 0x558b4d81a050 2024-03-21T08:31:40.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f355402d070 con 0x7f35540ffd80 2024-03-21T08:31:40.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d69600640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7d4c0a2f20 0x7f7d4c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d6ee14740 1 -- 172.21.15.3:0/3035242103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81a050 msgr2=0x558b4d81a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81a050 0x558b4d81a470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009bf0 tx=0x7f7d5c005fa0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d6ee14740 1 -- 172.21.15.3:0/3035242103 shutdown_connections 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81fbb0 0x558b4d822040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81aa40 0x558b4d81f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/3035242103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81a050 0x558b4d81a470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d6ee14740 1 -- 172.21.15.3:0/3035242103 >> 172.21.15.3:0/3035242103 conn(0x558b4d7e2200 msgr2=0x558b4d8159c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.910+0000 7f7d69600640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7d4c0a2f20 0x7f7d4c0a53e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x558b4d81ae40 tx=0x7f7d5800b550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.911+0000 7f7d6ee14740 1 -- 172.21.15.3:0/3035242103 shutdown_connections 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.911+0000 7f7d6ee14740 1 -- 172.21.15.3:0/3035242103 wait complete. 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.911+0000 7f7d6ee14740 1 -- start start 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.911+0000 7f7d6ee14740 1 Processor -- start 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.911+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81a050 0x558b4d824080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81aa40 0x558b4d8245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81fbb0 0x558b4d82b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4d822b00 con 0x558b4d81aa40 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4d73ae80 con 0x558b4d81a050 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4d822c80 con 0x558b4d81fbb0 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d6ee14740 10 client.0 _ll_register_callbacks cb 0x558b4d74d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81aa40 0x558b4d8245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81aa40 0x558b4d8245c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53372/0 (socket says 172.21.15.3:53372) 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 -- 172.21.15.3:0/1530191452 learned_addr learned my addr 172.21.15.3:0/1530191452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d69600640 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81a050 0x558b4d824080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d6a000640 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81fbb0 0x558b4d82b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 -- 172.21.15.3:0/1530191452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81fbb0 msgr2=0x558b4d82b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81fbb0 0x558b4d82b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 -- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81a050 msgr2=0x558b4d824080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81a050 0x558b4d824080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.912+0000 7f7d68c00640 1 -- 172.21.15.3:0/1530191452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b4d55d7d0 con 0x558b4d81aa40 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d68c00640 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81aa40 0x558b4d8245c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7d5009b090 tx=0x7f7d5009b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9e59da61c336e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d4aa00640 1 -- 172.21.15.3:0/1530191452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5009f250 con 0x558b4d81aa40 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d4aa00640 1 -- 172.21.15.3:0/1530191452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7d500a3680 con 0x558b4d81aa40 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d5c032df0 con 0x558b4d81aa40 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d4aa00640 1 -- 172.21.15.3:0/1530191452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d500a3900 con 0x558b4d81aa40 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d6ee14740 10 client.15636 fetch_fsmap learned FSMap version 23 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.913+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558b4d4933f0 con 0x558b4d81aa40 2024-03-21T08:31:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d4aa00640 1 -- 172.21.15.3:0/1530191452 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d5c032df0 con 0x558b4d81aa40 2024-03-21T08:31:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d4aa00640 1 -- 172.21.15.3:0/1530191452 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7d50096e50 con 0x558b4d81aa40 2024-03-21T08:31:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d6ee14740 10 client.15636 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:31:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d6ee14740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:31:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d6ee14740 20 client.15636 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d82fe40 0x558b4d832260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d6ee14740 4 client.15636 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:31:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.914+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b4d78c790 con 0x558b4d82fe40 2024-03-21T08:31:40.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.915+0000 7f7d69600640 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d82fe40 0x558b4d832260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.915+0000 7f7d69600640 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d82fe40 0x558b4d832260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807f42e6c407997a server_cookie=5d300282812ae768 in_seq=0 out_seq=0 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.915+0000 7f7d4aa00640 10 client.15636 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.917+0000 7f7d4aa00640 1 -- 172.21.15.3:0/1530191452 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558b4d78c790 con 0x558b4d82fe40 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.917+0000 7f7d4aa00640 10 client.15636 handle_command_reply: tid=0 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.917+0000 7f7d6ee14740 1 client.15636 shutdown 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.917+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81aa40 msgr2=0x558b4d8245c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.917+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81aa40 0x558b4d8245c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7d5009b090 tx=0x7f7d5009b800 comp rx=0 tx=0).stop 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 shutdown_connections 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d82fe40 0x558b4d832260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81fbb0 0x558b4d82b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81aa40 0x558b4d8245c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/1530191452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81a050 0x558b4d824080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 >> 172.21.15.3:0/1530191452 conn(0x558b4d7e2200 msgr2=0x558b4d814bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d4aa00640 0 client.15636 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 shutdown_connections 2024-03-21T08:31:40.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 1 -- 172.21.15.3:0/1530191452 wait complete. 2024-03-21T08:31:40.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.918+0000 7f7d6ee14740 20 client.15636 trim_cache size 0 max 16384 2024-03-21T08:31:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.935+0000 7f7d6ee14740 1 Processor -- start 2024-03-21T08:31:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.935+0000 7f7d6ee14740 1 -- start start 2024-03-21T08:31:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.935+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d810920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.935+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 0x558b4d82b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d818c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641b2c70 con 0x558b4d81a050 2024-03-21T08:31:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641b0770 con 0x558b4d81fbb0 2024-03-21T08:31:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641ae810 con 0x558b4d81aa40 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d810920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d810920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53388/0 (socket says 172.21.15.3:53388) 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 -- 172.21.15.3:0/709784394 learned_addr learned my addr 172.21.15.3:0/709784394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d6a000640 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d818c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 -- 172.21.15.3:0/709784394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 msgr2=0x558b4d82b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 0x558b4d82b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 -- 172.21.15.3:0/709784394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 msgr2=0x558b4d818c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d818c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.936+0000 7f7d69600640 1 -- 172.21.15.3:0/709784394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b4d4530f0 con 0x558b4d81a050 2024-03-21T08:31:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.937+0000 7f7d69600640 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d810920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x558b4d81b4b0 tx=0x7f7d58015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462c0cd5534e0325 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.937+0000 7f7d55c00640 1 -- 172.21.15.3:0/709784394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58021760 con 0x558b4d81a050 2024-03-21T08:31:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.937+0000 7f7d55c00640 1 -- 172.21.15.3:0/709784394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d5802e070 con 0x558b4d81a050 2024-03-21T08:31:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.937+0000 7f7d55c00640 1 -- 172.21.15.3:0/709784394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58028da0 con 0x558b4d81a050 2024-03-21T08:31:40.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 -- 172.21.15.3:0/709784394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 msgr2=0x558b4d810920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d810920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x558b4d81b4b0 tx=0x7f7d58015000 comp rx=0 tx=0).stop 2024-03-21T08:31:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 -- 172.21.15.3:0/709784394 shutdown_connections 2024-03-21T08:31:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d818c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 0x558b4d82b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/709784394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d810920 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 -- 172.21.15.3:0/709784394 >> 172.21.15.3:0/709784394 conn(0x558b4d7e2200 msgr2=0x558b4d81bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 -- 172.21.15.3:0/709784394 shutdown_connections 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 -- 172.21.15.3:0/709784394 wait complete. 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 -- start start 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.938+0000 7f7d6ee14740 1 Processor -- start 2024-03-21T08:31:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d829320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 0x558b4d829860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6ee14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d82a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641aee10 con 0x558b4d81a050 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641b0110 con 0x558b4d81fbb0 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6ee14740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641af450 con 0x558b4d81aa40 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6ee14740 10 client.0 _ll_register_callbacks cb 0x7f7d500046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d69600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d829320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d69600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d829320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53394/0 (socket says 172.21.15.3:53394) 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d69600640 1 -- 172.21.15.3:0/2493922555 learned_addr learned my addr 172.21.15.3:0/2493922555 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d6a000640 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d82a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d69600640 1 -- 172.21.15.3:0/2493922555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 msgr2=0x558b4d829860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d69600640 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 0x558b4d829860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d69600640 1 -- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 msgr2=0x558b4d82a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.939+0000 7f7d69600640 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d82a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d69600640 1 -- 172.21.15.3:0/2493922555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b4d82ae60 con 0x558b4d81a050 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d6a000640 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d82a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d69600640 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d829320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005850 tx=0x7f7d58028420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3979ba9deb61cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d4aa00640 1 -- 172.21.15.3:0/2493922555 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58036020 con 0x558b4d81a050 2024-03-21T08:31:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d4aa00640 1 -- 172.21.15.3:0/2493922555 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7d58028df0 con 0x558b4d81a050 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d5802e070 con 0x558b4d81a050 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d4aa00640 1 -- 172.21.15.3:0/2493922555 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d58041050 con 0x558b4d81a050 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d6ee14740 10 client.15642 fetch_fsmap learned FSMap version 23 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.940+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558b4d88bd30 con 0x558b4d81a050 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d4aa00640 1 -- 172.21.15.3:0/2493922555 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d5802e070 con 0x558b4d81a050 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d4aa00640 1 -- 172.21.15.3:0/2493922555 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7d580046c0 con 0x558b4d81a050 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d6ee14740 10 client.15642 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:31:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d6ee14740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d6ee14740 20 client.15642 populate_metadata read hostname 'smithi003' 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d88c6b0 0x558b4d88ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d6ee14740 4 client.15642 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.941+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b4d786ae0 con 0x558b4d88c6b0 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.942+0000 7f7d68c00640 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d88c6b0 0x558b4d88ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.942+0000 7f7d68c00640 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d88c6b0 0x558b4d88ead0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27b52ae92faf9a server_cookie=28462adf8c3d488 in_seq=0 out_seq=0 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.942+0000 7f7d4aa00640 10 client.15642 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d4aa00640 1 -- 172.21.15.3:0/2493922555 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558b4d786ae0 con 0x558b4d88c6b0 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d4aa00640 10 client.15642 handle_command_reply: tid=0 2024-03-21T08:31:40.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 client.15642 shutdown 2024-03-21T08:31:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 msgr2=0x558b4d829320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d829320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005850 tx=0x7f7d58028420 comp rx=0 tx=0).stop 2024-03-21T08:31:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 shutdown_connections 2024-03-21T08:31:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4d81fbb0 0x558b4d82a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558b4d88c6b0 0x558b4d88ead0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b4d81aa40 0x558b4d829860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 --2- 172.21.15.3:0/2493922555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558b4d81a050 0x558b4d829320 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d4aa00640 0 client.15642 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:31:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 >> 172.21.15.3:0/2493922555 conn(0x558b4d7e2200 msgr2=0x558b4d814330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 shutdown_connections 2024-03-21T08:31:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 1 -- 172.21.15.3:0/2493922555 wait complete. 2024-03-21T08:31:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.943+0000 7f7d6ee14740 20 client.15642 trim_cache size 0 max 16384 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: "19dec71f-54bc-4e43-b5b4-e5e8e7d58ad4": { 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "19dec71f-54bc-4e43-b5b4-e5e8e7d58ad4", 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:31:40.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:31:40.955 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:31:40.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7d4c0a2f20 msgr2=0x7f7d4c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7d4c0a2f20 0x7f7d4c0a53e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x558b4d81ae40 tx=0x7f7d5800b550 comp rx=0 tx=0).stop 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d640756e0 msgr2=0x7f7d6419e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d640756e0 0x7f7d6419e240 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000d950 tx=0x7f7d5000de10 comp rx=0 tx=0).stop 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 shutdown_connections 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f7d4c0a2f20 0x7f7d4c0a53e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d6410dac0 0x7f7d641a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d640756e0 0x7f7d6419e240 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 --2- 172.21.15.3:0/2291812741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64074100 0x7f7d6419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:40.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 >> 172.21.15.3:0/2291812741 conn(0x7f7d640fca60 msgr2=0x7f7d64100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:40.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 shutdown_connections 2024-03-21T08:31:40.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:40.944+0000 7f7d6aa00640 1 -- 172.21.15.3:0/2291812741 wait complete. 2024-03-21T08:31:40.961 INFO:tasks.cephfs.filesystem:scrub status for tag:19dec71f-54bc-4e43-b5b4-e5e8e7d58ad4 - {'path': '/', 'tag': '19dec71f-54bc-4e43-b5b4-e5e8e7d58ad4', 'options': 'recursive,force'} 2024-03-21T08:31:41.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.297+0000 7f3551600640 1 -- 172.21.15.3:0/366986128 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f35480be6c0 con 0x7f35540ffd80 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f353c0a2f00 msgr2=0x7f353c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f353c0a2f00 0x7f353c0a53c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3540009b60 tx=0x7f3540005f90 comp rx=0 tx=0).stop 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 msgr2=0x7f355419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f355419dda0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3548000c00 tx=0x7f35480042f0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 shutdown_connections 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541050a0 0x7f35541a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f353c0a2f00 0x7f353c0a53c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35541026f0 0x7f355419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 --2- 172.21.15.3:0/366986128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35540ffd80 0x7f355419dda0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 >> 172.21.15.3:0/366986128 conn(0x7f35540fb9a0 msgr2=0x7f3554103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:41.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 shutdown_connections 2024-03-21T08:31:41.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.301+0000 7f355a200640 1 -- 172.21.15.3:0/366986128 wait complete. 2024-03-21T08:31:41.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-21T08:31:41.396 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/535747257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:41.396 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-mon[31675]: scrub queued for path: / 2024-03-21T08:31:41.396 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:31:41.396 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:31:41.396 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:31:41.397 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/366986128' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 -- 172.21.15.3:0/3518782538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c102210 msgr2=0x7f725c104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 --2- 172.21.15.3:0/3518782538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c102210 0x7f725c104600 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f724c009900 tx=0x7f724c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 -- 172.21.15.3:0/3518782538 shutdown_connections 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 --2- 172.21.15.3:0/3518782538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f725c107470 0x7f725c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 --2- 172.21.15.3:0/3518782538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f725c104b40 0x7f725c106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 --2- 172.21.15.3:0/3518782538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c102210 0x7f725c104600 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 -- 172.21.15.3:0/3518782538 >> 172.21.15.3:0/3518782538 conn(0x7f725c0fbb40 msgr2=0x7f725c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 -- 172.21.15.3:0/3518782538 shutdown_connections 2024-03-21T08:31:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.514+0000 7f7262000640 1 -- 172.21.15.3:0/3518782538 wait complete. 2024-03-21T08:31:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 Processor -- start 2024-03-21T08:31:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 -- start start 2024-03-21T08:31:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f725c102210 0x7f725c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c104b40 0x7f725c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f725c107470 0x7f725c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c10d280 con 0x7f725c104b40 2024-03-21T08:31:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c10d100 con 0x7f725c107470 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.515+0000 7f7262000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c10d400 con 0x7f725c102210 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c104b40 0x7f725c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c104b40 0x7f725c19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53428/0 (socket says 172.21.15.3:53428) 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 -- 172.21.15.3:0/103268649 learned_addr learned my addr 172.21.15.3:0/103268649 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725be00640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f725c107470 0x7f725c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725b400640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f725c102210 0x7f725c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 -- 172.21.15.3:0/103268649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f725c102210 msgr2=0x7f725c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f725c102210 0x7f725c199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 -- 172.21.15.3:0/103268649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f725c107470 msgr2=0x7f725c1a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f725c107470 0x7f725c1a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.516+0000 7f725aa00640 1 -- 172.21.15.3:0/103268649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c1a1790 con 0x7f725c104b40 2024-03-21T08:31:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.517+0000 7f725aa00640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c104b40 0x7f725c19a0b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f724800e9f0 tx=0x7f724800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.517+0000 7f7253e00640 1 -- 172.21.15.3:0/103268649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800ccc0 con 0x7f725c104b40 2024-03-21T08:31:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.517+0000 7f7253e00640 1 -- 172.21.15.3:0/103268649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f72480054d0 con 0x7f725c104b40 2024-03-21T08:31:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.517+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c1a1a80 con 0x7f725c104b40 2024-03-21T08:31:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.517+0000 7f7253e00640 1 -- 172.21.15.3:0/103268649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248010430 con 0x7f725c104b40 2024-03-21T08:31:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.517+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c1a1f80 con 0x7f725c104b40 2024-03-21T08:31:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.518+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c108f60 con 0x7f725c104b40 2024-03-21T08:31:41.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.521+0000 7f7253e00640 1 -- 172.21.15.3:0/103268649 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) v1 ==== 137870+0+0 (secure 0 0 0) 0x7f7248005020 con 0x7f725c104b40 2024-03-21T08:31:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.523+0000 7f7253e00640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 0x7f72400a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.523+0000 7f725b400640 1 -- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 msgr2=0x7f72400a5420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.523+0000 7f7253e00640 1 -- 172.21.15.3:0/103268649 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f72480cfc60 con 0x7f725c104b40 2024-03-21T08:31:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.523+0000 7f725b400640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 0x7f72400a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:31:41.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.527+0000 7f7253e00640 1 -- 172.21.15.3:0/103268649 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f724809c5f0 con 0x7f725c104b40 2024-03-21T08:31:41.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/535747257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-mon[35477]: scrub queued for path: / 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/366986128' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: ignoring --setuser ceph since I am not root 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: ignoring --setgroup ceph since I am not root 2024-03-21T08:31:41.627 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:41 smithi084 ceph-mgr[36851]: -- 172.21.15.84:0/2728417576 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f54be2a5a0 con 0x55f54bdff000 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/535747257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-mon[35388]: scrub queued for path: / 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/366986128' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: ignoring --setuser ceph since I am not root 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: ignoring --setgroup ceph since I am not root 2024-03-21T08:31:41.634 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:41 smithi171 ceph-mgr[37238]: -- 172.21.15.171:0/3397214854 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d4aa86c5a0 con 0x55d4aa840c00 2024-03-21T08:31:41.676 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: ignoring --setuser ceph since I am not root 2024-03-21T08:31:41.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:41 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: ignoring --setgroup ceph since I am not root 2024-03-21T08:31:41.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.724+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f725c02d070 con 0x7f725c104b40 2024-03-21T08:31:41.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.724+0000 7f725b400640 1 -- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 msgr2=0x7f72400a5420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:41.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.724+0000 7f725b400640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 0x7f72400a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:31:41.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.730+0000 7f7253e00640 1 -- 172.21.15.3:0/103268649 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f724809bef0 con 0x7f725c104b40 2024-03-21T08:31:41.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 msgr2=0x7f72400a5420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:31:41.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 0x7f72400a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c104b40 msgr2=0x7f725c19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c104b40 0x7f725c19a0b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f724800e9f0 tx=0x7f724800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 shutdown_connections 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f725c107470 0x7f725c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c104b40 0x7f725c19a0b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f72400a2f60 0x7f72400a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 --2- 172.21.15.3:0/103268649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f725c102210 0x7f725c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.732+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 >> 172.21.15.3:0/103268649 conn(0x7f725c0fbb40 msgr2=0x7f725c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.733+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 shutdown_connections 2024-03-21T08:31:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.733+0000 7f7262000640 1 -- 172.21.15.3:0/103268649 wait complete. 2024-03-21T08:31:41.751 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-21T08:31:41.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 -- 172.21.15.3:0/1937271921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34104af0 msgr2=0x7f8b34104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1937271921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34104af0 0x7f8b34104f70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009900 tx=0x7f8b2002bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:41.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 -- 172.21.15.3:0/1937271921 shutdown_connections 2024-03-21T08:31:41.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1937271921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b341054b0 0x7f8b3410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1937271921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b34104af0 0x7f8b34104f70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1937271921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b34103900 0x7f8b34103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 -- 172.21.15.3:0/1937271921 >> 172.21.15.3:0/1937271921 conn(0x7f8b340fee90 msgr2=0x7f8b341012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:41.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 -- 172.21.15.3:0/1937271921 shutdown_connections 2024-03-21T08:31:41.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.945+0000 7f8b39e00640 1 -- 172.21.15.3:0/1937271921 wait complete. 2024-03-21T08:31:41.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 Processor -- start 2024-03-21T08:31:41.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 -- start start 2024-03-21T08:31:41.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b34103900 0x7f8b34199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b34104af0 0x7f8b34199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b341054b0 0x7f8b341a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3410f410 con 0x7f8b341054b0 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3410f290 con 0x7f8b34104af0 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.946+0000 7f8b39e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3410f590 con 0x7f8b34103900 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b341054b0 0x7f8b341a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b341054b0 0x7f8b341a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53454/0 (socket says 172.21.15.3:53454) 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 -- 172.21.15.3:0/1149731212 learned_addr learned my addr 172.21.15.3:0/1149731212 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b32a00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b34103900 0x7f8b34199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b32000640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b34104af0 0x7f8b34199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 -- 172.21.15.3:0/1149731212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b34103900 msgr2=0x7f8b34199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b34103900 0x7f8b34199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 -- 172.21.15.3:0/1149731212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b34104af0 msgr2=0x7f8b34199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b34104af0 0x7f8b34199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b33400640 1 -- 172.21.15.3:0/1149731212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b341a16b0 con 0x7f8b341054b0 2024-03-21T08:31:41.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.947+0000 7f8b32a00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b34103900 0x7f8b34199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:31:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.948+0000 7f8b33400640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b341054b0 0x7f8b341a1040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00cce0 tx=0x7f8b2c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.948+0000 7f8b38a00640 1 -- 172.21.15.3:0/1149731212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00f450 con 0x7f8b341054b0 2024-03-21T08:31:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.948+0000 7f8b38a00640 1 -- 172.21.15.3:0/1149731212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b2c00faa0 con 0x7f8b341054b0 2024-03-21T08:31:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.948+0000 7f8b38a00640 1 -- 172.21.15.3:0/1149731212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c018b70 con 0x7f8b341054b0 2024-03-21T08:31:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.948+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b341a19a0 con 0x7f8b341054b0 2024-03-21T08:31:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.948+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b341a1e70 con 0x7f8b341054b0 2024-03-21T08:31:41.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.949+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3410ac20 con 0x7f8b341054b0 2024-03-21T08:31:41.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.953+0000 7f8b38a00640 1 -- 172.21.15.3:0/1149731212 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) v1 ==== 137870+0+0 (secure 0 0 0) 0x7f8b2c018d10 con 0x7f8b341054b0 2024-03-21T08:31:41.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.954+0000 7f8b38a00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 0x7f8b180a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:41.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.954+0000 7f8b32a00640 1 -- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 msgr2=0x7f8b180a54a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:41.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.954+0000 7f8b38a00640 1 -- 172.21.15.3:0/1149731212 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8b2c0cfc80 con 0x7f8b341054b0 2024-03-21T08:31:41.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.954+0000 7f8b32a00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 0x7f8b180a54a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:31:41.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:41.958+0000 7f8b38a00640 1 -- 172.21.15.3:0/1149731212 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b2c014030 con 0x7f8b341054b0 2024-03-21T08:31:42.153 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:42 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:42.150+0000 7f718dedc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:31:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.155+0000 7f8b32a00640 1 -- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 msgr2=0x7f8b180a54a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.155+0000 7f8b32a00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 0x7f8b180a54a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:31:42.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.156+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f8b3402d2e0 con 0x7f8b341054b0 2024-03-21T08:31:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.163+0000 7f8b38a00640 1 -- 172.21.15.3:0/1149731212 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8b2c09c670 con 0x7f8b341054b0 2024-03-21T08:31:42.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 msgr2=0x7f8b180a54a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:31:42.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 0x7f8b180a54a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b341054b0 msgr2=0x7f8b341a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:42.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b341054b0 0x7f8b341a1040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00cce0 tx=0x7f8b2c00a620 comp rx=0 tx=0).stop 2024-03-21T08:31:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 shutdown_connections 2024-03-21T08:31:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b341054b0 0x7f8b341a1040 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b34104af0 0x7f8b34199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f8b180a2fe0 0x7f8b180a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 --2- 172.21.15.3:0/1149731212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b34103900 0x7f8b34199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.165+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 >> 172.21.15.3:0/1149731212 conn(0x7f8b340fee90 msgr2=0x7f8b34107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.166+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 shutdown_connections 2024-03-21T08:31:42.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.166+0000 7f8b39e00640 1 -- 172.21.15.3:0/1149731212 wait complete. 2024-03-21T08:31:42.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-21T08:31:42.383 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:42.113+0000 7ff2eaa1c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:31:42.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:42.235+0000 7ff2eaa1c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:31:42.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:42 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/366986128' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T08:31:42.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:42 smithi171 ceph-mon[35388]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-21T08:31:42.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:42 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/103268649' entity='client.admin' 2024-03-21T08:31:42.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:42 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1149731212' entity='client.admin' 2024-03-21T08:31:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 -- 172.21.15.3:0/4194190547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c100760 msgr2=0x7f371c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 --2- 172.21.15.3:0/4194190547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c100760 0x7f371c100bc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3704009900 tx=0x7f370402bcd0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 -- 172.21.15.3:0/4194190547 shutdown_connections 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 --2- 172.21.15.3:0/4194190547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f371c101100 0x7f371c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 --2- 172.21.15.3:0/4194190547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c100760 0x7f371c100bc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 --2- 172.21.15.3:0/4194190547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f371c0ffe00 0x7f371c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 -- 172.21.15.3:0/4194190547 >> 172.21.15.3:0/4194190547 conn(0x7f371c0fb960 msgr2=0x7f371c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 -- 172.21.15.3:0/4194190547 shutdown_connections 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.388+0000 7f371be00640 1 -- 172.21.15.3:0/4194190547 wait complete. 2024-03-21T08:31:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.389+0000 7f371be00640 1 Processor -- start 2024-03-21T08:31:42.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.389+0000 7f371be00640 1 -- start start 2024-03-21T08:31:42.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c0ffe00 0x7f371c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:42.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f371c100760 0x7f371c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:42.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f371c101100 0x7f371c1a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:42.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c10f220 con 0x7f371c0ffe00 2024-03-21T08:31:42.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c10f0a0 con 0x7f371c100760 2024-03-21T08:31:42.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c10f3a0 con 0x7f371c101100 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c0ffe00 0x7f371c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c0ffe00 0x7f371c199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53480/0 (socket says 172.21.15.3:53480) 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 -- 172.21.15.3:0/1268897333 learned_addr learned my addr 172.21.15.3:0/1268897333 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f371c100760 0x7f371c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371b400640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f371c101100 0x7f371c1a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f371c101100 msgr2=0x7f371c1a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f371c101100 0x7f371c1a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f371c100760 msgr2=0x7f371c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f371c100760 0x7f371c199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.390+0000 7f371aa00640 1 -- 172.21.15.3:0/1268897333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c1a1520 con 0x7f371c0ffe00 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.391+0000 7f371aa00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c0ffe00 0x7f371c199900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f370c00e9f0 tx=0x7f370c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.391+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c00cd80 con 0x7f371c0ffe00 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.391+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f370c0108e0 con 0x7f371c0ffe00 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.391+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c018920 con 0x7f371c0ffe00 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.391+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c1a1810 con 0x7f371c0ffe00 2024-03-21T08:31:42.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.391+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c1a1d10 con 0x7f371c0ffe00 2024-03-21T08:31:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.392+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c10ab50 con 0x7f371c0ffe00 2024-03-21T08:31:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.394+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) v1 ==== 137870+0+0 (secure 0 0 0) 0x7f370c010a90 con 0x7f371c0ffe00 2024-03-21T08:31:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.396+0000 7f3713400640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.396+0000 7f3713e00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 msgr2=0x7f37000a5410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.396+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f370c0cfd20 con 0x7f371c0ffe00 2024-03-21T08:31:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.396+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:31:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.400+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f370c09c6b0 con 0x7f371c0ffe00 2024-03-21T08:31:42.426 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:42 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:42.286+0000 7f718dedc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:31:42.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:42 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/366986128' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T08:31:42.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:42 smithi003 ceph-mon[31675]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-21T08:31:42.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:42 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/103268649' entity='client.admin' 2024-03-21T08:31:42.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:42 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1149731212' entity='client.admin' 2024-03-21T08:31:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.597+0000 7f3713e00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 msgr2=0x7f37000a5410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.597+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:31:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.619+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 --> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f371c061ae0 con 0x7f37000a2f50 2024-03-21T08:31:42.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:42 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/366986128' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T08:31:42.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:42 smithi084 ceph-mon[35477]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-21T08:31:42.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:42 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/103268649' entity='client.admin' 2024-03-21T08:31:42.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:42 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1149731212' entity='client.admin' 2024-03-21T08:31:42.626 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:42.226+0000 7fd854c7f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:31:42.626 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:42.365+0000 7fd854c7f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:31:43.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.998+0000 7f3713e00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 msgr2=0x7f37000a5410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:43.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:42.998+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T08:31:43.314 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:43.015+0000 7ff2eaa1c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:31:43.314 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:43.135+0000 7ff2eaa1c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:31:43.426 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:43.114+0000 7f718dedc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:31:43.426 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:43.240+0000 7f718dedc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:31:43.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:43.423+0000 7f718dedc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:31:43.625 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:43.345+0000 7fd854c7f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:31:43.625 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:43.543+0000 7fd854c7f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:31:43.633 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:43.312+0000 7ff2eaa1c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:31:43.633 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:43.435+0000 7ff2eaa1c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:31:43.634 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:43.544+0000 7ff2eaa1c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:31:43.676 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:43.552+0000 7f718dedc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:31:43.676 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:43.666+0000 7f718dedc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:31:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:43.799+0000 7f3713e00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 msgr2=0x7f37000a5410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:43.799+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T08:31:44.005 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:43.762+0000 7fd854c7f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:31:44.005 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:43.888+0000 7fd854c7f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:31:44.005 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:44.003+0000 7fd854c7f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:31:44.133 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:43.879+0000 7ff2eaa1c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:31:44.133 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:44.018+0000 7ff2eaa1c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:31:44.133 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:44.130+0000 7ff2eaa1c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:31:44.280 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:44.014+0000 7f718dedc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:31:44.280 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:44.160+0000 7f718dedc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:31:44.280 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:44.277+0000 7f718dedc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:31:44.383 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:44.253+0000 7ff2eaa1c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:31:44.636 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:44.406+0000 7f718dedc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:31:44.636 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:44.634+0000 7f718dedc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:31:44.815 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:44.445+0000 7fd854c7f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:31:44.815 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:44.666+0000 7fd854c7f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:31:44.815 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:44.813+0000 7fd854c7f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:31:44.883 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:44.473+0000 7ff2eaa1c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:31:45.125 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:45.004+0000 7fd854c7f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:31:45.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:45.401+0000 7f3713e00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 msgr2=0x7f37000a5410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:45.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:45.401+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T08:31:45.625 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:45.319+0000 7fd854c7f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:31:46.134 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:45.744+0000 7ff2eaa1c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:31:46.134 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:45.916+0000 7ff2eaa1c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:31:46.426 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:45 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:45.967+0000 7f718dedc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:31:46.426 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:46 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:46.148+0000 7f718dedc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:31:46.633 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:46.334+0000 7ff2eaa1c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:31:46.634 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:46.453+0000 7ff2eaa1c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:31:46.926 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:46 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:46.583+0000 7f718dedc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:31:46.927 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:46 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:46.707+0000 7f718dedc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:31:47.134 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:46.710+0000 7ff2eaa1c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:31:47.134 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:46.955+0000 7ff2eaa1c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:31:47.236 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:46 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:46.977+0000 7f718dedc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:31:47.236 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:47 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:47.233+0000 7f718dedc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:31:47.375 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:46.888+0000 7fd854c7f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:31:47.375 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:47.132+0000 7fd854c7f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:31:47.631 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:47 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:47.628+0000 7f718dedc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:31:47.633 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:47.328+0000 7ff2eaa1c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:31:47.956 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:47.649+0000 7ff2eaa1c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:31:48.099 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:47.695+0000 7fd854c7f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:31:48.099 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:47.818+0000 7fd854c7f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:31:48.279 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:47 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:47.957+0000 7f718dedc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:31:48.375 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:48.097+0000 7fd854c7f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:31:48.384 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:47.953+0000 7ff2eaa1c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:31:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:48.605+0000 7f3713e00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 msgr2=0x7f37000a5410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1520535032 2024-03-21T08:31:48.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:48.605+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-21T08:31:48.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:48 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:48.276+0000 7f718dedc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:31:48.876 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:48.390+0000 7fd854c7f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:31:49.291 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:48.895+0000 7fd854c7f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:31:49.383 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:48.984+0000 7ff2eaa1c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:31:49.625 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:49.289+0000 7fd854c7f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:31:49.676 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:49 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:49.367+0000 7f718dedc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:31:50.095 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:49.683+0000 7ff2eaa1c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:31:50.096 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:49.813+0000 7ff2eaa1c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:31:50.125 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:49.634+0000 7fd854c7f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:31:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:50.178+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 20) v1 ==== 137870+0+0 (secure 0 0 0) 0x7f370c098620 con 0x7f371c0ffe00 2024-03-21T08:31:50.383 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:31:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z[37234]: 2024-03-21T08:31:50.093+0000 7ff2eaa1c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:31:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:50 smithi171 ceph-mon[35388]: Standby manager daemon z restarted 2024-03-21T08:31:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:50 smithi171 ceph-mon[35388]: Standby manager daemon z started 2024-03-21T08:31:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:50 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:31:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:50 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:31:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:50 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:31:50.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:50 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:31:50.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-mon[31675]: Standby manager daemon z restarted 2024-03-21T08:31:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-mon[31675]: Standby manager daemon z started 2024-03-21T08:31:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:31:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:31:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:31:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:31:50.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:50.095+0000 7f718dedc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:31:50.427 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:50.242+0000 7f718dedc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:31:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:50.574+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 21) v1 ==== 137539+0+0 (secure 0 0 0) 0x7f370c0cd400 con 0x7f371c0ffe00 2024-03-21T08:31:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:50.575+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 msgr2=0x7f37000a5410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:31:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:50.575+0000 7f3713400640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1520535032,v1:172.21.15.3:6801/1520535032] conn(0x7f37000a2f50 0x7f37000a5410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:50.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:50 smithi084 ceph-mon[35477]: Standby manager daemon z restarted 2024-03-21T08:31:50.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:50 smithi084 ceph-mon[35477]: Standby manager daemon z started 2024-03-21T08:31:50.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:50 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:31:50.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:50 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:31:50.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:50 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:31:50.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:50 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.171:0/2743582566' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:31:50.784 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:50 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:50.553+0000 7f718dedc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:31:51.122 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.117+0000 7f70e2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.117+0000 7f70e2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.117+0000 7f70e2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.117+0000 7f70e2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.117+0000 7f70e2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.118+0000 7f70e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.118+0000 7f70e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.118+0000 7f70e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.118+0000 7f70e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.123 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:31:51.118+0000 7f70e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:31:51.125 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:50.983+0000 7fd854c7f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: Active manager daemon x restarted 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: Activating manager daemon x 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: mgrmap e21: x(active, starting, since 0.0169517s), standbys: z, y 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:31:51.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:31:51.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:31:51.430 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: Manager daemon x is now available 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:31:51.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:51 smithi003 ceph-mon[31675]: scrub summary: idle 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: Active manager daemon x restarted 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: Activating manager daemon x 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: mgrmap e21: x(active, starting, since 0.0169517s), standbys: z, y 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:31:51.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:31:51.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:31:51.519 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: Manager daemon x is now available 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:31:51.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:51 smithi171 ceph-mon[35388]: scrub summary: idle 2024-03-21T08:31:51.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.595+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mgrmap(e 22) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f370c0d1d20 con 0x7f371c0ffe00 2024-03-21T08:31:51.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.596+0000 7f3713400640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3700035020 0x7f3700035400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:51.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.596+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 --> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f370c0bde70 con 0x7f3700035020 2024-03-21T08:31:51.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.602+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3700035020 0x7f3700035400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:51.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.613+0000 7f3713e00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3700035020 0x7f3700035400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f371c19ae10 tx=0x7f3704000bc0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: Active manager daemon x restarted 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: Activating manager daemon x 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: mgrmap e21: x(active, starting, since 0.0169517s), standbys: z, y 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:31:51.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:31:51.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:31:51.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: Manager daemon x is now available 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:31:51.630 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-mon[35477]: scrub summary: idle 2024-03-21T08:31:51.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.751+0000 7f3713400640 1 -- 172.21.15.3:0/1268897333 <== mgr.15654 v2:172.21.15.3:6800/1070134681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f370c0bde70 con 0x7f3700035020 2024-03-21T08:31:51.753 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path / 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3700035020 msgr2=0x7f3700035400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3700035020 0x7f3700035400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f371c19ae10 tx=0x7f3704000bc0 comp rx=0 tx=0).stop 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c0ffe00 msgr2=0x7f371c199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 --2- 172.21.15.3:0/1268897333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f371c0ffe00 0x7f371c199900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f370c00e9f0 tx=0x7f370c00eeb0 comp rx=0 tx=0).stop 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371b400640 1 -- 172.21.15.3:0/1268897333 reap_dead start 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 shutdown_connections 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 >> 172.21.15.3:0/1268897333 conn(0x7f371c0fb960 msgr2=0x7f371c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 shutdown_connections 2024-03-21T08:31:51.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.754+0000 7f371be00640 1 -- 172.21.15.3:0/1268897333 wait complete. 2024-03-21T08:31:51.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-21T08:31:51.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 -- 172.21.15.3:0/1238147873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301016f0 msgr2=0x7f4d30101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:51.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 --2- 172.21.15.3:0/1238147873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301016f0 0x7f4d30101b10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009900 tx=0x7f4d2402bc30 comp rx=0 tx=0).stop 2024-03-21T08:31:51.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 -- 172.21.15.3:0/1238147873 shutdown_connections 2024-03-21T08:31:51.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 --2- 172.21.15.3:0/1238147873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d301032a0 0x7f4d30109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:51.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 --2- 172.21.15.3:0/1238147873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d301028e0 0x7f4d30102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:51.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 --2- 172.21.15.3:0/1238147873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301016f0 0x7f4d30101b10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 -- 172.21.15.3:0/1238147873 >> 172.21.15.3:0/1238147873 conn(0x7f4d300fcc60 msgr2=0x7f4d300ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 -- 172.21.15.3:0/1238147873 shutdown_connections 2024-03-21T08:31:51.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.970+0000 7f4d37a00640 1 -- 172.21.15.3:0/1238147873 wait complete. 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 Processor -- start 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 -- start start 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d301016f0 0x7f4d30197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301028e0 0x7f4d30197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d301032a0 0x7f4d3019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3010d240 con 0x7f4d301028e0 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3010d0c0 con 0x7f4d301032a0 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.971+0000 7f4d37a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3010d3c0 con 0x7f4d301016f0 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301028e0 0x7f4d30197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301028e0 0x7f4d30197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46456/0 (socket says 172.21.15.3:46456) 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 -- 172.21.15.3:0/2034111256 learned_addr learned my addr 172.21.15.3:0/2034111256 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:51.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d35000640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d301016f0 0x7f4d30197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d35a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d301032a0 0x7f4d3019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 -- 172.21.15.3:0/2034111256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d301016f0 msgr2=0x7f4d30197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d301016f0 0x7f4d30197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 -- 172.21.15.3:0/2034111256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d301032a0 msgr2=0x7f4d3019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d301032a0 0x7f4d3019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d2fe00640 1 -- 172.21.15.3:0/2034111256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3019f580 con 0x7f4d301028e0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.972+0000 7f4d35a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d301032a0 0x7f4d3019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.973+0000 7f4d2fe00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301028e0 0x7f4d30197ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00d950 tx=0x7f4d1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.973+0000 7f4d2d600640 1 -- 172.21.15.3:0/2034111256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c017070 con 0x7f4d301028e0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.973+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3019f870 con 0x7f4d301028e0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.973+0000 7f4d2d600640 1 -- 172.21.15.3:0/2034111256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4d1c009c30 con 0x7f4d301028e0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.973+0000 7f4d2d600640 1 -- 172.21.15.3:0/2034111256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00f620 con 0x7f4d301028e0 2024-03-21T08:31:51.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.973+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3019fdc0 con 0x7f4d301028e0 2024-03-21T08:31:51.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.974+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d30108b30 con 0x7f4d301028e0 2024-03-21T08:31:51.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.979+0000 7f4d2d600640 1 -- 172.21.15.3:0/2034111256 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f4d1c0109a0 con 0x7f4d301028e0 2024-03-21T08:31:51.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.981+0000 7f4d2d600640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4d180a2d00 0x7f4d180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:51.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.982+0000 7f4d2d600640 1 -- 172.21.15.3:0/2034111256 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f4d1c0cf170 con 0x7f4d301028e0 2024-03-21T08:31:51.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.982+0000 7f4d35000640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4d180a2d00 0x7f4d180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:51.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.982+0000 7f4d35000640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4d180a2d00 0x7f4d180a51c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4d2402c240 tx=0x7f4d2402c7e0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:51.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:51.987+0000 7f4d2d600640 1 -- 172.21.15.3:0/2034111256 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d1c009de0 con 0x7f4d301028e0 2024-03-21T08:31:52.185 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:51 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:51.918+0000 7fd854c7f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:31:52.186 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:52 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:52.094+0000 7fd854c7f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:31:52.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.213+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 --> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d30061c40 con 0x7f4d180a2d00 2024-03-21T08:31:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.366+0000 7f4d2d600640 1 -- 172.21.15.3:0/2034111256 <== mgr.15654 v2:172.21.15.3:6800/1070134681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f4d30061c40 con 0x7f4d180a2d00 2024-03-21T08:31:52.369 INFO:teuthology.orchestra.run.smithi003.stdout:Retention added to path / 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4d180a2d00 msgr2=0x7f4d180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4d180a2d00 0x7f4d180a51c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4d2402c240 tx=0x7f4d2402c7e0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301028e0 msgr2=0x7f4d30197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301028e0 0x7f4d30197ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00d950 tx=0x7f4d1c00de10 comp rx=0 tx=0).stop 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 shutdown_connections 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d301032a0 0x7f4d3019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d301028e0 0x7f4d30197ea0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4d180a2d00 0x7f4d180a51c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 --2- 172.21.15.3:0/2034111256 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d301016f0 0x7f4d30197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 >> 172.21.15.3:0/2034111256 conn(0x7f4d300fcc60 msgr2=0x7f4d30105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:52.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 shutdown_connections 2024-03-21T08:31:52.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.370+0000 7f4d37a00640 1 -- 172.21.15.3:0/2034111256 wait complete. 2024-03-21T08:31:52.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-21T08:31:52.459 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:52 smithi003 ceph-mon[31675]: [21/Mar/2024:08:31:51] ENGINE Bus STARTING 2024-03-21T08:31:52.459 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:52 smithi003 ceph-mon[31675]: mgrmap e22: x(active, since 1.04178s), standbys: z, y 2024-03-21T08:31:52.478 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:52 smithi084 ceph-mon[35477]: [21/Mar/2024:08:31:51] ENGINE Bus STARTING 2024-03-21T08:31:52.479 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:52 smithi084 ceph-mon[35477]: mgrmap e22: x(active, since 1.04178s), standbys: z, y 2024-03-21T08:31:52.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:52 smithi171 ceph-mon[35388]: [21/Mar/2024:08:31:51] ENGINE Bus STARTING 2024-03-21T08:31:52.518 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:52 smithi171 ceph-mon[35388]: mgrmap e22: x(active, since 1.04178s), standbys: z, y 2024-03-21T08:31:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.583+0000 7f207f000640 1 -- 172.21.15.3:0/3359403903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781016f0 msgr2=0x7f2078101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.583+0000 7f207f000640 1 --2- 172.21.15.3:0/3359403903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781016f0 0x7f2078101b10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2068009900 tx=0x7f206802bc40 comp rx=0 tx=0).stop 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.584+0000 7f207f000640 1 -- 172.21.15.3:0/3359403903 shutdown_connections 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.584+0000 7f207f000640 1 --2- 172.21.15.3:0/3359403903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20781032a0 0x7f2078109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.584+0000 7f207f000640 1 --2- 172.21.15.3:0/3359403903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20781028e0 0x7f2078102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.584+0000 7f207f000640 1 --2- 172.21.15.3:0/3359403903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781016f0 0x7f2078101b10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.584+0000 7f207f000640 1 -- 172.21.15.3:0/3359403903 >> 172.21.15.3:0/3359403903 conn(0x7f20780fcc60 msgr2=0x7f20780ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.584+0000 7f207f000640 1 -- 172.21.15.3:0/3359403903 shutdown_connections 2024-03-21T08:31:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.584+0000 7f207f000640 1 -- 172.21.15.3:0/3359403903 wait complete. 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 Processor -- start 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 -- start start 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20781016f0 0x7f2078199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20781028e0 0x7f207819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781032a0 0x7f20781a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207810d240 con 0x7f20781032a0 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207810d0c0 con 0x7f20781016f0 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207f000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207810d3c0 con 0x7f20781028e0 2024-03-21T08:31:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781032a0 0x7f20781a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f2077e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20781016f0 0x7f2078199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781032a0 0x7f20781a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46476/0 (socket says 172.21.15.3:46476) 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.585+0000 7f207d000640 1 -- 172.21.15.3:0/1046137779 learned_addr learned my addr 172.21.15.3:0/1046137779 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.586+0000 7f2077400640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20781028e0 0x7f207819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.586+0000 7f2077e00640 1 -- 172.21.15.3:0/1046137779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20781028e0 msgr2=0x7f207819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.586+0000 7f2077e00640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20781028e0 0x7f207819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.586+0000 7f2077e00640 1 -- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781032a0 msgr2=0x7f20781a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.586+0000 7f2077e00640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781032a0 0x7f20781a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.586+0000 7f2077e00640 1 -- 172.21.15.3:0/1046137779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20781a16e0 con 0x7f20781016f0 2024-03-21T08:31:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.586+0000 7f2077e00640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20781016f0 0x7f2078199ac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2068036040 tx=0x7f20680040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.587+0000 7f2074c00640 1 -- 172.21.15.3:0/1046137779 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068043020 con 0x7f20781016f0 2024-03-21T08:31:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.587+0000 7f2074c00640 1 -- 172.21.15.3:0/1046137779 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20680044c0 con 0x7f20781016f0 2024-03-21T08:31:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.587+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20781a1970 con 0x7f20781016f0 2024-03-21T08:31:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.587+0000 7f2074c00640 1 -- 172.21.15.3:0/1046137779 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068003bf0 con 0x7f20781016f0 2024-03-21T08:31:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.587+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20781a1e90 con 0x7f20781016f0 2024-03-21T08:31:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.588+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207804fc50 con 0x7f20781016f0 2024-03-21T08:31:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.590+0000 7f2074c00640 1 -- 172.21.15.3:0/1046137779 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f2068046020 con 0x7f20781016f0 2024-03-21T08:31:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.591+0000 7f2074c00640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f205c0a2d00 0x7f205c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.591+0000 7f2074c00640 1 -- 172.21.15.3:0/1046137779 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f20680f1790 con 0x7f20781016f0 2024-03-21T08:31:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.592+0000 7f2077400640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f205c0a2d00 0x7f205c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.592+0000 7f2077400640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f205c0a2d00 0x7f205c0a51c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2064005e10 tx=0x7f2064005da0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:52.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.601+0000 7f2074c00640 1 -- 172.21.15.3:0/1046137779 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f206804d020 con 0x7f20781016f0 2024-03-21T08:31:52.748 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:31:52 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y[36847]: 2024-03-21T08:31:52.477+0000 7fd854c7f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:31:52.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.840+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 --> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2078074ff0 con 0x7f205c0a2d00 2024-03-21T08:31:52.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.856+0000 7f2074c00640 1 -- 172.21.15.3:0/1046137779 <== mgr.15654 v2:172.21.15.3:6800/1070134681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f2078074ff0 con 0x7f205c0a2d00 2024-03-21T08:31:52.858 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-21T00:00:00", "created": "2024-03-21T08:31:51", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-21T08:31:52.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.858+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f205c0a2d00 msgr2=0x7f205c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:52.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.858+0000 7f207f000640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f205c0a2d00 0x7f205c0a51c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2064005e10 tx=0x7f2064005da0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.858+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20781016f0 msgr2=0x7f2078199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:52.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.858+0000 7f207f000640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20781016f0 0x7f2078199ac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2068036040 tx=0x7f20680040c0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 shutdown_connections 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20781032a0 0x7f20781a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f205c0a2d00 0x7f205c0a51c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20781028e0 0x7f207819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 --2- 172.21.15.3:0/1046137779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20781016f0 0x7f2078199ac0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 >> 172.21.15.3:0/1046137779 conn(0x7f20780fcc60 msgr2=0x7f20780738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 shutdown_connections 2024-03-21T08:31:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:52.859+0000 7f207f000640 1 -- 172.21.15.3:0/1046137779 wait complete. 2024-03-21T08:31:52.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-21T08:31:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 -- 172.21.15.3:0/2076101939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701032a0 msgr2=0x7f6570109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 --2- 172.21.15.3:0/2076101939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701032a0 0x7f6570109b30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6568009ca0 tx=0x7f656802c0f0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 -- 172.21.15.3:0/2076101939 shutdown_connections 2024-03-21T08:31:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 --2- 172.21.15.3:0/2076101939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701032a0 0x7f6570109b30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 --2- 172.21.15.3:0/2076101939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65701028e0 0x7f6570102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 --2- 172.21.15.3:0/2076101939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65701016f0 0x7f6570101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 -- 172.21.15.3:0/2076101939 >> 172.21.15.3:0/2076101939 conn(0x7f65700fcc60 msgr2=0x7f65700ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:53.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 -- 172.21.15.3:0/2076101939 shutdown_connections 2024-03-21T08:31:53.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.087+0000 7f6575e00640 1 -- 172.21.15.3:0/2076101939 wait complete. 2024-03-21T08:31:53.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.088+0000 7f6575e00640 1 Processor -- start 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f6575e00640 1 -- start start 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f6575e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701016f0 0x7f6570075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f6575e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65701028e0 0x7f6570073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f6575e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65701032a0 0x7f6570073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f6575e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657010d200 con 0x7f65701016f0 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f6575e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657010d080 con 0x7f65701032a0 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f6575e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657010d380 con 0x7f65701028e0 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f656ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701016f0 0x7f6570075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f656ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701016f0 0x7f6570075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46514/0 (socket says 172.21.15.3:46514) 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f656ea00640 1 -- 172.21.15.3:0/560519546 learned_addr learned my addr 172.21.15.3:0/560519546 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:31:53.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.089+0000 7f656f400640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65701032a0 0x7f6570073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f656e000640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65701028e0 0x7f6570073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f656ea00640 1 -- 172.21.15.3:0/560519546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65701028e0 msgr2=0x7f6570073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f656ea00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65701028e0 0x7f6570073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f656ea00640 1 -- 172.21.15.3:0/560519546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65701032a0 msgr2=0x7f6570073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f656ea00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65701032a0 0x7f6570073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f656ea00640 1 -- 172.21.15.3:0/560519546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570074860 con 0x7f65701016f0 2024-03-21T08:31:53.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f656ea00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701016f0 0x7f6570075450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f656000e9c0 tx=0x7f656000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:53.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.090+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000cd80 con 0x7f65701016f0 2024-03-21T08:31:53.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.091+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f65600035f0 con 0x7f65701016f0 2024-03-21T08:31:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.091+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65701a19f0 con 0x7f65701016f0 2024-03-21T08:31:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.091+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560010690 con 0x7f65701016f0 2024-03-21T08:31:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.091+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65701a1f90 con 0x7f65701016f0 2024-03-21T08:31:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.092+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6570108ab0 con 0x7f65701016f0 2024-03-21T08:31:53.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.094+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f65600037a0 con 0x7f65701016f0 2024-03-21T08:31:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.095+0000 7f6574a00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f65540a2d80 0x7f65540a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:31:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.096+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f65600cf990 con 0x7f65701016f0 2024-03-21T08:31:53.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.096+0000 7f656e000640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f65540a2d80 0x7f65540a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:31:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.097+0000 7f656e000640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f65540a2d80 0x7f65540a5240 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65700751c0 tx=0x7f655c00b390 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:31:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.100+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f656009c530 con 0x7f65701016f0 2024-03-21T08:31:53.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.215+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 23) v1 ==== 137973+0+0 (secure 0 0 0) 0x7f65600984a0 con 0x7f65701016f0 2024-03-21T08:31:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.385+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 --> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6570061ae0 con 0x7f65540a2d80 2024-03-21T08:31:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.397+0000 7f6574a00640 1 -- 172.21.15.3:0/560519546 <== mgr.15654 v2:172.21.15.3:6800/1070134681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f6570061ae0 con 0x7f65540a2d80 2024-03-21T08:31:53.399 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-03-21T08:31:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f65540a2d80 msgr2=0x7f65540a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f65540a2d80 0x7f65540a5240 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65700751c0 tx=0x7f655c00b390 comp rx=0 tx=0).stop 2024-03-21T08:31:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701016f0 msgr2=0x7f6570075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:31:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701016f0 0x7f6570075450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f656000e9c0 tx=0x7f656000ee80 comp rx=0 tx=0).stop 2024-03-21T08:31:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 shutdown_connections 2024-03-21T08:31:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65701032a0 0x7f6570073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f65540a2d80 0x7f65540a5240 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65701028e0 0x7f6570073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 --2- 172.21.15.3:0/560519546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65701016f0 0x7f6570075450 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:31:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 >> 172.21.15.3:0/560519546 conn(0x7f65700fcc60 msgr2=0x7f6570105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:31:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 shutdown_connections 2024-03-21T08:31:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:31:53.400+0000 7f6575e00640 1 -- 172.21.15.3:0/560519546 wait complete. 2024-03-21T08:31:53.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-21T08:31:53.488 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: [21/Mar/2024:08:31:51] ENGINE Serving on https://172.21.15.3:7150 2024-03-21T08:31:53.488 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: [21/Mar/2024:08:31:51] ENGINE Serving on http://172.21.15.3:8765 2024-03-21T08:31:53.488 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: [21/Mar/2024:08:31:51] ENGINE Bus STARTED 2024-03-21T08:31:53.488 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: Standby manager daemon y restarted 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: Standby manager daemon y started 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:31:53.489 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:53 smithi003 ceph-mon[31675]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:31:53.506 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-21T08:31:53.519 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-21T08:31:53.529 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:31:53.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-21T08:31:53.534 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T08:31:53.548 INFO:tasks.workunit:Pulling workunits from ref 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:31:53.548 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T08:31:53.548 INFO:tasks.workunit:timeout=3h 2024-03-21T08:31:53.548 INFO:tasks.workunit:cleanup=True 2024-03-21T08:31:53.549 DEBUG:teuthology.orchestra.run.smithi003:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 76h/118d Inode: 1 Links: 2 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-03-21 08:29:10.346253894 +0000 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-03-21 08:29:10.346253894 +0000 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-03-21 08:31:28.855740620 +0000 2024-03-21T08:31:53.591 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-03-21T08:31:53.591 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-21T08:31:53.592 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: [21/Mar/2024:08:31:51] ENGINE Serving on https://172.21.15.3:7150 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: [21/Mar/2024:08:31:51] ENGINE Serving on http://172.21.15.3:8765 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: [21/Mar/2024:08:31:51] ENGINE Bus STARTED 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: Standby manager daemon y restarted 2024-03-21T08:31:53.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: Standby manager daemon y started 2024-03-21T08:31:53.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:31:53.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:31:53.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:31:53.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:53 smithi084 ceph-mon[35477]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: [21/Mar/2024:08:31:51] ENGINE Serving on https://172.21.15.3:7150 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: [21/Mar/2024:08:31:51] ENGINE Serving on http://172.21.15.3:8765 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: [21/Mar/2024:08:31:51] ENGINE Bus STARTED 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: Standby manager daemon y restarted 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: Standby manager daemon y started 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:31:53.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:53 smithi171 ceph-mon[35388]: from='mgr.? 172.21.15.84:0/2233422369' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:31:53.719 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:31:53.765 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T08:31:54.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:54 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:54 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:54 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:56.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:56 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:56 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:56.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:56.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:56 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi171 to 176.8M 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Unable to set osd_memory_target on smithi171 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi003 to 177.8M 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Unable to set osd_memory_target on smithi003 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi084 to 176.7M 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Unable to set osd_memory_target on smithi084 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Updating smithi003:/etc/ceph/ceph.conf 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:31:57.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:57 smithi084 ceph-mon[35477]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:31:57.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi171 to 176.8M 2024-03-21T08:31:57.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Unable to set osd_memory_target on smithi171 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T08:31:57.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi003 to 177.8M 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Unable to set osd_memory_target on smithi003 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi084 to 176.7M 2024-03-21T08:31:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Unable to set osd_memory_target on smithi084 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-21T08:31:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:31:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:31:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Updating smithi003:/etc/ceph/ceph.conf 2024-03-21T08:31:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:31:57.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:57 smithi171 ceph-mon[35388]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:31:57.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi171 to 176.8M 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Unable to set osd_memory_target on smithi171 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi003 to 177.8M 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Unable to set osd_memory_target on smithi003 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:31:57.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi084 to 176.7M 2024-03-21T08:31:57.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Unable to set osd_memory_target on smithi084 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-21T08:31:57.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:31:57.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:31:57.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Updating smithi003:/etc/ceph/ceph.conf 2024-03-21T08:31:57.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Updating smithi084:/etc/ceph/ceph.conf 2024-03-21T08:31:57.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:57 smithi003 ceph-mon[31675]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-21T08:31:58.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:58 smithi084 ceph-mon[35477]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:58.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:58 smithi084 ceph-mon[35477]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:58 smithi084 ceph-mon[35477]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:58 smithi084 ceph-mon[35477]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:58 smithi171 ceph-mon[35388]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:58 smithi171 ceph-mon[35388]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:58 smithi171 ceph-mon[35388]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:58 smithi171 ceph-mon[35388]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:58 smithi003 ceph-mon[31675]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:31:58.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:58 smithi003 ceph-mon[31675]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:58 smithi003 ceph-mon[31675]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:58.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:58 smithi003 ceph-mon[31675]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:31:59.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:59 smithi084 ceph-mon[35477]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:59 smithi084 ceph-mon[35477]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:59 smithi084 ceph-mon[35477]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:59 smithi084 ceph-mon[35477]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:31:59.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:59 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:59.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:31:59 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:59.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:59 smithi171 ceph-mon[35388]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:59 smithi171 ceph-mon[35388]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:59 smithi171 ceph-mon[35388]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:59 smithi171 ceph-mon[35388]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:31:59.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:59 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:59.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:31:59 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:59.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:59 smithi003 ceph-mon[31675]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:59 smithi003 ceph-mon[31675]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:59 smithi003 ceph-mon[31675]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:31:59.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:59 smithi003 ceph-mon[31675]: Updating smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:31:59.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:59 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:31:59.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:31:59 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:32:00.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:32:00.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:32:00.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:00 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:32:00.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:00 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:32:00.395+0000 7f70b9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:32:00.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:32:00.395+0000 7f70b9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:32:00.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:32:00.395+0000 7f70b9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:32:00.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:32:00.395+0000 7f70b9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:32:00.677 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:32:00.395+0000 7f70b9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: Updating smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: Updating smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:32:00.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:00 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:32:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:02 smithi084 ceph-mon[35477]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-21T08:32:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:02 smithi084 ceph-mon[35477]: mgrmap e25: x(active, since 10s), standbys: y, z 2024-03-21T08:32:02.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:02 smithi171 ceph-mon[35388]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-21T08:32:02.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:02 smithi171 ceph-mon[35388]: mgrmap e25: x(active, since 10s), standbys: y, z 2024-03-21T08:32:02.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:02 smithi003 ceph-mon[31675]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-21T08:32:02.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:02 smithi003 ceph-mon[31675]: mgrmap e25: x(active, since 10s), standbys: y, z 2024-03-21T08:32:04.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:04 smithi084 ceph-mon[35477]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-03-21T08:32:04.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:04 smithi171 ceph-mon[35388]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-03-21T08:32:04.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:04 smithi003 ceph-mon[31675]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-03-21T08:32:06.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:06 smithi084 ceph-mon[35477]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-03-21T08:32:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:06 smithi171 ceph-mon[35388]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-03-21T08:32:06.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:06 smithi003 ceph-mon[31675]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 -- 172.21.15.3:0/2887814763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941088f0 msgr2=0x7f019410c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 --2- 172.21.15.3:0/2887814763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941088f0 0x7f019410c8e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f017c009900 tx=0x7f017c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 -- 172.21.15.3:0/2887814763 shutdown_connections 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 --2- 172.21.15.3:0/2887814763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f019410cfc0 0x7f0194114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 --2- 172.21.15.3:0/2887814763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941088f0 0x7f019410c8e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 --2- 172.21.15.3:0/2887814763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0194107f40 0x7f0194108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 -- 172.21.15.3:0/2887814763 >> 172.21.15.3:0/2887814763 conn(0x7f01940774e0 msgr2=0x7f01941064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 -- 172.21.15.3:0/2887814763 shutdown_connections 2024-03-21T08:32:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.776+0000 7f0199800640 1 -- 172.21.15.3:0/2887814763 wait complete. 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.777+0000 7f0199800640 1 Processor -- start 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.777+0000 7f0199800640 1 -- start start 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.777+0000 7f0199800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0194107f40 0x7f019419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.777+0000 7f0199800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f019410cfc0 0x7f019419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.777+0000 7f0199800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941a2d70 0x7f01941a3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f0199800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194117f30 con 0x7f01941a2d70 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f0199800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194117db0 con 0x7f0194107f40 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f0199800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941180b0 con 0x7f019410cfc0 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f0192a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0194107f40 0x7f019419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f0193400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941a2d70 0x7f01941a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f0192a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0194107f40 0x7f019419e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:43560/0 (socket says 172.21.15.3:43560) 2024-03-21T08:32:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f0192a00640 1 -- 172.21.15.3:0/1986374869 learned_addr learned my addr 172.21.15.3:0/1986374869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.778+0000 7f018be00640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f019410cfc0 0x7f019419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0193400640 1 -- 172.21.15.3:0/1986374869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f019410cfc0 msgr2=0x7f019419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0193400640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f019410cfc0 0x7f019419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0193400640 1 -- 172.21.15.3:0/1986374869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0194107f40 msgr2=0x7f019419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0193400640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0194107f40 0x7f019419e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0193400640 1 -- 172.21.15.3:0/1986374869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01941a3860 con 0x7f01941a2d70 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f018be00640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f019410cfc0 0x7f019419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0193400640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941a2d70 0x7f01941a3220 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f018c005fd0 tx=0x7f018c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f018b400640 1 -- 172.21.15.3:0/1986374869 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00cd80 con 0x7f01941a2d70 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f018b400640 1 -- 172.21.15.3:0/1986374869 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f018c005020 con 0x7f01941a2d70 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f018b400640 1 -- 172.21.15.3:0/1986374869 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c010690 con 0x7f01941a2d70 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01941a3b50 con 0x7f01941a2d70 2024-03-21T08:32:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.779+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01941ab2c0 con 0x7f01941a2d70 2024-03-21T08:32:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.783+0000 7f019c56e740 1 Processor -- start 2024-03-21T08:32:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.783+0000 7f018b400640 1 -- 172.21.15.3:0/1986374869 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f018c0051d0 con 0x7f01941a2d70 2024-03-21T08:32:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.783+0000 7f019c56e740 1 -- start start 2024-03-21T08:32:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f019c56e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f019c56e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1dac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f019c56e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f019c56e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1d653b0 con 0x55bdb1dac6c0 2024-03-21T08:32:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f019c56e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1d07830 con 0x55bdb1da2fd0 2024-03-21T08:32:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f019c56e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1cb05d0 con 0x55bdb1da38f0 2024-03-21T08:32:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f018be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f018be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52994/0 (socket says 172.21.15.3:52994) 2024-03-21T08:32:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f018be00640 1 -- 172.21.15.3:0/622100843 learned_addr learned my addr 172.21.15.3:0/622100843 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f018be00640 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.784+0000 7f0192a00640 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1dac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018be00640 1 -- 172.21.15.3:0/622100843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 msgr2=0x55bdb1dac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018be00640 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1dac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018be00640 1 -- 172.21.15.3:0/622100843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 msgr2=0x55bdb1da33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018be00640 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018be00640 1 -- 172.21.15.3:0/622100843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb1cc82c0 con 0x55bdb1dac6c0 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018b400640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f01780a31a0 0x7f01780a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018be00640 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daeab0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f019406ac90 tx=0x7f017c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7245e51f4c751711 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f018b400640 1 -- 172.21.15.3:0/1986374869 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f018c0d0580 con 0x7f01941a2d70 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f0192a00640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f01780a31a0 0x7f01780a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f0183e00640 1 -- 172.21.15.3:0/622100843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c049070 con 0x55bdb1dac6c0 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f0183e00640 1 -- 172.21.15.3:0/622100843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f017c005500 con 0x55bdb1dac6c0 2024-03-21T08:32:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.785+0000 7f0183e00640 1 -- 172.21.15.3:0/622100843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c004920 con 0x55bdb1dac6c0 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 -- 172.21.15.3:0/622100843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 msgr2=0x55bdb1daeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daeab0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f019406ac90 tx=0x7f017c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f0192a00640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f01780a31a0 0x7f01780a5660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55bdb1da2a80 tx=0x7f018400b480 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 -- 172.21.15.3:0/622100843 shutdown_connections 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daeab0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1dac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 --2- 172.21.15.3:0/622100843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 -- 172.21.15.3:0/622100843 >> 172.21.15.3:0/622100843 conn(0x55bdb1d9d8f0 msgr2=0x55bdb1d9dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 -- 172.21.15.3:0/622100843 shutdown_connections 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.786+0000 7f019c56e740 1 -- 172.21.15.3:0/622100843 wait complete. 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.787+0000 7f019c56e740 1 -- start start 2024-03-21T08:32:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.787+0000 7f019c56e740 1 Processor -- start 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f019c56e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f019c56e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da38f0 0x55bdb1da9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f019c56e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f019c56e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1cc3a20 con 0x55bdb1dac6c0 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f019c56e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1d038d0 con 0x55bdb1da38f0 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f019c56e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1daf290 con 0x55bdb1da2fd0 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f019c56e740 10 client.0 _ll_register_callbacks cb 0x55bdb1d001c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f0193400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f0193400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53006/0 (socket says 172.21.15.3:53006) 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f0193400640 1 -- 172.21.15.3:0/193160995 learned_addr learned my addr 172.21.15.3:0/193160995 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f018be00640 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da38f0 0x55bdb1da9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f0192a00640 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f0193400640 1 -- 172.21.15.3:0/193160995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 msgr2=0x55bdb1da97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.788+0000 7f0193400640 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0193400640 1 -- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da38f0 msgr2=0x55bdb1da9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0193400640 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da38f0 0x55bdb1da9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0193400640 1 -- 172.21.15.3:0/193160995 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdb1a1c210 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0193400640 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daa240 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c6a0 tx=0x7f018c09c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd521589bad9386 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0180c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0180c00640 1 -- 172.21.15.3:0/193160995 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c0b5060 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0180c00640 1 -- 172.21.15.3:0/193160995 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f018c0b5200 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f017c005500 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f0180c00640 1 -- 172.21.15.3:0/193160995 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f018c0b5400 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f019c56e740 10 client.15747 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.789+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bdb1ae8620 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f0180c00640 1 -- 172.21.15.3:0/193160995 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f017c005500 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f0180c00640 1 -- 172.21.15.3:0/193160995 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f018c0cda10 con 0x55bdb1dac6c0 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f019c56e740 10 client.15747 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f019c56e740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f019c56e740 20 client.15747 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f019c56e740 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1dbc7e0 0x55bdb1dbebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f019c56e740 4 client.15747 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.790+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdb1d14d70 con 0x55bdb1dbc7e0 2024-03-21T08:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.791+0000 7f0192a00640 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1dbc7e0 0x55bdb1dbebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.791+0000 7f0192a00640 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1dbc7e0 0x55bdb1dbebd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f565a2ebf53613 server_cookie=c389bd4c01b1f508 in_seq=0 out_seq=0 2024-03-21T08:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.791+0000 7f0180c00640 10 client.15747 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f0180c00640 1 -- 172.21.15.3:0/193160995 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bdb1d14d70 con 0x55bdb1dbc7e0 2024-03-21T08:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f0180c00640 10 client.15747 handle_command_reply: tid=0 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 client.15747 shutdown 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 msgr2=0x55bdb1daa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daa240 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c6a0 tx=0x7f018c09c300 comp rx=0 tx=0).stop 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 shutdown_connections 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1daa240 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da38f0 0x55bdb1da9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1dbc7e0 0x55bdb1dbebd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 --2- 172.21.15.3:0/193160995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1da97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 >> 172.21.15.3:0/193160995 conn(0x55bdb1d9d8f0 msgr2=0x55bdb1d9f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f0180c00640 0 client.15747 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 shutdown_connections 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 1 -- 172.21.15.3:0/193160995 wait complete. 2024-03-21T08:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.793+0000 7f019c56e740 20 client.15747 trim_cache size 0 max 16384 2024-03-21T08:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 Processor -- start 2024-03-21T08:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 -- start start 2024-03-21T08:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1dabef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1da38f0 0x55bdb1d9f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1dac6c0 0x55bdb1db79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941b76e0 con 0x55bdb1da38f0 2024-03-21T08:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941b51e0 con 0x55bdb1dac6c0 2024-03-21T08:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.809+0000 7f019c56e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941b3280 con 0x55bdb1da2fd0 2024-03-21T08:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1dac6c0 0x55bdb1db79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f018be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1da38f0 0x55bdb1d9f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1dac6c0 0x55bdb1db79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:43586/0 (socket says 172.21.15.3:43586) 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 -- 172.21.15.3:0/1764149710 learned_addr learned my addr 172.21.15.3:0/1764149710 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0192a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1dabef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 -- 172.21.15.3:0/1764149710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 msgr2=0x55bdb1dabef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1dabef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 -- 172.21.15.3:0/1764149710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1da38f0 msgr2=0x55bdb1d9f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1da38f0 0x55bdb1d9f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 -- 172.21.15.3:0/1764149710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb1da7d30 con 0x55bdb1dac6c0 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f018be00640 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1da38f0 0x55bdb1d9f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.810+0000 7f0193400640 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1dac6c0 0x55bdb1db79a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f018c009fd0 tx=0x7f018c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=261d02c3dccf4d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.811+0000 7f0183400640 1 -- 172.21.15.3:0/1764149710 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c09c0d0 con 0x55bdb1dac6c0 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.811+0000 7f0183400640 1 -- 172.21.15.3:0/1764149710 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f018c0cda10 con 0x55bdb1dac6c0 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.811+0000 7f0183400640 1 -- 172.21.15.3:0/1764149710 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c0b5060 con 0x55bdb1dac6c0 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.811+0000 7f019c56e740 1 -- 172.21.15.3:0/1764149710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1dac6c0 msgr2=0x55bdb1db79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.811+0000 7f019c56e740 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1dac6c0 0x55bdb1db79a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f018c009fd0 tx=0x7f018c00ef90 comp rx=0 tx=0).stop 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 -- 172.21.15.3:0/1764149710 shutdown_connections 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1dac6c0 0x55bdb1db79a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1da38f0 0x55bdb1d9f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 --2- 172.21.15.3:0/1764149710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da2fd0 0x55bdb1dabef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 -- 172.21.15.3:0/1764149710 >> 172.21.15.3:0/1764149710 conn(0x55bdb1d9d8f0 msgr2=0x55bdb1d9ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 -- 172.21.15.3:0/1764149710 shutdown_connections 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 -- 172.21.15.3:0/1764149710 wait complete. 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 -- start start 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.812+0000 7f019c56e740 1 Processor -- start 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f019c56e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1db3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f019c56e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1da67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f019c56e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1da6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f019c56e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941b3880 con 0x55bdb1dac6c0 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f019c56e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941b4b80 con 0x55bdb1da2fd0 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f019c56e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941b3ec0 con 0x55bdb1da38f0 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f019c56e740 10 client.0 _ll_register_callbacks cb 0x7f018c0985a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0192a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1db3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0193400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1da6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0192a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1db3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:43596/0 (socket says 172.21.15.3:43596) 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0193400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1da6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53034/0 (socket says 172.21.15.3:53034) 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0192a00640 1 -- 172.21.15.3:0/1531512082 learned_addr learned my addr 172.21.15.3:0/1531512082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f018be00640 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1da67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0193400640 1 -- 172.21.15.3:0/1531512082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 msgr2=0x55bdb1da67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0193400640 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1da67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0193400640 1 -- 172.21.15.3:0/1531512082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 msgr2=0x55bdb1db3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0193400640 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1db3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f0193400640 1 -- 172.21.15.3:0/1531512082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdb1da7260 con 0x55bdb1dac6c0 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.813+0000 7f018be00640 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1da67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f0193400640 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1da6d20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55bdb1daaba0 tx=0x7f018c0984a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce84c4e9a349645 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f0180c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f0180c00640 1 -- 172.21.15.3:0/1531512082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c09c0d0 con 0x55bdb1dac6c0 2024-03-21T08:32:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f0180c00640 1 -- 172.21.15.3:0/1531512082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f018c0d0160 con 0x55bdb1dac6c0 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f018c0cda10 con 0x55bdb1dac6c0 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f0180c00640 1 -- 172.21.15.3:0/1531512082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f018c0d0360 con 0x55bdb1dac6c0 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f019c56e740 10 client.15753 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.814+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bdb1e18bf0 con 0x55bdb1dac6c0 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f0180c00640 1 -- 172.21.15.3:0/1531512082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f018c0cda10 con 0x55bdb1dac6c0 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f0180c00640 1 -- 172.21.15.3:0/1531512082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f018c09b3c0 con 0x55bdb1dac6c0 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f019c56e740 10 client.15753 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f019c56e740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f019c56e740 20 client.15753 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f019c56e740 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1e19000 0x55bdb1e1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f019c56e740 4 client.15753 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdb1d0f0c0 con 0x55bdb1e19000 2024-03-21T08:32:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.815+0000 7f0192a00640 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1e19000 0x55bdb1e1b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.816+0000 7f0192a00640 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1e19000 0x55bdb1e1b420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb2d220ad39bac2 server_cookie=6af529c25f3e4fae in_seq=0 out_seq=0 2024-03-21T08:32:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.816+0000 7f0180c00640 10 client.15753 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f0180c00640 1 -- 172.21.15.3:0/1531512082 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdb1d0f0c0 con 0x55bdb1e19000 2024-03-21T08:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f0180c00640 10 client.15753 handle_command_reply: tid=0 2024-03-21T08:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 client.15753 shutdown 2024-03-21T08:32:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 msgr2=0x55bdb1da6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1da6d20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55bdb1daaba0 tx=0x7f018c0984a0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 shutdown_connections 2024-03-21T08:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bdb1dac6c0 0x55bdb1da6d20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55bdb1e19000 0x55bdb1e1b420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bdb1da38f0 0x55bdb1da67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 --2- 172.21.15.3:0/1531512082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bdb1da2fd0 0x55bdb1db3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 >> 172.21.15.3:0/1531512082 conn(0x55bdb1d9d8f0 msgr2=0x55bdb1d9f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.840+0000 7f0180c00640 0 client.15753 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.841+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 shutdown_connections 2024-03-21T08:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.841+0000 7f019c56e740 1 -- 172.21.15.3:0/1531512082 wait complete. 2024-03-21T08:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.841+0000 7f019c56e740 20 client.15753 trim_cache size 0 max 16384 2024-03-21T08:32:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:32:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:32:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:32:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:32:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f01780a31a0 msgr2=0x7f01780a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f01780a31a0 0x7f01780a5660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55bdb1da2a80 tx=0x7f018400b480 comp rx=0 tx=0).stop 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941a2d70 msgr2=0x7f01941a3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941a2d70 0x7f01941a3220 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f018c005fd0 tx=0x7f018c00ee60 comp rx=0 tx=0).stop 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 shutdown_connections 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01941a2d70 0x7f01941a3220 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f01780a31a0 0x7f01780a5660 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f019410cfc0 0x7f019419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 --2- 172.21.15.3:0/1986374869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0194107f40 0x7f019419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 >> 172.21.15.3:0/1986374869 conn(0x7f01940774e0 msgr2=0x7f0194107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 shutdown_connections 2024-03-21T08:32:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:06.842+0000 7f0199800640 1 -- 172.21.15.3:0/1986374869 wait complete. 2024-03-21T08:32:08.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:08 smithi084 ceph-mon[35477]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-03-21T08:32:08.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:08 smithi171 ceph-mon[35388]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-03-21T08:32:08.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:08 smithi003 ceph-mon[31675]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-03-21T08:32:10.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:10 smithi084 ceph-mon[35477]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-03-21T08:32:10.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:10 smithi171 ceph-mon[35388]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-03-21T08:32:10.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:10 smithi003 ceph-mon[31675]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-03-21T08:32:10.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:32:10.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:32:11.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 -- 172.21.15.3:0/3135270237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 msgr2=0x7fc664110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3135270237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc664110530 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009cd0 tx=0x7fc65c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 -- 172.21.15.3:0/3135270237 shutdown_connections 2024-03-21T08:32:11.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3135270237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc664110530 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3135270237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc664103660 0x7fc664103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3135270237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc664109670 0x7fc664109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 -- 172.21.15.3:0/3135270237 >> 172.21.15.3:0/3135270237 conn(0x7fc6640ff110 msgr2=0x7fc664101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 -- 172.21.15.3:0/3135270237 shutdown_connections 2024-03-21T08:32:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.165+0000 7fc66ac00640 1 -- 172.21.15.3:0/3135270237 wait complete. 2024-03-21T08:32:11.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.166+0000 7fc66ac00640 1 Processor -- start 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.166+0000 7fc66ac00640 1 -- start start 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc66ac00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc664103660 0x7fc664199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc66ac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc66419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc66ac00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc664109670 0x7fc66419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc66ac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc664113930 con 0x7fc664104020 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc66ac00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641137b0 con 0x7fc664103660 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc66ac00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc664113ab0 con 0x7fc664109670 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc668e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc66419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc668e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc66419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33778/0 (socket says 172.21.15.3:33778) 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc668e00640 1 -- 172.21.15.3:0/3648689930 learned_addr learned my addr 172.21.15.3:0/3648689930 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc669800640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc664103660 0x7fc664199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.167+0000 7fc66a200640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc664109670 0x7fc66419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc668e00640 1 -- 172.21.15.3:0/3648689930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc664109670 msgr2=0x7fc66419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc668e00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc664109670 0x7fc66419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc668e00640 1 -- 172.21.15.3:0/3648689930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc664103660 msgr2=0x7fc664199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc668e00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc664103660 0x7fc664199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc668e00640 1 -- 172.21.15.3:0/3648689930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66419ee60 con 0x7fc664104020 2024-03-21T08:32:11.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc668e00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc66419a3b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc65800d950 tx=0x7fc65800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc662000640 1 -- 172.21.15.3:0/3648689930 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658017030 con 0x7fc664104020 2024-03-21T08:32:11.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc662000640 1 -- 172.21.15.3:0/3648689930 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc658004020 con 0x7fc664104020 2024-03-21T08:32:11.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66419f150 con 0x7fc664104020 2024-03-21T08:32:11.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.168+0000 7fc662000640 1 -- 172.21.15.3:0/3648689930 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580047a0 con 0x7fc664104020 2024-03-21T08:32:11.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.169+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6641a69e0 con 0x7fc664104020 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 Processor -- start 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 -- start start 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc662000640 1 -- 172.21.15.3:0/3648689930 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7fc6580041d0 con 0x7fc664104020 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f50f0 0x561a367f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a36800850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a36759250 con 0x561a367f50f0 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a36701a30 con 0x561a367f5dd0 2024-03-21T08:32:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.172+0000 7fc66efa2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a367552f0 con 0x561a367fe460 2024-03-21T08:32:11.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f50f0 0x561a367f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f50f0 0x561a367f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33782/0 (socket says 172.21.15.3:33782) 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 -- 172.21.15.3:0/2483129136 learned_addr learned my addr 172.21.15.3:0/2483129136 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc66a200640 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc668e00640 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a36800850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 -- 172.21.15.3:0/2483129136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 msgr2=0x561a36800850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a36800850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 -- 172.21.15.3:0/2483129136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 msgr2=0x561a367f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.173+0000 7fc669800640 1 -- 172.21.15.3:0/2483129136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a366ec690 con 0x561a367f50f0 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.174+0000 7fc669800640 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f50f0 0x561a367f5570 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc65400b710 tx=0x7fc654004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8f686eb7e664ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.174+0000 7fc662000640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc64c0a31b0 0x7fc64c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.174+0000 7fc652a00640 1 -- 172.21.15.3:0/2483129136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654003bd0 con 0x561a367f50f0 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.174+0000 7fc652a00640 1 -- 172.21.15.3:0/2483129136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc654015070 con 0x561a367f50f0 2024-03-21T08:32:11.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.174+0000 7fc652a00640 1 -- 172.21.15.3:0/2483129136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654004d80 con 0x561a367f50f0 2024-03-21T08:32:11.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.174+0000 7fc662000640 1 -- 172.21.15.3:0/3648689930 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fc6580cf910 con 0x7fc664104020 2024-03-21T08:32:11.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66a200640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc64c0a31b0 0x7fc64c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 -- 172.21.15.3:0/2483129136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f50f0 msgr2=0x561a367f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f50f0 0x561a367f5570 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc65400b710 tx=0x7fc654004610 comp rx=0 tx=0).stop 2024-03-21T08:32:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 -- 172.21.15.3:0/2483129136 shutdown_connections 2024-03-21T08:32:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a36800850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f50f0 0x561a367f5570 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 --2- 172.21.15.3:0/2483129136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 -- 172.21.15.3:0/2483129136 >> 172.21.15.3:0/2483129136 conn(0x561a367bc200 msgr2=0x561a367efb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.175+0000 7fc66efa2740 1 -- 172.21.15.3:0/2483129136 shutdown_connections 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.176+0000 7fc66efa2740 1 -- 172.21.15.3:0/2483129136 wait complete. 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.176+0000 7fc66a200640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc64c0a31b0 0x7fc64c0a5670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005cc0 tx=0x7fc65c005c50 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.176+0000 7fc66efa2740 1 -- start start 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.176+0000 7fc66efa2740 1 Processor -- start 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367f50f0 0x561a367fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367faf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367fe460 0x561a367fb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66efa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a368012d0 con 0x561a367fe460 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66efa2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a36714e80 con 0x561a367f5dd0 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66efa2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a36801450 con 0x561a367f50f0 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66efa2740 10 client.0 _ll_register_callbacks cb 0x561a36727130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367fe460 0x561a367fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367fe460 0x561a367fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33798/0 (socket says 172.21.15.3:33798) 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc66a200640 1 -- 172.21.15.3:0/3848055796 learned_addr learned my addr 172.21.15.3:0/3848055796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc668e00640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.177+0000 7fc669800640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367f50f0 0x561a367fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc66a200640 1 -- 172.21.15.3:0/3848055796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367f50f0 msgr2=0x561a367fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc66a200640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367f50f0 0x561a367fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc66a200640 1 -- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 msgr2=0x561a367faf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc66a200640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367faf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc66a200640 1 -- 172.21.15.3:0/3848055796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a365377d0 con 0x561a367fe460 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc668e00640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc66a200640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367fe460 0x561a367fb450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc65c036000 tx=0x7fc65c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36253f50b69316 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc647400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc647400640 1 -- 172.21.15.3:0/3848055796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c051020 con 0x561a367fe460 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc647400640 1 -- 172.21.15.3:0/3848055796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fc65c003fb0 con 0x561a367fe460 2024-03-21T08:32:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.178+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc654015070 con 0x561a367fe460 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc647400640 1 -- 172.21.15.3:0/3848055796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc65c05c050 con 0x561a367fe460 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 10 client.15765 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561a3646d3f0 con 0x561a367fe460 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc647400640 1 -- 172.21.15.3:0/3848055796 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc654015070 con 0x561a367fe460 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc647400640 1 -- 172.21.15.3:0/3848055796 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc65c04d830 con 0x561a367fe460 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 10 client.15765 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 20 client.15765 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a3680e3d0 0x561a368107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 4 client.15765 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.179+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a36766790 con 0x561a3680e3d0 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.180+0000 7fc669800640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a3680e3d0 0x561a368107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.181+0000 7fc669800640 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a3680e3d0 0x561a368107e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c273cd9778ee1 server_cookie=2e851e4090f67329 in_seq=0 out_seq=0 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.181+0000 7fc647400640 10 client.15765 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc647400640 1 -- 172.21.15.3:0/3848055796 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561a36766790 con 0x561a3680e3d0 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc647400640 10 client.15765 handle_command_reply: tid=0 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 client.15765 shutdown 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367fe460 msgr2=0x561a367fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367fe460 0x561a367fb450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc65c036000 tx=0x7fc65c0095c0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 shutdown_connections 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367fe460 0x561a367fb450 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f5dd0 0x561a367faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a3680e3d0 0x561a368107e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 --2- 172.21.15.3:0/3848055796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367f50f0 0x561a367fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 >> 172.21.15.3:0/3848055796 conn(0x561a367bc200 msgr2=0x561a367f1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc647400640 0 client.15765 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:11.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.183+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 shutdown_connections 2024-03-21T08:32:11.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.184+0000 7fc66efa2740 1 -- 172.21.15.3:0/3848055796 wait complete. 2024-03-21T08:32:11.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.184+0000 7fc66efa2740 20 client.15765 trim_cache size 0 max 16384 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 Processor -- start 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 -- start start 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a36800430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367fd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641b3130 con 0x561a367f5dd0 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641b0c30 con 0x561a367f50f0 2024-03-21T08:32:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.202+0000 7fc66efa2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641aecd0 con 0x561a367fe460 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367fd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33802/0 (socket says 172.21.15.3:33802) 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 -- 172.21.15.3:0/1156685737 learned_addr learned my addr 172.21.15.3:0/1156685737 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc669800640 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a36800430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc66a200640 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 -- 172.21.15.3:0/1156685737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 msgr2=0x561a367fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 -- 172.21.15.3:0/1156685737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 msgr2=0x561a36800430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a36800430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 -- 172.21.15.3:0/1156685737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a3642d0f0 con 0x561a367f5dd0 2024-03-21T08:32:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc669800640 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a36800430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.203+0000 7fc668e00640 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367fd640 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc658097a90 tx=0x7fc65809b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2369915e41e7c6cb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.204+0000 7fc652000640 1 -- 172.21.15.3:0/1156685737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65809bd30 con 0x561a367f5dd0 2024-03-21T08:32:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.204+0000 7fc652000640 1 -- 172.21.15.3:0/1156685737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc65809b630 con 0x561a367f5dd0 2024-03-21T08:32:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.204+0000 7fc652000640 1 -- 172.21.15.3:0/1156685737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580cf470 con 0x561a367f5dd0 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 -- 172.21.15.3:0/1156685737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 msgr2=0x561a367fd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367fd640 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc658097a90 tx=0x7fc65809b400 comp rx=0 tx=0).stop 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 -- 172.21.15.3:0/1156685737 shutdown_connections 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367fd640 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 --2- 172.21.15.3:0/1156685737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a36800430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 -- 172.21.15.3:0/1156685737 >> 172.21.15.3:0/1156685737 conn(0x561a367bc200 msgr2=0x561a367ef670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 -- 172.21.15.3:0/1156685737 shutdown_connections 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.205+0000 7fc66efa2740 1 -- 172.21.15.3:0/1156685737 wait complete. 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.206+0000 7fc66efa2740 1 -- start start 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.206+0000 7fc66efa2740 1 Processor -- start 2024-03-21T08:32:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a367f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66efa2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66efa2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641af2d0 con 0x561a367f5dd0 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66efa2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641b05d0 con 0x561a367f50f0 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66efa2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641af910 con 0x561a367fe460 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66efa2740 10 client.0 _ll_register_callbacks cb 0x7fc65c044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33804/0 (socket says 172.21.15.3:33804) 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 -- 172.21.15.3:0/346636672 learned_addr learned my addr 172.21.15.3:0/346636672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc669800640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a367f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66a200640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 -- 172.21.15.3:0/346636672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 msgr2=0x561a367f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 -- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 msgr2=0x561a367f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a367f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc668e00640 1 -- 172.21.15.3:0/346636672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a36805a00 con 0x561a367f5dd0 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.207+0000 7fc66a200640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc668e00640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367f78c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc658005170 tx=0x7fc658005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24dfde8eb799e9d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc647400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc647400640 1 -- 172.21.15.3:0/346636672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580b9060 con 0x561a367f5dd0 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc647400640 1 -- 172.21.15.3:0/346636672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fc6580cf670 con 0x561a367f5dd0 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc65809b630 con 0x561a367f5dd0 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc647400640 1 -- 172.21.15.3:0/346636672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6580b4070 con 0x561a367f5dd0 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc66efa2740 10 client.15771 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.208+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561a36872600 con 0x561a367f5dd0 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc647400640 1 -- 172.21.15.3:0/346636672 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc65809b630 con 0x561a367f5dd0 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc647400640 1 -- 172.21.15.3:0/346636672 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc658097380 con 0x561a367f5dd0 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc66efa2740 10 client.15771 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc66efa2740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc66efa2740 20 client.15771 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc66efa2740 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a36872c50 0x561a36875070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc66efa2740 4 client.15771 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a36760ae0 con 0x561a36872c50 2024-03-21T08:32:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.209+0000 7fc669800640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a36872c50 0x561a36875070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.210+0000 7fc669800640 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a36872c50 0x561a36875070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c9e9846cdd0325 server_cookie=b323bf54934050de in_seq=0 out_seq=0 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.210+0000 7fc647400640 10 client.15771 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc647400640 1 -- 172.21.15.3:0/346636672 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561a36760ae0 con 0x561a36872c50 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc647400640 10 client.15771 handle_command_reply: tid=0 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 client.15771 shutdown 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 msgr2=0x561a367f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367f78c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc658005170 tx=0x7fc658005d50 comp rx=0 tx=0).stop 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 shutdown_connections 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561a36872c50 0x561a36875070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a367fe460 0x561a367f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561a367f5dd0 0x561a367f78c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 --2- 172.21.15.3:0/346636672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561a367f50f0 0x561a367f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc647400640 0 client.15771 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 >> 172.21.15.3:0/346636672 conn(0x561a367bc200 msgr2=0x561a367f9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 shutdown_connections 2024-03-21T08:32:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 1 -- 172.21.15.3:0/346636672 wait complete. 2024-03-21T08:32:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.211+0000 7fc66efa2740 20 client.15771 trim_cache size 0 max 16384 2024-03-21T08:32:11.215 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:32:11.215 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-03-21T08:32:11.215 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-03-21T08:32:11.215 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc64c0a31b0 msgr2=0x7fc64c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc64c0a31b0 0x7fc64c0a5670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005cc0 tx=0x7fc65c005c50 comp rx=0 tx=0).stop 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 msgr2=0x7fc66419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc66419a3b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc65800d950 tx=0x7fc65800de10 comp rx=0 tx=0).stop 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 shutdown_connections 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc64c0a31b0 0x7fc64c0a5670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc664109670 0x7fc66419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc664104020 0x7fc66419a3b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 --2- 172.21.15.3:0/3648689930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc664103660 0x7fc664199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.212+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 >> 172.21.15.3:0/3648689930 conn(0x7fc6640ff110 msgr2=0x7fc6640ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.213+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 shutdown_connections 2024-03-21T08:32:11.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.213+0000 7fc66ac00640 1 -- 172.21.15.3:0/3648689930 wait complete. 2024-03-21T08:32:11.238 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T08:32:11.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 -- 172.21.15.3:0/1625475142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41058f0 msgr2=0x7f57b4105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 --2- 172.21.15.3:0/1625475142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41058f0 0x7f57b4105d80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009ca0 tx=0x7f57ac02c1d0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 -- 172.21.15.3:0/1625475142 shutdown_connections 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 --2- 172.21.15.3:0/1625475142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41058f0 0x7f57b4105d80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 --2- 172.21.15.3:0/1625475142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57b4100e10 0x7f57b41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 --2- 172.21.15.3:0/1625475142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57b41004f0 0x7f57b41008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 -- 172.21.15.3:0/1625475142 >> 172.21.15.3:0/1625475142 conn(0x7f57b40fbf00 msgr2=0x7f57b40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 -- 172.21.15.3:0/1625475142 shutdown_connections 2024-03-21T08:32:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.445+0000 7f57b9400640 1 -- 172.21.15.3:0/1625475142 wait complete. 2024-03-21T08:32:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.446+0000 7f57b9400640 1 Processor -- start 2024-03-21T08:32:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.446+0000 7f57b9400640 1 -- start start 2024-03-21T08:32:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.446+0000 7f57b9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41004f0 0x7f57b419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.446+0000 7f57b9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57b4100e10 0x7f57b419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b9400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57b41058f0 0x7f57b41a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b9400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4117de0 con 0x7f57b41004f0 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b9400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4117c60 con 0x7f57b4100e10 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b9400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4117f60 con 0x7f57b41058f0 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41004f0 0x7f57b419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41004f0 0x7f57b419e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33818/0 (socket says 172.21.15.3:33818) 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 -- 172.21.15.3:0/2764223275 learned_addr learned my addr 172.21.15.3:0/2764223275 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57abe00640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57b4100e10 0x7f57b419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 -- 172.21.15.3:0/2764223275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57b41058f0 msgr2=0x7f57b41a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57b41058f0 0x7f57b41a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 -- 172.21.15.3:0/2764223275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57b4100e10 msgr2=0x7f57b419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57b4100e10 0x7f57b419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57b3e00640 1 -- 172.21.15.3:0/2764223275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b41a3330 con 0x7f57b41004f0 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.447+0000 7f57abe00640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57b4100e10 0x7f57b419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.448+0000 7f57b3e00640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41004f0 0x7f57b419e340 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57a400d950 tx=0x7f57a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.448+0000 7f57b1600640 1 -- 172.21.15.3:0/2764223275 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4017030 con 0x7f57b41004f0 2024-03-21T08:32:11.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.448+0000 7f57b1600640 1 -- 172.21.15.3:0/2764223275 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57a4004020 con 0x7f57b41004f0 2024-03-21T08:32:11.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.448+0000 7f57b1600640 1 -- 172.21.15.3:0/2764223275 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a40047a0 con 0x7f57b41004f0 2024-03-21T08:32:11.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.448+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b41a3620 con 0x7f57b41004f0 2024-03-21T08:32:11.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.448+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b41aaeb0 con 0x7f57b41004f0 2024-03-21T08:32:11.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.449+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b404f470 con 0x7f57b41004f0 2024-03-21T08:32:11.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.452+0000 7f57b1600640 1 -- 172.21.15.3:0/2764223275 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f57a40041d0 con 0x7f57b41004f0 2024-03-21T08:32:11.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.453+0000 7f57b1600640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f57980a31a0 0x7f57980a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.453+0000 7f57b1600640 1 -- 172.21.15.3:0/2764223275 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f57a40d00f0 con 0x7f57b41004f0 2024-03-21T08:32:11.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.453+0000 7f57abe00640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f57980a31a0 0x7f57980a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.454+0000 7f57abe00640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f57980a31a0 0x7f57980a5660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f579c005fd0 tx=0x7f579c005f00 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.456+0000 7f57b1600640 1 -- 172.21.15.3:0/2764223275 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f57a409c510 con 0x7f57b41004f0 2024-03-21T08:32:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.700+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57b419f690 con 0x7f57b41004f0 2024-03-21T08:32:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.701+0000 7f57b1600640 1 -- 172.21.15.3:0/2764223275 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f57a409be10 con 0x7f57b41004f0 2024-03-21T08:32:11.702 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:32:11.702 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:31:29.351404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34472,34409]},"id":1}]} 2024-03-21T08:32:11.703 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.704+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f57980a31a0 msgr2=0x7f57980a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.704+0000 7f57b9400640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f57980a31a0 0x7f57980a5660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f579c005fd0 tx=0x7f579c005f00 comp rx=0 tx=0).stop 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41004f0 msgr2=0x7f57b419e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41004f0 0x7f57b419e340 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57a400d950 tx=0x7f57a400de10 comp rx=0 tx=0).stop 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 shutdown_connections 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f57980a31a0 0x7f57980a5660 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57b41058f0 0x7f57b41a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57b4100e10 0x7f57b419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 --2- 172.21.15.3:0/2764223275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b41004f0 0x7f57b419e340 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 >> 172.21.15.3:0/2764223275 conn(0x7f57b40fbf00 msgr2=0x7f57b41032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 shutdown_connections 2024-03-21T08:32:11.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.705+0000 7f57b9400640 1 -- 172.21.15.3:0/2764223275 wait complete. 2024-03-21T08:32:11.724 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:32:11.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T08:32:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 -- 172.21.15.3:0/2191050333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101460 msgr2=0x7f67681018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 --2- 172.21.15.3:0/2191050333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101460 0x7f67681018e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6750009930 tx=0x7f675002bc30 comp rx=0 tx=0).stop 2024-03-21T08:32:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 -- 172.21.15.3:0/2191050333 shutdown_connections 2024-03-21T08:32:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 --2- 172.21.15.3:0/2191050333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6768101e20 0x7f676810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 --2- 172.21.15.3:0/2191050333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101460 0x7f67681018e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 --2- 172.21.15.3:0/2191050333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6768107500 0x7f67681078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 -- 172.21.15.3:0/2191050333 >> 172.21.15.3:0/2191050333 conn(0x7f67680fcff0 msgr2=0x7f67680ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 -- 172.21.15.3:0/2191050333 shutdown_connections 2024-03-21T08:32:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.920+0000 7f6767200640 1 -- 172.21.15.3:0/2191050333 wait complete. 2024-03-21T08:32:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 Processor -- start 2024-03-21T08:32:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 -- start start 2024-03-21T08:32:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6768101460 0x7f67681083b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101e20 0x7f67681088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6768107500 0x7f6768108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768111720 con 0x7f6768101e20 2024-03-21T08:32:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681115a0 con 0x7f6768107500 2024-03-21T08:32:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.921+0000 7f6767200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681118a0 con 0x7f6768101460 2024-03-21T08:32:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101e20 0x7f67681088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101e20 0x7f67681088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33834/0 (socket says 172.21.15.3:33834) 2024-03-21T08:32:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 -- 172.21.15.3:0/2044221619 learned_addr learned my addr 172.21.15.3:0/2044221619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f6766800640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6768107500 0x7f6768108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f6765e00640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6768101460 0x7f67681083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 -- 172.21.15.3:0/2044221619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6768101460 msgr2=0x7f67681083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6768101460 0x7f67681083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 -- 172.21.15.3:0/2044221619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6768107500 msgr2=0x7f6768108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6768107500 0x7f6768108e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 -- 172.21.15.3:0/2044221619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67681aa5a0 con 0x7f6768101e20 2024-03-21T08:32:11.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.922+0000 7f675d400640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101e20 0x7f67681088f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6750005ab0 tx=0x7f675002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.923+0000 7f675f400640 1 -- 172.21.15.3:0/2044221619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750043020 con 0x7f6768101e20 2024-03-21T08:32:11.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.923+0000 7f675f400640 1 -- 172.21.15.3:0/2044221619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675002cc60 con 0x7f6768101e20 2024-03-21T08:32:11.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.923+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67681aa830 con 0x7f6768101e20 2024-03-21T08:32:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.923+0000 7f675f400640 1 -- 172.21.15.3:0/2044221619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750003c30 con 0x7f6768101e20 2024-03-21T08:32:11.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.923+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67681aad80 con 0x7f6768101e20 2024-03-21T08:32:11.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.926+0000 7f676f967740 1 Processor -- start 2024-03-21T08:32:11.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.926+0000 7f675f400640 1 -- 172.21.15.3:0/2044221619 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f6750046020 con 0x7f6768101e20 2024-03-21T08:32:11.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.926+0000 7f676f967740 1 -- start start 2024-03-21T08:32:11.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f676f967740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa77c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f676f967740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa78260 0x55d82fa80810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f676f967740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa80de0 0x55d82fa83230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f676f967740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82f9db2c0 con 0x55d82fa80de0 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f676f967740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82f983c90 con 0x55d82fa78260 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f676f967740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82f9d7360 con 0x55d82fa778b0 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa80de0 0x55d82fa83230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f6765e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa78260 0x55d82fa80810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa80de0 0x55d82fa83230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33838/0 (socket says 172.21.15.3:33838) 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f675d400640 1 -- 172.21.15.3:0/2702413426 learned_addr learned my addr 172.21.15.3:0/2702413426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.927+0000 7f6766800640 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa77c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6765e00640 1 -- 172.21.15.3:0/2702413426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 msgr2=0x55d82fa77c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6765e00640 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa77c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6765e00640 1 -- 172.21.15.3:0/2702413426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa80de0 msgr2=0x55d82fa83230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6765e00640 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa80de0 0x55d82fa83230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6765e00640 1 -- 172.21.15.3:0/2702413426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d82f9a8080 con 0x55d82fa78260 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f675f400640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f674c0a31b0 0x7f674c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f675f400640 1 -- 172.21.15.3:0/2044221619 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f67500f1970 con 0x7f6768101e20 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6765e00640 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa78260 0x55d82fa80810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f675800bb60 tx=0x7f675800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b3a789bcaa11bdd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6766800640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f674c0a31b0 0x7f674c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.928+0000 7f6757000640 1 -- 172.21.15.3:0/2702413426 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758015070 con 0x55d82fa78260 2024-03-21T08:32:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.929+0000 7f6757000640 1 -- 172.21.15.3:0/2702413426 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675800be50 con 0x55d82fa78260 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.929+0000 7f6757000640 1 -- 172.21.15.3:0/2702413426 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758011440 con 0x55d82fa78260 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.929+0000 7f676f967740 1 -- 172.21.15.3:0/2702413426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa78260 msgr2=0x55d82fa80810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.929+0000 7f6766800640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f674c0a31b0 0x7f674c0a5670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6760005fd0 tx=0x7f67600096a0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.929+0000 7f676f967740 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa78260 0x55d82fa80810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f675800bb60 tx=0x7f675800f730 comp rx=0 tx=0).stop 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 -- 172.21.15.3:0/2702413426 shutdown_connections 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa80de0 0x55d82fa83230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa78260 0x55d82fa80810 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 --2- 172.21.15.3:0/2702413426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa77c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 -- 172.21.15.3:0/2702413426 >> 172.21.15.3:0/2702413426 conn(0x55d82fa71f80 msgr2=0x55d82fa72360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 -- 172.21.15.3:0/2702413426 shutdown_connections 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 -- 172.21.15.3:0/2702413426 wait complete. 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.930+0000 7f676f967740 1 -- start start 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 1 Processor -- start 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa778b0 0x55d82fa806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa80de0 0x55d82fa7dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82fa83d50 con 0x55d82fa78260 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82f9970d0 con 0x55d82fa778b0 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82fa83ed0 con 0x55d82fa80de0 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f676f967740 10 client.0 _ll_register_callbacks cb 0x55d82f99be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.931+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa7da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33840/0 (socket says 172.21.15.3:33840) 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f675d400640 1 -- 172.21.15.3:0/788617050 learned_addr learned my addr 172.21.15.3:0/788617050 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f6765e00640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa778b0 0x55d82fa806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f6766800640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa80de0 0x55d82fa7dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f675d400640 1 -- 172.21.15.3:0/788617050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa80de0 msgr2=0x55d82fa7dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f675d400640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa80de0 0x55d82fa7dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f675d400640 1 -- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa778b0 msgr2=0x55d82fa806b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f675d400640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa778b0 0x55d82fa806b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f675d400640 1 -- 172.21.15.3:0/788617050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d82f6ef3f0 con 0x55d82fa78260 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f6765e00640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa778b0 0x55d82fa806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f675d400640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa7da40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f67500eecf0 tx=0x7f6750005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ce0795657100f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f674b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f674b400640 1 -- 172.21.15.3:0/788617050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500bd340 con 0x55d82fa78260 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.932+0000 7f674b400640 1 -- 172.21.15.3:0/788617050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f67500a80b0 con 0x55d82fa78260 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f675800be50 con 0x55d82fa78260 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f674b400640 1 -- 172.21.15.3:0/788617050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67500bd160 con 0x55d82fa78260 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 10 client.15789 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d82f7b97d0 con 0x55d82fa78260 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f674b400640 1 -- 172.21.15.3:0/788617050 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675800be50 con 0x55d82fa78260 2024-03-21T08:32:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f674b400640 1 -- 172.21.15.3:0/788617050 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f67500c5d80 con 0x55d82fa78260 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 10 client.15789 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 10 client.15789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 20 client.15789 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82fa90c10 0x55d82fa93020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 4 client.15789 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.933+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d82f9e8800 con 0x55d82fa90c10 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.934+0000 7f6765e00640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82fa90c10 0x55d82fa93020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.935+0000 7f6765e00640 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82fa90c10 0x55d82fa93020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cea82e15cc398ff server_cookie=44eac1c30d0418a8 in_seq=0 out_seq=0 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.935+0000 7f674b400640 10 client.15789 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f674b400640 1 -- 172.21.15.3:0/788617050 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d82f9e8800 con 0x55d82fa90c10 2024-03-21T08:32:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f674b400640 10 client.15789 handle_command_reply: tid=0 2024-03-21T08:32:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 client.15789 shutdown 2024-03-21T08:32:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 msgr2=0x55d82fa7da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa7da40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f67500eecf0 tx=0x7f6750005bd0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 shutdown_connections 2024-03-21T08:32:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82fa90c10 0x55d82fa93020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa80de0 0x55d82fa7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f674b400640 0 client.15789 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa7da40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 --2- 172.21.15.3:0/788617050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa778b0 0x55d82fa806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 >> 172.21.15.3:0/788617050 conn(0x55d82fa71f80 msgr2=0x55d82fa73cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 shutdown_connections 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.936+0000 7f676f967740 1 -- 172.21.15.3:0/788617050 wait complete. 2024-03-21T08:32:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.937+0000 7f676f967740 20 client.15789 trim_cache size 0 max 16384 2024-03-21T08:32:11.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 Processor -- start 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 -- start start 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa778b0 0x55d82fa80170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa78260 0x55d82fa806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82fa7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681b7470 con 0x55d82fa778b0 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681b4f70 con 0x55d82fa80de0 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.950+0000 7f676f967740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681b3010 con 0x55d82fa78260 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa778b0 0x55d82fa80170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa778b0 0x55d82fa80170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33844/0 (socket says 172.21.15.3:33844) 2024-03-21T08:32:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 -- 172.21.15.3:0/806852594 learned_addr learned my addr 172.21.15.3:0/806852594 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6766800640 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82fa7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f675d400640 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa78260 0x55d82fa806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 -- 172.21.15.3:0/806852594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa78260 msgr2=0x55d82fa806b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa78260 0x55d82fa806b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 -- 172.21.15.3:0/806852594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 msgr2=0x55d82fa7aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82fa7aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 -- 172.21.15.3:0/806852594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d82fa773b0 con 0x55d82fa778b0 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.951+0000 7f6765e00640 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa778b0 0x55d82fa80170 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d82fa77380 tx=0x7f675800b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43131bf9aad80e05 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f6756600640 1 -- 172.21.15.3:0/806852594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675802e070 con 0x55d82fa778b0 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f6756600640 1 -- 172.21.15.3:0/806852594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675800bd50 con 0x55d82fa778b0 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f6756600640 1 -- 172.21.15.3:0/806852594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758015040 con 0x55d82fa778b0 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f676f967740 1 -- 172.21.15.3:0/806852594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa778b0 msgr2=0x55d82fa80170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f676f967740 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa778b0 0x55d82fa80170 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d82fa77380 tx=0x7f675800b4d0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f676f967740 1 -- 172.21.15.3:0/806852594 shutdown_connections 2024-03-21T08:32:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f676f967740 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82fa7aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f676f967740 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa78260 0x55d82fa806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f676f967740 1 --2- 172.21.15.3:0/806852594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa778b0 0x55d82fa80170 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.952+0000 7f676f967740 1 -- 172.21.15.3:0/806852594 >> 172.21.15.3:0/806852594 conn(0x55d82fa71f80 msgr2=0x55d82fa71160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.953+0000 7f676f967740 1 -- 172.21.15.3:0/806852594 shutdown_connections 2024-03-21T08:32:11.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.953+0000 7f676f967740 1 -- 172.21.15.3:0/806852594 wait complete. 2024-03-21T08:32:11.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.953+0000 7f676f967740 1 -- start start 2024-03-21T08:32:11.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.953+0000 7f676f967740 1 Processor -- start 2024-03-21T08:32:11.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.953+0000 7f676f967740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.953+0000 7f676f967740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa84e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.953+0000 7f676f967740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82faeb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f676f967740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681b3610 con 0x55d82fa78260 2024-03-21T08:32:11.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f676f967740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681b4910 con 0x55d82fa80de0 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f676f967740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681b3c50 con 0x55d82fa778b0 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f676f967740 10 client.0 _ll_register_callbacks cb 0x7f6750009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa84e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa84e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33856/0 (socket says 172.21.15.3:33856) 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 -- 172.21.15.3:0/3216206836 learned_addr learned my addr 172.21.15.3:0/3216206836 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f6766800640 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82faeb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f6765e00640 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 -- 172.21.15.3:0/3216206836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 msgr2=0x55d82fa848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa848e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 -- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 msgr2=0x55d82faeb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82faeb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.954+0000 7f675d400640 1 -- 172.21.15.3:0/3216206836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d82faeba30 con 0x55d82fa78260 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f675d400640 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa84e20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6750005c40 tx=0x7f67500bbd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527acdb0aec221b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:11.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f674b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f674b400640 1 -- 172.21.15.3:0/3216206836 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500bd340 con 0x55d82fa78260 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f674b400640 1 -- 172.21.15.3:0/3216206836 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f67500b9040 con 0x55d82fa78260 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f675800bd50 con 0x55d82fa78260 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f674b400640 1 -- 172.21.15.3:0/3216206836 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67500b9240 con 0x55d82fa78260 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f676f967740 10 client.15795 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d82faecbd0 con 0x55d82fa78260 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f674b400640 1 -- 172.21.15.3:0/3216206836 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675800bd50 con 0x55d82fa78260 2024-03-21T08:32:11.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.955+0000 7f674b400640 1 -- 172.21.15.3:0/3216206836 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6750082bd0 con 0x55d82fa78260 2024-03-21T08:32:11.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.956+0000 7f676f967740 10 client.15795 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:11.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.956+0000 7f676f967740 10 client.15795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:32:11.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.956+0000 7f676f967740 20 client.15795 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:11.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.956+0000 7f676f967740 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82faed1c0 0x55d82faef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:11.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.956+0000 7f676f967740 4 client.15795 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:32:11.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.956+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d82f9e2b50 con 0x55d82faed1c0 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.956+0000 7f6765e00640 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82faed1c0 0x55d82faef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.957+0000 7f6765e00640 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82faed1c0 0x55d82faef5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b576c70c3c33b671 server_cookie=bd1491fcaea97946 in_seq=0 out_seq=0 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.957+0000 7f674b400640 10 client.15795 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.957+0000 7f674b400640 1 -- 172.21.15.3:0/3216206836 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d82f9e2b50 con 0x55d82faed1c0 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.957+0000 7f674b400640 10 client.15795 handle_command_reply: tid=0 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.957+0000 7f676f967740 1 client.15795 shutdown 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 msgr2=0x55d82fa84e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa84e20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6750005c40 tx=0x7f67500bbd10 comp rx=0 tx=0).stop 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 shutdown_connections 2024-03-21T08:32:11.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d82faed1c0 0x55d82faef5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d82fa80de0 0x55d82faeb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82fa78260 0x55d82fa84e20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f674b400640 0 client.15795 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:11.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 --2- 172.21.15.3:0/3216206836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d82fa778b0 0x55d82fa848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 >> 172.21.15.3:0/3216206836 conn(0x55d82fa71f80 msgr2=0x55d82fa78640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 shutdown_connections 2024-03-21T08:32:11.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 1 -- 172.21.15.3:0/3216206836 wait complete. 2024-03-21T08:32:11.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f676f967740 20 client.15795 trim_cache size 0 max 16384 2024-03-21T08:32:11.966 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:32:11.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f674c0a31b0 msgr2=0x7f674c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.958+0000 7f6767200640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f674c0a31b0 0x7f674c0a5670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6760005fd0 tx=0x7f67600096a0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101e20 msgr2=0x7f67681088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:11.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101e20 0x7f67681088f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6750005ab0 tx=0x7f675002c860 comp rx=0 tx=0).stop 2024-03-21T08:32:11.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 shutdown_connections 2024-03-21T08:32:11.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6768107500 0x7f6768108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6768101e20 0x7f67681088f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f674c0a31b0 0x7f674c0a5670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 --2- 172.21.15.3:0/2044221619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6768101460 0x7f67681083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:11.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 >> 172.21.15.3:0/2044221619 conn(0x7f67680fcff0 msgr2=0x7f676810cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:11.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 shutdown_connections 2024-03-21T08:32:11.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:11.959+0000 7f6767200640 1 -- 172.21.15.3:0/2044221619 wait complete. 2024-03-21T08:32:11.976 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:32:11.977 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T08:32:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.182+0000 7f9a0be00640 1 -- 172.21.15.3:0/3571544533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 msgr2=0x7f9a0410e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.182+0000 7f9a0be00640 1 --2- 172.21.15.3:0/3571544533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 0x7f9a0410e0c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009ca0 tx=0x7f99fc02c180 comp rx=0 tx=0).stop 2024-03-21T08:32:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.183+0000 7f9a0be00640 1 -- 172.21.15.3:0/3571544533 shutdown_connections 2024-03-21T08:32:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.183+0000 7f9a0be00640 1 --2- 172.21.15.3:0/3571544533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 0x7f9a0410e0c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.183+0000 7f9a0be00640 1 --2- 172.21.15.3:0/3571544533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a041031e0 0x7f9a0410b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.183+0000 7f9a0be00640 1 --2- 172.21.15.3:0/3571544533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a041028c0 0x7f9a04102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.183+0000 7f9a0be00640 1 -- 172.21.15.3:0/3571544533 >> 172.21.15.3:0/3571544533 conn(0x7f9a040776c0 msgr2=0x7f9a04077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.183+0000 7f9a0be00640 1 -- 172.21.15.3:0/3571544533 shutdown_connections 2024-03-21T08:32:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.183+0000 7f9a0be00640 1 -- 172.21.15.3:0/3571544533 wait complete. 2024-03-21T08:32:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 Processor -- start 2024-03-21T08:32:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 -- start start 2024-03-21T08:32:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a041028c0 0x7f9a04100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a041031e0 0x7f9a04100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 0x7f9a04100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041118d0 con 0x7f9a0410bc90 2024-03-21T08:32:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04111750 con 0x7f9a041028c0 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a0be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04111a50 con 0x7f9a041031e0 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.184+0000 7f9a09e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 0x7f9a04100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a041028c0 0x7f9a04100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a041028c0 0x7f9a04100510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55770/0 (socket says 172.21.15.3:55770) 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09400640 1 -- 172.21.15.3:0/2319634192 learned_addr learned my addr 172.21.15.3:0/2319634192 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a08a00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a041031e0 0x7f9a04100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09e00640 1 -- 172.21.15.3:0/2319634192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a041031e0 msgr2=0x7f9a04100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09e00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a041031e0 0x7f9a04100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09e00640 1 -- 172.21.15.3:0/2319634192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a041028c0 msgr2=0x7f9a04100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09e00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a041028c0 0x7f9a04100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09e00640 1 -- 172.21.15.3:0/2319634192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a041a63d0 con 0x7f9a0410bc90 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.185+0000 7f9a09e00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 0x7f9a04100f90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f99fc037040 tx=0x7f99fc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.186+0000 7f9a02000640 1 -- 172.21.15.3:0/2319634192 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc039040 con 0x7f9a0410bc90 2024-03-21T08:32:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.186+0000 7f9a02000640 1 -- 172.21.15.3:0/2319634192 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99fc044070 con 0x7f9a0410bc90 2024-03-21T08:32:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.186+0000 7f9a02000640 1 -- 172.21.15.3:0/2319634192 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc032850 con 0x7f9a0410bc90 2024-03-21T08:32:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.186+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a041a6660 con 0x7f9a0410bc90 2024-03-21T08:32:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.186+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a041a6ad0 con 0x7f9a0410bc90 2024-03-21T08:32:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.189+0000 7f9a0ebdd740 1 Processor -- start 2024-03-21T08:32:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.189+0000 7f9a02000640 1 -- 172.21.15.3:0/2319634192 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f99fc0329f0 con 0x7f9a0410bc90 2024-03-21T08:32:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.189+0000 7f9a0ebdd740 1 -- start start 2024-03-21T08:32:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53efc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53e542c0 con 0x55ab53ef1080 2024-03-21T08:32:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53dfcc90 con 0x55ab53ef06d0 2024-03-21T08:32:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53e50360 con 0x55ab53ef9c00 2024-03-21T08:32:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33896/0 (socket says 172.21.15.3:33896) 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 -- 172.21.15.3:0/868126056 learned_addr learned my addr 172.21.15.3:0/868126056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a08a00640 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a08a00640 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53efc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 -- 172.21.15.3:0/868126056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 msgr2=0x55ab53efc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53efc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 -- 172.21.15.3:0/868126056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 msgr2=0x55ab53ef0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a09400640 1 -- 172.21.15.3:0/868126056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab53e21080 con 0x55ab53ef1080 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.190+0000 7f9a08a00640 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53efc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.191+0000 7f9a09400640 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef9630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f99f800bdd0 tx=0x7f99f80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dacb166f2ea7f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.191+0000 7f9a02000640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99ec0a3230 0x7f99ec0a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.191+0000 7f9a02000640 1 -- 172.21.15.3:0/2319634192 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f99fc0f1f80 con 0x7f9a0410bc90 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.191+0000 7f99f6600640 1 -- 172.21.15.3:0/868126056 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8005220 con 0x55ab53ef1080 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.191+0000 7f99f6600640 1 -- 172.21.15.3:0/868126056 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99f8004c50 con 0x55ab53ef1080 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.191+0000 7f9a08a00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99ec0a3230 0x7f99ec0a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.191+0000 7f99f6600640 1 -- 172.21.15.3:0/868126056 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8011800 con 0x55ab53ef1080 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/868126056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 msgr2=0x55ab53ef9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a08a00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99ec0a3230 0x7f99ec0a56f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a04101fa0 tx=0x7f99f0014040 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef9630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f99f800bdd0 tx=0x7f99f80107a0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/868126056 shutdown_connections 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53efc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef9630 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/868126056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/868126056 >> 172.21.15.3:0/868126056 conn(0x55ab53eeada0 msgr2=0x55ab53eeb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/868126056 shutdown_connections 2024-03-21T08:32:12.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.192+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/868126056 wait complete. 2024-03-21T08:32:12.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 -- start start 2024-03-21T08:32:12.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 Processor -- start 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef06d0 0x55ab53ef9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef9c00 0x55ab53ef6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53efcb70 con 0x55ab53ef1080 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53e100d0 con 0x55ab53ef9c00 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab53efccf0 con 0x55ab53ef06d0 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.193+0000 7f9a0ebdd740 10 client.0 _ll_register_callbacks cb 0x55ab53e14e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33898/0 (socket says 172.21.15.3:33898) 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 -- 172.21.15.3:0/1106185237 learned_addr learned my addr 172.21.15.3:0/1106185237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a09400640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef06d0 0x55ab53ef9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a09e00640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef9c00 0x55ab53ef6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 -- 172.21.15.3:0/1106185237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef06d0 msgr2=0x55ab53ef9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef06d0 0x55ab53ef9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 -- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef9c00 msgr2=0x55ab53ef6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef9c00 0x55ab53ef6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 -- 172.21.15.3:0/1106185237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab53b683f0 con 0x55ab53ef1080 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a09e00640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef9c00 0x55ab53ef6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f9a08a00640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef68c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f99f001f720 tx=0x7f99f000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42337ee77cfebff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.194+0000 7f99eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f99eaa00640 1 -- 172.21.15.3:0/1106185237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f001fe50 con 0x55ab53ef1080 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f99eaa00640 1 -- 172.21.15.3:0/1106185237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f99f0019070 con 0x55ab53ef1080 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99f8004c50 con 0x55ab53ef1080 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f99eaa00640 1 -- 172.21.15.3:0/1106185237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f0032050 con 0x55ab53ef1080 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f9a0ebdd740 10 client.15807 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ab53c327d0 con 0x55ab53ef1080 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f99eaa00640 1 -- 172.21.15.3:0/1106185237 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99f8004c50 con 0x55ab53ef1080 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f99eaa00640 1 -- 172.21.15.3:0/1106185237 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f99f0027860 con 0x55ab53ef1080 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.195+0000 7f9a0ebdd740 10 client.15807 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.196+0000 7f9a0ebdd740 10 client.15807 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.196+0000 7f9a0ebdd740 20 client.15807 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.196+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f09aa0 0x55ab53f0bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.196+0000 7f9a0ebdd740 4 client.15807 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.196+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab53e61800 con 0x55ab53f09aa0 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.196+0000 7f9a09400640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f09aa0 0x55ab53f0bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.197+0000 7f9a09400640 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f09aa0 0x55ab53f0bee0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1031d9a2b0c762bc server_cookie=448425fceda47d5d in_seq=0 out_seq=0 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.197+0000 7f99eaa00640 10 client.15807 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f99eaa00640 1 -- 172.21.15.3:0/1106185237 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ab53e61800 con 0x55ab53f09aa0 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f99eaa00640 10 client.15807 handle_command_reply: tid=0 2024-03-21T08:32:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 client.15807 shutdown 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 msgr2=0x55ab53ef68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef68c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f99f001f720 tx=0x7f99f000b5b0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 shutdown_connections 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef9c00 0x55ab53ef6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef1080 0x55ab53ef68c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f09aa0 0x55ab53f0bee0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1106185237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef06d0 0x55ab53ef9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f99eaa00640 0 client.15807 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:12.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 >> 172.21.15.3:0/1106185237 conn(0x55ab53eeada0 msgr2=0x55ab53eecaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 shutdown_connections 2024-03-21T08:32:12.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.199+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1106185237 wait complete. 2024-03-21T08:32:12.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.200+0000 7f9a0ebdd740 20 client.15807 trim_cache size 0 max 16384 2024-03-21T08:32:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.213+0000 7f9a0ebdd740 1 Processor -- start 2024-03-21T08:32:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.213+0000 7f9a0ebdd740 1 -- start start 2024-03-21T08:32:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.213+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.213+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef1080 0x55ab53ef9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef9c00 0x55ab53ef3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041b32b0 con 0x55ab53ef9c00 2024-03-21T08:32:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041b0db0 con 0x55ab53ef06d0 2024-03-21T08:32:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041aee50 con 0x55ab53ef1080 2024-03-21T08:32:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a09400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a09e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef9c00 0x55ab53ef3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a08a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef1080 0x55ab53ef9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a09400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55794/0 (socket says 172.21.15.3:55794) 2024-03-21T08:32:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a09400640 1 -- 172.21.15.3:0/3124466959 learned_addr learned my addr 172.21.15.3:0/3124466959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.214+0000 7f9a09e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef9c00 0x55ab53ef3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33908/0 (socket says 172.21.15.3:33908) 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f9a09400640 1 -- 172.21.15.3:0/3124466959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef1080 msgr2=0x55ab53ef9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f9a09400640 1 --2- 172.21.15.3:0/3124466959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef1080 0x55ab53ef9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f9a09400640 1 -- 172.21.15.3:0/3124466959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef9c00 msgr2=0x55ab53ef3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f9a09400640 1 --2- 172.21.15.3:0/3124466959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef9c00 0x55ab53ef3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f9a09400640 1 -- 172.21.15.3:0/3124466959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab53ef4e10 con 0x55ab53ef06d0 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f9a09400640 1 --2- 172.21.15.3:0/3124466959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef8ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f99f800bdd0 tx=0x7f99f800ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba49af598e1bfa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f99f5c00640 1 -- 172.21.15.3:0/3124466959 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800b410 con 0x55ab53ef06d0 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f99f5c00640 1 -- 172.21.15.3:0/3124466959 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99f80292f0 con 0x55ab53ef06d0 2024-03-21T08:32:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.215+0000 7f99f5c00640 1 -- 172.21.15.3:0/3124466959 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8011a50 con 0x55ab53ef06d0 2024-03-21T08:32:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/3124466959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 msgr2=0x55ab53ef8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/3124466959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef8ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f99f800bdd0 tx=0x7f99f800ba00 comp rx=0 tx=0).stop 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/3124466959 shutdown_connections 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/3124466959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef9c00 0x55ab53ef3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/3124466959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef1080 0x55ab53ef9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/3124466959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef06d0 0x55ab53ef8ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/3124466959 >> 172.21.15.3:0/3124466959 conn(0x55ab53eeada0 msgr2=0x55ab53ee9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/3124466959 shutdown_connections 2024-03-21T08:32:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/3124466959 wait complete. 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.216+0000 7f9a0ebdd740 1 -- start start 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 1 Processor -- start 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef06d0 0x55ab53f01d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef1080 0x55ab53f000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53f00650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041af450 con 0x55ab53ef06d0 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041b0750 con 0x55ab53ef1080 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041afa90 con 0x55ab53ef9c00 2024-03-21T08:32:12.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a0ebdd740 10 client.0 _ll_register_callbacks cb 0x7f99f000ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a09400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef06d0 0x55ab53f01d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a09400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef06d0 0x55ab53f01d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33920/0 (socket says 172.21.15.3:33920) 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.217+0000 7f9a09400640 1 -- 172.21.15.3:0/1999891672 learned_addr learned my addr 172.21.15.3:0/1999891672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a08a00640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef1080 0x55ab53f000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09400640 1 -- 172.21.15.3:0/1999891672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 msgr2=0x55ab53f00650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09e00640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53f00650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09400640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53f00650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09400640 1 -- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef1080 msgr2=0x55ab53f000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09400640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef1080 0x55ab53f000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09400640 1 -- 172.21.15.3:0/1999891672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab53f00e80 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a08a00640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef1080 0x55ab53f000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09e00640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53f00650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f9a09400640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef06d0 0x55ab53f01d10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f99f80107e0 tx=0x7f99f800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ece33fccc586d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f99eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.218+0000 7f99eaa00640 1 -- 172.21.15.3:0/1999891672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8035030 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f99eaa00640 1 -- 172.21.15.3:0/1999891672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f99f80290f0 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab53e5bb50 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f99eaa00640 1 -- 172.21.15.3:0/1999891672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f8042050 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 10 client.15813 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ab53f66460 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f99eaa00640 1 -- 172.21.15.3:0/1999891672 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ab53e5bb50 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f99eaa00640 1 -- 172.21.15.3:0/1999891672 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f99f8011be0 con 0x55ab53ef06d0 2024-03-21T08:32:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 10 client.15813 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 10 client.15813 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 20 client.15813 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f66890 0x55ab53f68c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 4 client.15813 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.219+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab53dd9500 con 0x55ab53f66890 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.220+0000 7f9a08a00640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f66890 0x55ab53f68c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.221+0000 7f9a08a00640 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f66890 0x55ab53f68c80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a42e682b34cbbfe server_cookie=aa3e20a880ae6606 in_seq=0 out_seq=0 2024-03-21T08:32:12.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.221+0000 7f99eaa00640 10 client.15813 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:12.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.221+0000 7f99eaa00640 1 -- 172.21.15.3:0/1999891672 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab53dd9500 con 0x55ab53f66890 2024-03-21T08:32:12.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.221+0000 7f99eaa00640 10 client.15813 handle_command_reply: tid=0 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.221+0000 7f9a0ebdd740 1 client.15813 shutdown 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.221+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef06d0 msgr2=0x55ab53f01d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.221+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef06d0 0x55ab53f01d10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f99f80107e0 tx=0x7f99f800b410 comp rx=0 tx=0).stop 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 shutdown_connections 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ab53f66890 0x55ab53f68c80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ab53ef9c00 0x55ab53f00650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab53ef1080 0x55ab53f000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 --2- 172.21.15.3:0/1999891672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab53ef06d0 0x55ab53f01d10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f99eaa00640 0 client.15813 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 >> 172.21.15.3:0/1999891672 conn(0x55ab53eeada0 msgr2=0x55ab53ef1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 shutdown_connections 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 1 -- 172.21.15.3:0/1999891672 wait complete. 2024-03-21T08:32:12.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0ebdd740 20 client.15813 trim_cache size 0 max 16384 2024-03-21T08:32:12.229 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:32:12.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99ec0a3230 msgr2=0x7f99ec0a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0be00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99ec0a3230 0x7f99ec0a56f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a04101fa0 tx=0x7f99f0014040 comp rx=0 tx=0).stop 2024-03-21T08:32:12.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 msgr2=0x7f9a04100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.222+0000 7f9a0be00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 0x7f9a04100f90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f99fc037040 tx=0x7f99fc003ba0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 shutdown_connections 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a0410bc90 0x7f9a04100f90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99ec0a3230 0x7f99ec0a56f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a041031e0 0x7f9a04100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 --2- 172.21.15.3:0/2319634192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a041028c0 0x7f9a04100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 >> 172.21.15.3:0/2319634192 conn(0x7f9a040776c0 msgr2=0x7f9a040fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 shutdown_connections 2024-03-21T08:32:12.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.223+0000 7f9a0be00640 1 -- 172.21.15.3:0/2319634192 wait complete. 2024-03-21T08:32:12.240 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:32:12.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T08:32:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 -- 172.21.15.3:0/3635473852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe8073a70 msgr2=0x7f9fe8073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 --2- 172.21.15.3:0/3635473852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe8073a70 0x7f9fe8073e50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009900 tx=0x7f9fd802bcc0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 -- 172.21.15.3:0/3635473852 shutdown_connections 2024-03-21T08:32:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 --2- 172.21.15.3:0/3635473852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe810dce0 0x7f9fe81100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 --2- 172.21.15.3:0/3635473852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8074390 0x7f9fe810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 --2- 172.21.15.3:0/3635473852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe8073a70 0x7f9fe8073e50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 -- 172.21.15.3:0/3635473852 >> 172.21.15.3:0/3635473852 conn(0x7f9fe80fcf50 msgr2=0x7f9fe80ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 -- 172.21.15.3:0/3635473852 shutdown_connections 2024-03-21T08:32:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.441+0000 7f9feea00640 1 -- 172.21.15.3:0/3635473852 wait complete. 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.442+0000 7f9feea00640 1 Processor -- start 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.442+0000 7f9feea00640 1 -- start start 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9feea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe8073a70 0x7f9fe8199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9feea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8074390 0x7f9fe819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9feea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe810dce0 0x7f9fe819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9feea00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8113960 con 0x7f9fe810dce0 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9feea00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81137e0 con 0x7f9fe8073a70 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9feea00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8113ae0 con 0x7f9fe8074390 2024-03-21T08:32:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9fed600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe8073a70 0x7f9fe8199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9fed600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe8073a70 0x7f9fe8199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55824/0 (socket says 172.21.15.3:55824) 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9fed600640 1 -- 172.21.15.3:0/2014336903 learned_addr learned my addr 172.21.15.3:0/2014336903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.443+0000 7f9fee000640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe810dce0 0x7f9fe819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fecc00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8074390 0x7f9fe819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fed600640 1 -- 172.21.15.3:0/2014336903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8074390 msgr2=0x7f9fe819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fed600640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8074390 0x7f9fe819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fed600640 1 -- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe810dce0 msgr2=0x7f9fe819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fed600640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe810dce0 0x7f9fe819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fed600640 1 -- 172.21.15.3:0/2014336903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe819ef10 con 0x7f9fe8073a70 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fecc00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8074390 0x7f9fe819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.444+0000 7f9fed600640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe8073a70 0x7f9fe8199f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000c00 tx=0x7f9fd802d880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.445+0000 7f9fe6000640 1 -- 172.21.15.3:0/2014336903 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8043020 con 0x7f9fe8073a70 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.445+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe819f1a0 con 0x7f9fe8073a70 2024-03-21T08:32:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.445+0000 7f9fe6000640 1 -- 172.21.15.3:0/2014336903 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9fd802dd40 con 0x7f9fe8073a70 2024-03-21T08:32:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.445+0000 7f9fe6000640 1 -- 172.21.15.3:0/2014336903 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8003ae0 con 0x7f9fe8073a70 2024-03-21T08:32:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.445+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe81a6a90 con 0x7f9fe8073a70 2024-03-21T08:32:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 Processor -- start 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 -- start start 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d8a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213d8062c0 con 0x56213d8abde0 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213d7aec90 con 0x56213d8a3260 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.448+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213d802360 con 0x56213d8a28b0 2024-03-21T08:32:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33962/0 (socket says 172.21.15.3:33962) 2024-03-21T08:32:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 -- 172.21.15.3:0/1754415306 learned_addr learned my addr 172.21.15.3:0/1754415306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fee000640 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d8a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 -- 172.21.15.3:0/1754415306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 msgr2=0x56213d8a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d8a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 -- 172.21.15.3:0/1754415306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 msgr2=0x56213d8ab810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T08:32:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.449+0000 7f9fecc00640 1 -- 172.21.15.3:0/1754415306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56213d7d3080 con 0x56213d8abde0 2024-03-21T08:32:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.450+0000 7f9fecc00640 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ae230 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4008e40 tx=0x7f9fd400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711eb52be9979eff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.450+0000 7f9fe6000640 1 -- 172.21.15.3:0/2014336903 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f9fd8046020 con 0x7f9fe8073a70 2024-03-21T08:32:12.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.450+0000 7f9fde600640 1 -- 172.21.15.3:0/1754415306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400f830 con 0x56213d8abde0 2024-03-21T08:32:12.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.450+0000 7f9fde600640 1 -- 172.21.15.3:0/1754415306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9fd400e3b0 con 0x56213d8abde0 2024-03-21T08:32:12.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.450+0000 7f9fde600640 1 -- 172.21.15.3:0/1754415306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400b600 con 0x56213d8abde0 2024-03-21T08:32:12.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/1754415306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 msgr2=0x56213d8ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ae230 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4008e40 tx=0x7f9fd400d710 comp rx=0 tx=0).stop 2024-03-21T08:32:12.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/1754415306 shutdown_connections 2024-03-21T08:32:12.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ae230 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/1754415306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d8a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/1754415306 >> 172.21.15.3:0/1754415306 conn(0x56213d89cf80 msgr2=0x56213d89d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/1754415306 shutdown_connections 2024-03-21T08:32:12.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/1754415306 wait complete. 2024-03-21T08:32:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.451+0000 7f9ff2d9c740 1 -- start start 2024-03-21T08:32:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 1 Processor -- start 2024-03-21T08:32:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9fe6000640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9fcc0a30c0 0x7f9fcc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a28b0 0x56213d8ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213d8aed50 con 0x56213d8abde0 2024-03-21T08:32:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213d7c20d0 con 0x56213d8a28b0 2024-03-21T08:32:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213d8aeed0 con 0x56213d8a3260 2024-03-21T08:32:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9fe6000640 1 -- 172.21.15.3:0/2014336903 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f9fd80f1ef0 con 0x7f9fe8073a70 2024-03-21T08:32:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9ff2d9c740 10 client.0 _ll_register_callbacks cb 0x56213d7c6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:12.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.452+0000 7f9fee000640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9fcc0a30c0 0x7f9fcc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33978/0 (socket says 172.21.15.3:33978) 2024-03-21T08:32:12.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 -- 172.21.15.3:0/91949170 learned_addr learned my addr 172.21.15.3:0/91949170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fed600640 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a28b0 0x56213d8ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fecc00640 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 -- 172.21.15.3:0/91949170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 msgr2=0x56213d8a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 -- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a28b0 msgr2=0x56213d8ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a28b0 0x56213d8ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 -- 172.21.15.3:0/91949170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56213d51a3f0 con 0x56213d8abde0 2024-03-21T08:32:12.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fee000640 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8a9260 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0003860 tx=0x7f9fe0003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296696ce76d3f0f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.453+0000 7f9fd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:12.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9fee000640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9fcc0a30c0 0x7f9fcc0a5580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0012820 tx=0x7f9fe0019000 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9fd2a00640 1 -- 172.21.15.3:0/91949170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0023070 con 0x56213d8abde0 2024-03-21T08:32:12.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9fd2a00640 1 -- 172.21.15.3:0/91949170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f9fe000f990 con 0x56213d8abde0 2024-03-21T08:32:12.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fd400e3b0 con 0x56213d8abde0 2024-03-21T08:32:12.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9fd2a00640 1 -- 172.21.15.3:0/91949170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe002f050 con 0x56213d8abde0 2024-03-21T08:32:12.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9ff2d9c740 10 client.15825 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:12.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56213d5e47d0 con 0x56213d8abde0 2024-03-21T08:32:12.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.454+0000 7f9fd2a00640 1 -- 172.21.15.3:0/91949170 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9fd400e3b0 con 0x56213d8abde0 2024-03-21T08:32:12.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9fd2a00640 1 -- 172.21.15.3:0/91949170 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9fe001f960 con 0x56213d8abde0 2024-03-21T08:32:12.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9ff2d9c740 10 client.15825 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:12.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9ff2d9c740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:12.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9ff2d9c740 20 client.15825 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:12.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d8bbcc0 0x56213d8be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9ff2d9c740 4 client.15825 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:12.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56213d813800 con 0x56213d8bbcc0 2024-03-21T08:32:12.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.455+0000 7f9fed600640 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d8bbcc0 0x56213d8be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.456+0000 7f9fd2a00640 10 client.15825 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.456+0000 7f9fed600640 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d8bbcc0 0x56213d8be0e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7079f12c69ab37fb server_cookie=5d7c06887d332372 in_seq=0 out_seq=0 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9fd2a00640 1 -- 172.21.15.3:0/91949170 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56213d813800 con 0x56213d8bbcc0 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9fd2a00640 10 client.15825 handle_command_reply: tid=0 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 client.15825 shutdown 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 msgr2=0x56213d8a9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8a9260 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0003860 tx=0x7f9fe0003c60 comp rx=0 tx=0).stop 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 shutdown_connections 2024-03-21T08:32:12.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8a9260 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d8bbcc0 0x56213d8be0e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9fd2a00640 0 client.15825 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/91949170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a28b0 0x56213d8ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 >> 172.21.15.3:0/91949170 conn(0x56213d89cf80 msgr2=0x56213d89ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 shutdown_connections 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.458+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/91949170 wait complete. 2024-03-21T08:32:12.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.459+0000 7f9ff2d9c740 20 client.15825 trim_cache size 0 max 16384 2024-03-21T08:32:12.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 Processor -- start 2024-03-21T08:32:12.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 -- start start 2024-03-21T08:32:12.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d89b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81b3150 con 0x56213d8abde0 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81b0c50 con 0x56213d8a3260 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81aecf0 con 0x56213d8a28b0 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9fee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9fee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33992/0 (socket says 172.21.15.3:33992) 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.473+0000 7f9fee000640 1 -- 172.21.15.3:0/3477647314 learned_addr learned my addr 172.21.15.3:0/3477647314 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fecc00640 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fed600640 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d89b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fee000640 1 -- 172.21.15.3:0/3477647314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 msgr2=0x56213d89b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fee000640 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d89b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fee000640 1 -- 172.21.15.3:0/3477647314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 msgr2=0x56213d8ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fee000640 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fee000640 1 -- 172.21.15.3:0/3477647314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56213d8a2ff0 con 0x56213d8abde0 2024-03-21T08:32:12.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fecc00640 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:12.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fee000640 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ab7b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9fe002d040 tx=0x7f9fe00040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8802da46c6c3016 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fddc00640 1 -- 172.21.15.3:0/3477647314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0023070 con 0x56213d8abde0 2024-03-21T08:32:12.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fddc00640 1 -- 172.21.15.3:0/3477647314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9fe000b3d0 con 0x56213d8abde0 2024-03-21T08:32:12.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.474+0000 7f9fddc00640 1 -- 172.21.15.3:0/3477647314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000f790 con 0x56213d8abde0 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/3477647314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 msgr2=0x56213d8ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ab7b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9fe002d040 tx=0x7f9fe00040b0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/3477647314 shutdown_connections 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8abde0 0x56213d8ab7b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8a3260 0x56213d8ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/3477647314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a28b0 0x56213d89b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/3477647314 >> 172.21.15.3:0/3477647314 conn(0x56213d89cf80 msgr2=0x56213d8a3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/3477647314 shutdown_connections 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.475+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/3477647314 wait complete. 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 -- start start 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 Processor -- start 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8a28b0 0x56213d8b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8abde0 0x56213d8b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81af2f0 con 0x56213d8a28b0 2024-03-21T08:32:12.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81b05f0 con 0x56213d8abde0 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81af930 con 0x56213d8a3260 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.476+0000 7f9ff2d9c740 10 client.0 _ll_register_callbacks cb 0x7f9fe0004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8a28b0 0x56213d8b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8a28b0 0x56213d8b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33998/0 (socket says 172.21.15.3:33998) 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 -- 172.21.15.3:0/4002916712 learned_addr learned my addr 172.21.15.3:0/4002916712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fee000640 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8abde0 0x56213d8b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fecc00640 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 -- 172.21.15.3:0/4002916712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 msgr2=0x56213d8b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 -- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8abde0 msgr2=0x56213d8b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8abde0 0x56213d8b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 -- 172.21.15.3:0/4002916712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56213d917590 con 0x56213d8a28b0 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fed600640 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8a28b0 0x56213d8b2500 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80ba750 tx=0x7f9fd802bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da61256a563dc0c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fd2a00640 1 -- 172.21.15.3:0/4002916712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80c6790 con 0x56213d8a28b0 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.477+0000 7f9fd2a00640 1 -- 172.21.15.3:0/4002916712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f9fd8003c80 con 0x56213d8a28b0 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fe000b3d0 con 0x56213d8a28b0 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9fd2a00640 1 -- 172.21.15.3:0/4002916712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd8005490 con 0x56213d8a28b0 2024-03-21T08:32:12.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 10 client.15831 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56213d918840 con 0x56213d8a28b0 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9fd2a00640 1 -- 172.21.15.3:0/4002916712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9fe000b3d0 con 0x56213d8a28b0 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9fd2a00640 1 -- 172.21.15.3:0/4002916712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9fd80ef380 con 0x56213d8a28b0 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 10 client.15831 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 20 client.15831 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d918ea0 0x56213d91b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 4 client.15831 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.478+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56213d80db50 con 0x56213d918ea0 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.479+0000 7f9fecc00640 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d918ea0 0x56213d91b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.479+0000 7f9fecc00640 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d918ea0 0x56213d91b290 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e79acf172adefff server_cookie=362b43dbcc3bec75 in_seq=0 out_seq=0 2024-03-21T08:32:12.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.479+0000 7f9fd2a00640 10 client.15831 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9fd2a00640 1 -- 172.21.15.3:0/4002916712 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56213d80db50 con 0x56213d918ea0 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9fd2a00640 10 client.15831 handle_command_reply: tid=0 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 client.15831 shutdown 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8a28b0 msgr2=0x56213d8b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8a28b0 0x56213d8b2500 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80ba750 tx=0x7f9fd802bcc0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 shutdown_connections 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56213d8abde0 0x56213d8b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56213d918ea0 0x56213d91b290 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213d8a3260 0x56213d8b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 --2- 172.21.15.3:0/4002916712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56213d8a28b0 0x56213d8b2500 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9fd2a00640 0 client.15831 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 >> 172.21.15.3:0/4002916712 conn(0x56213d89cf80 msgr2=0x56213d89dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 shutdown_connections 2024-03-21T08:32:12.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 1 -- 172.21.15.3:0/4002916712 wait complete. 2024-03-21T08:32:12.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.480+0000 7f9ff2d9c740 20 client.15831 trim_cache size 0 max 16384 2024-03-21T08:32:12.488 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:32:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9fcc0a30c0 msgr2=0x7f9fcc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9fcc0a30c0 0x7f9fcc0a5580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0012820 tx=0x7f9fe0019000 comp rx=0 tx=0).stop 2024-03-21T08:32:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe8073a70 msgr2=0x7f9fe8199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe8073a70 0x7f9fe8199f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000c00 tx=0x7f9fd802d880 comp rx=0 tx=0).stop 2024-03-21T08:32:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 shutdown_connections 2024-03-21T08:32:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fe810dce0 0x7f9fe819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9fcc0a30c0 0x7f9fcc0a5580 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fe8074390 0x7f9fe819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 --2- 172.21.15.3:0/2014336903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe8073a70 0x7f9fe8199f20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 >> 172.21.15.3:0/2014336903 conn(0x7f9fe80fcf50 msgr2=0x7f9fe80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 shutdown_connections 2024-03-21T08:32:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:12.481+0000 7f9feea00640 1 -- 172.21.15.3:0/2014336903 wait complete. 2024-03-21T08:32:12.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:12 smithi003 ceph-mon[31675]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-21T08:32:12.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:12 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2764223275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:32:12.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:12 smithi084 ceph-mon[35477]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-21T08:32:12.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:12 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2764223275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:32:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:12 smithi171 ceph-mon[35388]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-21T08:32:12.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:12 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2764223275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:32:13.500 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T08:32:13.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:32:13.501 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T08:32:13.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 -- 172.21.15.3:0/2278416331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 msgr2=0x7f0a34110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 --2- 172.21.15.3:0/2278416331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a34110710 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009cd0 tx=0x7f0a2c02c0e0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 -- 172.21.15.3:0/2278416331 shutdown_connections 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 --2- 172.21.15.3:0/2278416331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a34110710 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 --2- 172.21.15.3:0/2278416331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a34103840 0x7f0a34103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 --2- 172.21.15.3:0/2278416331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a34109850 0x7f0a34109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 -- 172.21.15.3:0/2278416331 >> 172.21.15.3:0/2278416331 conn(0x7f0a340ff310 msgr2=0x7f0a34101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.738+0000 7f0a3b200640 1 -- 172.21.15.3:0/2278416331 shutdown_connections 2024-03-21T08:32:13.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.739+0000 7f0a3b200640 1 -- 172.21.15.3:0/2278416331 wait complete. 2024-03-21T08:32:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.739+0000 7f0a3b200640 1 Processor -- start 2024-03-21T08:32:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a3b200640 1 -- start start 2024-03-21T08:32:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a3b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a34103840 0x7f0a3419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a3b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a3419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a3b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a34109850 0x7f0a3419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a3b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a34113b10 con 0x7f0a34104200 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a3b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a34113990 con 0x7f0a34103840 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a3b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a34113c90 con 0x7f0a34109850 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a33400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a3419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a33400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a3419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34026/0 (socket says 172.21.15.3:34026) 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a33400640 1 -- 172.21.15.3:0/3367301451 learned_addr learned my addr 172.21.15.3:0/3367301451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.740+0000 7f0a33e00640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a34103840 0x7f0a3419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a39200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a34109850 0x7f0a3419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a33400640 1 -- 172.21.15.3:0/3367301451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a34109850 msgr2=0x7f0a3419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a33400640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a34109850 0x7f0a3419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a33400640 1 -- 172.21.15.3:0/3367301451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a34103840 msgr2=0x7f0a3419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a33400640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a34103840 0x7f0a3419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a33400640 1 -- 172.21.15.3:0/3367301451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3419f020 con 0x7f0a34104200 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a33e00640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a34103840 0x7f0a3419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a33400640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a3419a570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000d770 tx=0x7f0a2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a30c00640 1 -- 172.21.15.3:0/3367301451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20017070 con 0x7f0a34104200 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a30c00640 1 -- 172.21.15.3:0/3367301451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a200054d0 con 0x7f0a34104200 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3419f310 con 0x7f0a34104200 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.741+0000 7f0a30c00640 1 -- 172.21.15.3:0/3367301451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20004cf0 con 0x7f0a34104200 2024-03-21T08:32:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.742+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a341a6ba0 con 0x7f0a34104200 2024-03-21T08:32:13.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 Processor -- start 2024-03-21T08:32:13.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 -- start start 2024-03-21T08:32:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d22080 0x563722d22460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563722c86eb0 con 0x563722d2b580 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563722c2fa60 con 0x563722d229a0 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.745+0000 7f0a3e025740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563722c82f50 con 0x563722d22080 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.746+0000 7f0a30c00640 1 -- 172.21.15.3:0/3367301451 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f0a20004e90 con 0x7f0a34104200 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.746+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.746+0000 7f0a39200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d22080 0x563722d22460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.746+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55862/0 (socket says 172.21.15.3:55862) 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.746+0000 7f0a33e00640 1 -- 172.21.15.3:0/809249758 learned_addr learned my addr 172.21.15.3:0/809249758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.746+0000 7f0a39200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d22080 0x563722d22460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.3:51984/0 (socket says 172.21.15.3:51984) 2024-03-21T08:32:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.746+0000 7f0a33400640 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a33400640 1 -- 172.21.15.3:0/809249758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d22080 msgr2=0x563722d22460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a33400640 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d22080 0x563722d22460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a33400640 1 -- 172.21.15.3:0/809249758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 msgr2=0x563722d2b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a33400640 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a33400640 1 -- 172.21.15.3:0/809249758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563722a637e0 con 0x563722d2b580 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a39200640 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d22080 0x563722d22460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a33400640 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d2d970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0a200ccb00 tx=0x7f0a200d7650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc35f6bdc4dfff4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a29600640 1 -- 172.21.15.3:0/809249758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200de070 con 0x563722d2b580 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a29600640 1 -- 172.21.15.3:0/809249758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a200d7e50 con 0x563722d2b580 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a29600640 1 -- 172.21.15.3:0/809249758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200e15c0 con 0x563722d2b580 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.747+0000 7f0a30c00640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0a180a3230 0x7f0a180a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a30c00640 1 -- 172.21.15.3:0/3367301451 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f0a200cf950 con 0x7f0a34104200 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a39200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0a180a3230 0x7f0a180a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 -- 172.21.15.3:0/809249758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 msgr2=0x563722d2d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d2d970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0a200ccb00 tx=0x7f0a200d7650 comp rx=0 tx=0).stop 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 -- 172.21.15.3:0/809249758 shutdown_connections 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d2d970 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 --2- 172.21.15.3:0/809249758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d22080 0x563722d22460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 -- 172.21.15.3:0/809249758 >> 172.21.15.3:0/809249758 conn(0x563722d1c7b0 msgr2=0x563722d1cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 -- 172.21.15.3:0/809249758 shutdown_connections 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.748+0000 7f0a3e025740 1 -- 172.21.15.3:0/809249758 wait complete. 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.749+0000 7f0a39200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0a180a3230 0x7f0a180a56f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c02c620 tx=0x7f0a2c004a40 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.749+0000 7f0a3e025740 1 -- start start 2024-03-21T08:32:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.749+0000 7f0a3e025740 1 Processor -- start 2024-03-21T08:32:13.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.749+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d22080 0x563722d2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.749+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d229a0 0x563722d282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a3e025740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563722d2e370 con 0x563722d2b580 2024-03-21T08:32:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a3e025740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563722c42eb0 con 0x563722d22080 2024-03-21T08:32:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a3e025740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563722d2e4f0 con 0x563722d229a0 2024-03-21T08:32:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a3e025740 10 client.0 _ll_register_callbacks cb 0x563722c37150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d288f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34046/0 (socket says 172.21.15.3:34046) 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 -- 172.21.15.3:0/2854853528 learned_addr learned my addr 172.21.15.3:0/2854853528 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a33400640 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d229a0 0x563722d282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a33e00640 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d22080 0x563722d2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 -- 172.21.15.3:0/2854853528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d229a0 msgr2=0x563722d282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d229a0 0x563722d282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 -- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d22080 msgr2=0x563722d2af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d22080 0x563722d2af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.750+0000 7f0a39200640 1 -- 172.21.15.3:0/2854853528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563722ac3f70 con 0x563722d2b580 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a39200640 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d288f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009cd0 tx=0x7f0a2c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed4ba8504f4129f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a1e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a1e000640 1 -- 172.21.15.3:0/2854853528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c039040 con 0x563722d2b580 2024-03-21T08:32:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a1e000640 1 -- 172.21.15.3:0/2854853528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f0a2c04b660 con 0x563722d2b580 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a200d7e50 con 0x563722d2b580 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a1e000640 1 -- 172.21.15.3:0/2854853528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a2c059050 con 0x563722d2b580 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a3e025740 10 client.15843 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.751+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56372295b170 con 0x563722d2b580 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a1e000640 1 -- 172.21.15.3:0/2854853528 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a200d7e50 con 0x563722d2b580 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a1e000640 1 -- 172.21.15.3:0/2854853528 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0a2c034a50 con 0x563722d2b580 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a3e025740 10 client.15843 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a3e025740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a3e025740 20 client.15843 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722d3b660 0x563722d3da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a3e025740 4 client.15843 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.752+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563722c943f0 con 0x563722d3b660 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.753+0000 7f0a33e00640 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722d3b660 0x563722d3da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.753+0000 7f0a33e00640 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722d3b660 0x563722d3da90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa003d22435ccd65 server_cookie=882e30378008adc6 in_seq=0 out_seq=0 2024-03-21T08:32:13.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.753+0000 7f0a1e000640 10 client.15843 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a1e000640 1 -- 172.21.15.3:0/2854853528 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563722c943f0 con 0x563722d3b660 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a1e000640 10 client.15843 handle_command_reply: tid=0 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 client.15843 shutdown 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 msgr2=0x563722d288f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d288f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009cd0 tx=0x7f0a2c0095c0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 shutdown_connections 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d2b580 0x563722d288f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722d3b660 0x563722d3da90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d229a0 0x563722d282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2854853528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d22080 0x563722d2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a1e000640 0 client.15843 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.755+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 >> 172.21.15.3:0/2854853528 conn(0x563722d1c7b0 msgr2=0x563722d1e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.756+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 shutdown_connections 2024-03-21T08:32:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.756+0000 7f0a3e025740 1 -- 172.21.15.3:0/2854853528 wait complete. 2024-03-21T08:32:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.756+0000 7f0a3e025740 20 client.15843 trim_cache size 0 max 16384 2024-03-21T08:32:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 Processor -- start 2024-03-21T08:32:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 -- start start 2024-03-21T08:32:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d22080 0x563722d2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341b3260 con 0x563722d22080 2024-03-21T08:32:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341b0d60 con 0x563722d229a0 2024-03-21T08:32:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.773+0000 7f0a3e025740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341aee00 con 0x563722d2b580 2024-03-21T08:32:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d22080 0x563722d2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55886/0 (socket says 172.21.15.3:55886) 2024-03-21T08:32:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33400640 1 -- 172.21.15.3:0/1239637833 learned_addr learned my addr 172.21.15.3:0/1239637833 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a39200640 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33e00640 1 -- 172.21.15.3:0/1239637833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 msgr2=0x563722d235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33e00640 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33e00640 1 -- 172.21.15.3:0/1239637833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 msgr2=0x563722d2af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33e00640 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.774+0000 7f0a33e00640 1 -- 172.21.15.3:0/1239637833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563722d17c20 con 0x563722d22080 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.775+0000 7f0a33e00640 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d22080 0x563722d2aa10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x563722d3e2d0 tx=0x7f0a24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05f499f5e3afac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.775+0000 7f0a28c00640 1 -- 172.21.15.3:0/1239637833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2401d680 con 0x563722d22080 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.775+0000 7f0a28c00640 1 -- 172.21.15.3:0/1239637833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a24022070 con 0x563722d22080 2024-03-21T08:32:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.775+0000 7f0a28c00640 1 -- 172.21.15.3:0/1239637833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24011b60 con 0x563722d22080 2024-03-21T08:32:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 -- 172.21.15.3:0/1239637833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d22080 msgr2=0x563722d2aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d22080 0x563722d2aa10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x563722d3e2d0 tx=0x7f0a24000bc0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 -- 172.21.15.3:0/1239637833 shutdown_connections 2024-03-21T08:32:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d229a0 0x563722d2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 --2- 172.21.15.3:0/1239637833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d22080 0x563722d2aa10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 -- 172.21.15.3:0/1239637833 >> 172.21.15.3:0/1239637833 conn(0x563722d1c7b0 msgr2=0x563722d22ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 -- 172.21.15.3:0/1239637833 shutdown_connections 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 -- 172.21.15.3:0/1239637833 wait complete. 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.776+0000 7f0a3e025740 1 -- start start 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 1 Processor -- start 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d229a0 0x563722d2fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d305d0 0x563722d9f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341af400 con 0x563722d229a0 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341b0700 con 0x563722d305d0 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341afa40 con 0x563722d2b580 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a3e025740 10 client.0 _ll_register_callbacks cb 0x7f0a2c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d229a0 0x563722d2fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d229a0 0x563722d2fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34078/0 (socket says 172.21.15.3:34078) 2024-03-21T08:32:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.777+0000 7f0a33e00640 1 -- 172.21.15.3:0/2039352102 learned_addr learned my addr 172.21.15.3:0/2039352102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a39200640 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d305d0 0x563722d9f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a33400640 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a33e00640 1 -- 172.21.15.3:0/2039352102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 msgr2=0x563722d30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a33e00640 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d30090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a33e00640 1 -- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d305d0 msgr2=0x563722d9f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a33e00640 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d305d0 0x563722d9f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a33e00640 1 -- 172.21.15.3:0/2039352102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563722d28ef0 con 0x563722d229a0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a33e00640 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d229a0 0x563722d2fb50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400bdf0 tx=0x7f0a240056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831e557e6c868275 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a1e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.778+0000 7f0a1e000640 1 -- 172.21.15.3:0/2039352102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2401d880 con 0x563722d229a0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a1e000640 1 -- 172.21.15.3:0/2039352102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f0a2401da20 con 0x563722d229a0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a24022070 con 0x563722d229a0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a1e000640 1 -- 172.21.15.3:0/2039352102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a2403b050 con 0x563722d229a0 2024-03-21T08:32:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 10 client.15849 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563722d1d970 con 0x563722d229a0 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a1e000640 1 -- 172.21.15.3:0/2039352102 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a24022070 con 0x563722d229a0 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a1e000640 1 -- 172.21.15.3:0/2039352102 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0a24032ae0 con 0x563722d229a0 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 10 client.15849 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 20 client.15849 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722da1120 0x563722da3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 4 client.15849 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T08:32:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.779+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563722c8e740 con 0x563722da1120 2024-03-21T08:32:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.780+0000 7f0a33400640 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722da1120 0x563722da3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.781+0000 7f0a33400640 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722da1120 0x563722da3540 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b146f3a28d52ef8 server_cookie=df487fcaa45d4734 in_seq=0 out_seq=0 2024-03-21T08:32:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.781+0000 7f0a1e000640 10 client.15849 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a1e000640 1 -- 172.21.15.3:0/2039352102 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563722c8e740 con 0x563722da1120 2024-03-21T08:32:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a1e000640 10 client.15849 handle_command_reply: tid=0 2024-03-21T08:32:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 client.15849 shutdown 2024-03-21T08:32:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d229a0 msgr2=0x563722d2fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d229a0 0x563722d2fb50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400bdf0 tx=0x7f0a240056d0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 shutdown_connections 2024-03-21T08:32:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563722d305d0 0x563722d9f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563722da1120 0x563722da3540 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563722d2b580 0x563722d30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 --2- 172.21.15.3:0/2039352102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563722d229a0 0x563722d2fb50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 >> 172.21.15.3:0/2039352102 conn(0x563722d1c7b0 msgr2=0x563722d23750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a1e000640 0 client.15849 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 shutdown_connections 2024-03-21T08:32:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.784+0000 7f0a3e025740 1 -- 172.21.15.3:0/2039352102 wait complete. 2024-03-21T08:32:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3e025740 20 client.15849 trim_cache size 0 max 16384 2024-03-21T08:32:13.787 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:32:13.788 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-03-21T08:32:13.788 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "a10a11d6-5897-4711-a066-007f889dc5b7", 2024-03-21T08:32:13.788 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-03-21T08:32:13.788 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:32:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0a180a3230 msgr2=0x7f0a180a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0a180a3230 0x7f0a180a56f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c02c620 tx=0x7f0a2c004a40 comp rx=0 tx=0).stop 2024-03-21T08:32:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 msgr2=0x7f0a3419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a3419a570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000d770 tx=0x7f0a2000dc30 comp rx=0 tx=0).stop 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 shutdown_connections 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0a180a3230 0x7f0a180a56f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a34109850 0x7f0a3419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34104200 0x7f0a3419a570 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 --2- 172.21.15.3:0/3367301451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a34103840 0x7f0a3419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.785+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 >> 172.21.15.3:0/3367301451 conn(0x7f0a340ff310 msgr2=0x7f0a340ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.786+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 shutdown_connections 2024-03-21T08:32:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:13.786+0000 7f0a3b200640 1 -- 172.21.15.3:0/3367301451 wait complete. 2024-03-21T08:32:13.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:32:13.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:32:14.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 -- 172.21.15.3:0/915591905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 msgr2=0x7fa360103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 --2- 172.21.15.3:0/915591905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 0x7fa360103cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009900 tx=0x7fa34c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:32:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 -- 172.21.15.3:0/915591905 shutdown_connections 2024-03-21T08:32:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 --2- 172.21.15.3:0/915591905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360104200 0x7fa360110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 --2- 172.21.15.3:0/915591905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 0x7fa360103cc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 --2- 172.21.15.3:0/915591905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360109850 0x7fa360109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 -- 172.21.15.3:0/915591905 >> 172.21.15.3:0/915591905 conn(0x7fa3600ff310 msgr2=0x7fa360101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 -- 172.21.15.3:0/915591905 shutdown_connections 2024-03-21T08:32:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.022+0000 7fa364e00640 1 -- 172.21.15.3:0/915591905 wait complete. 2024-03-21T08:32:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.023+0000 7fa364e00640 1 Processor -- start 2024-03-21T08:32:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.023+0000 7fa364e00640 1 -- start start 2024-03-21T08:32:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa364e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 0x7fa36019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa364e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360104200 0x7fa36019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa364e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360109850 0x7fa36019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa364e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360113b60 con 0x7fa360104200 2024-03-21T08:32:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa364e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601139e0 con 0x7fa360103840 2024-03-21T08:32:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa364e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360113ce0 con 0x7fa360109850 2024-03-21T08:32:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa35d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360104200 0x7fa36019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa35d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360104200 0x7fa36019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34104/0 (socket says 172.21.15.3:34104) 2024-03-21T08:32:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa35d400640 1 -- 172.21.15.3:0/3336803196 learned_addr learned my addr 172.21.15.3:0/3336803196 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa35de00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 0x7fa36019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.024+0000 7fa35e800640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360109850 0x7fa36019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa35d400640 1 -- 172.21.15.3:0/3336803196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360109850 msgr2=0x7fa36019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa35d400640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360109850 0x7fa36019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa35d400640 1 -- 172.21.15.3:0/3336803196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 msgr2=0x7fa36019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa35d400640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 0x7fa36019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa35d400640 1 -- 172.21.15.3:0/3336803196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36019f060 con 0x7fa360104200 2024-03-21T08:32:14.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa35e800640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360109850 0x7fa36019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:14.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa35d400640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360104200 0x7fa36019a600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa34c000c00 tx=0x7fa34c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:14.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa356a00640 1 -- 172.21.15.3:0/3336803196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c043020 con 0x7fa360104200 2024-03-21T08:32:14.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36019f2f0 con 0x7fa360104200 2024-03-21T08:32:14.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.025+0000 7fa356a00640 1 -- 172.21.15.3:0/3336803196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa34c02cbd0 con 0x7fa360104200 2024-03-21T08:32:14.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.026+0000 7fa356a00640 1 -- 172.21.15.3:0/3336803196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c003bf0 con 0x7fa360104200 2024-03-21T08:32:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.026+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3601a6be0 con 0x7fa360104200 2024-03-21T08:32:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 Processor -- start 2024-03-21T08:32:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 -- start start 2024-03-21T08:32:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa356a00640 1 -- 172.21.15.3:0/3336803196 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7fa34c046020 con 0x7fa360104200 2024-03-21T08:32:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fb9c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036fb01250 con 0x56036fba6290 2024-03-21T08:32:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036faa9a30 con 0x56036fb9cf20 2024-03-21T08:32:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa367bbc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036fafd2f0 con 0x56036fb9dc00 2024-03-21T08:32:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.029+0000 7fa35d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34114/0 (socket says 172.21.15.3:34114) 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 -- 172.21.15.3:0/2030498310 learned_addr learned my addr 172.21.15.3:0/2030498310 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35de00640 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35e800640 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fb9c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 -- 172.21.15.3:0/2030498310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 msgr2=0x56036fb9c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fb9c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 -- 172.21.15.3:0/2030498310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 msgr2=0x56036fb9d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 -- 172.21.15.3:0/2030498310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036fa94690 con 0x56036fba6290 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35e800640 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fb9c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35de00640 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.030+0000 7fa35d400640 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba8680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ef3e0 tx=0x7fa34c0fa1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798aa282133d447e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.031+0000 7fa34b400640 1 -- 172.21.15.3:0/2030498310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0fa9a0 con 0x56036fba6290 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.031+0000 7fa34b400640 1 -- 172.21.15.3:0/2030498310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa34c0ff030 con 0x56036fba6290 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.031+0000 7fa34b400640 1 -- 172.21.15.3:0/2030498310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c1033d0 con 0x56036fba6290 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.031+0000 7fa356a00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa3440a3230 0x7fa3440a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.031+0000 7fa356a00640 1 -- 172.21.15.3:0/3336803196 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fa34c0f2010 con 0x7fa360104200 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.031+0000 7fa35e800640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa3440a3230 0x7fa3440a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.031+0000 7fa367bbc740 1 -- 172.21.15.3:0/2030498310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 msgr2=0x56036fba8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba8680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ef3e0 tx=0x7fa34c0fa1a0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 -- 172.21.15.3:0/2030498310 shutdown_connections 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba8680 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 --2- 172.21.15.3:0/2030498310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fb9c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 -- 172.21.15.3:0/2030498310 >> 172.21.15.3:0/2030498310 conn(0x56036fb64200 msgr2=0x56036fb97970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 -- 172.21.15.3:0/2030498310 shutdown_connections 2024-03-21T08:32:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa35e800640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa3440a3230 0x7fa3440a56f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3580095f0 tx=0x7fa358009580 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 -- 172.21.15.3:0/2030498310 wait complete. 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.032+0000 7fa367bbc740 1 -- start start 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 1 Processor -- start 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fba5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036fba8ec0 con 0x56036fba6290 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036fabce80 con 0x56036fb9cf20 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036fba9040 con 0x56036fb9dc00 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.033+0000 7fa367bbc740 10 client.0 _ll_register_callbacks cb 0x56036facf130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34122/0 (socket says 172.21.15.3:34122) 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 -- 172.21.15.3:0/3386536251 learned_addr learned my addr 172.21.15.3:0/3386536251 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35de00640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fba5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35d400640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 -- 172.21.15.3:0/3386536251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 msgr2=0x56036fba2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 -- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 msgr2=0x56036fba5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fba5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 -- 172.21.15.3:0/3386536251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56036f8df7d0 con 0x56036fba6290 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35d400640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa35e800640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba3200 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa358009970 tx=0x7fa358004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bd6b74a0bf5ece server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.034+0000 7fa33fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.035+0000 7fa33fe00640 1 -- 172.21.15.3:0/3386536251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35802c060 con 0x56036fba6290 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.035+0000 7fa33fe00640 1 -- 172.21.15.3:0/3386536251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fa358010930 con 0x56036fba6290 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.035+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa34c0ff030 con 0x56036fba6290 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.035+0000 7fa33fe00640 1 -- 172.21.15.3:0/3386536251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa358033050 con 0x56036fba6290 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.035+0000 7fa367bbc740 10 client.15861 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.035+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56036f8153f0 con 0x56036fba6290 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.035+0000 7fa33fe00640 1 -- 172.21.15.3:0/3386536251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa34c0ff030 con 0x56036fba6290 2024-03-21T08:32:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa33fe00640 1 -- 172.21.15.3:0/3386536251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa35800f040 con 0x56036fba6290 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa367bbc740 10 client.15861 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa367bbc740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa367bbc740 20 client.15861 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa367bbc740 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fbb5ef0 0x56036fbb82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa367bbc740 4 client.15861 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56036fb0e790 con 0x56036fbb5ef0 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.036+0000 7fa35de00640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fbb5ef0 0x56036fbb82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.037+0000 7fa35de00640 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fbb5ef0 0x56036fbb82e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8762ca92b039dc43 server_cookie=40bade55d5cb4e35 in_seq=0 out_seq=0 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.037+0000 7fa33fe00640 10 client.15861 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa33fe00640 1 -- 172.21.15.3:0/3386536251 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56036fb0e790 con 0x56036fbb5ef0 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa33fe00640 10 client.15861 handle_command_reply: tid=0 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 client.15861 shutdown 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 msgr2=0x56036fba3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba3200 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa358009970 tx=0x7fa358004a40 comp rx=0 tx=0).stop 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 shutdown_connections 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba3200 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fbb5ef0 0x56036fbb82e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 --2- 172.21.15.3:0/3386536251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fba5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa33fe00640 0 client.15861 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 >> 172.21.15.3:0/3386536251 conn(0x56036fb64200 msgr2=0x56036fb996c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 shutdown_connections 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.038+0000 7fa367bbc740 1 -- 172.21.15.3:0/3386536251 wait complete. 2024-03-21T08:32:14.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.039+0000 7fa367bbc740 20 client.15861 trim_cache size 0 max 16384 2024-03-21T08:32:14.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 Processor -- start 2024-03-21T08:32:14.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 -- start start 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9cf20 0x56036fba32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9dc00 0x56036fba53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601b3390 con 0x56036fba6290 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601b0e90 con 0x56036fb9dc00 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.056+0000 7fa367bbc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601aef30 con 0x56036fb9cf20 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34128/0 (socket says 172.21.15.3:34128) 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 -- 172.21.15.3:0/1659527403 learned_addr learned my addr 172.21.15.3:0/1659527403 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35d400640 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9dc00 0x56036fba53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 -- 172.21.15.3:0/1659527403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9cf20 msgr2=0x56036fba32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9cf20 0x56036fba32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 -- 172.21.15.3:0/1659527403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9dc00 msgr2=0x56036fba53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9dc00 0x56036fba53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.057+0000 7fa35e800640 1 -- 172.21.15.3:0/1659527403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036fbb8650 con 0x56036fba6290 2024-03-21T08:32:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.058+0000 7fa35e800640 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba5930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa358009860 tx=0x7fa358012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e9ea267073351d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.058+0000 7fa34aa00640 1 -- 172.21.15.3:0/1659527403 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358009da0 con 0x56036fba6290 2024-03-21T08:32:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.058+0000 7fa34aa00640 1 -- 172.21.15.3:0/1659527403 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa35803c070 con 0x56036fba6290 2024-03-21T08:32:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.058+0000 7fa34aa00640 1 -- 172.21.15.3:0/1659527403 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358027a80 con 0x56036fba6290 2024-03-21T08:32:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.058+0000 7fa367bbc740 1 -- 172.21.15.3:0/1659527403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 msgr2=0x56036fba5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.058+0000 7fa367bbc740 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba5930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa358009860 tx=0x7fa358012e10 comp rx=0 tx=0).stop 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 -- 172.21.15.3:0/1659527403 shutdown_connections 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fba5930 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9dc00 0x56036fba53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 --2- 172.21.15.3:0/1659527403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9cf20 0x56036fba32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 -- 172.21.15.3:0/1659527403 >> 172.21.15.3:0/1659527403 conn(0x56036fb64200 msgr2=0x56036fba15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 -- 172.21.15.3:0/1659527403 shutdown_connections 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 -- 172.21.15.3:0/1659527403 wait complete. 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 -- start start 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.059+0000 7fa367bbc740 1 Processor -- start 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa367bbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fc107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa367bbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601af530 con 0x56036fba6290 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa367bbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601b0830 con 0x56036fb9cf20 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa367bbc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601afb70 con 0x56036fb9dc00 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa367bbc740 10 client.0 _ll_register_callbacks cb 0x7fa358009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fc107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fc107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34138/0 (socket says 172.21.15.3:34138) 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 -- 172.21.15.3:0/981029042 learned_addr learned my addr 172.21.15.3:0/981029042 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35de00640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35d400640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 -- 172.21.15.3:0/981029042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 msgr2=0x56036fba0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 -- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 msgr2=0x56036fb9fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35e800640 1 -- 172.21.15.3:0/981029042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56036f7d50f0 con 0x56036fba6290 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35de00640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.060+0000 7fa35d400640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa35e800640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fc107b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa358027d10 tx=0x7fa358009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e8f7b2381bbd6d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa33fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa33fe00640 1 -- 172.21.15.3:0/981029042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3580279c0 con 0x56036fba6290 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa33fe00640 1 -- 172.21.15.3:0/981029042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fa358004a40 con 0x56036fba6290 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa35803c070 con 0x56036fba6290 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa33fe00640 1 -- 172.21.15.3:0/981029042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa358050050 con 0x56036fba6290 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa367bbc740 10 client.15867 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.061+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56036fc11de0 con 0x56036fba6290 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa33fe00640 1 -- 172.21.15.3:0/981029042 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa35803c070 con 0x56036fba6290 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa33fe00640 1 -- 172.21.15.3:0/981029042 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa35801bcf0 con 0x56036fba6290 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa367bbc740 10 client.15867 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa367bbc740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa367bbc740 20 client.15867 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa367bbc740 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fc12410 0x56036fc14830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa367bbc740 4 client.15867 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.062+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56036fb08ae0 con 0x56036fc12410 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.063+0000 7fa35de00640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fc12410 0x56036fc14830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.063+0000 7fa35de00640 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fc12410 0x56036fc14830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a60df223231915f server_cookie=9d3ecb7b1649710a in_seq=0 out_seq=0 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.063+0000 7fa33fe00640 10 client.15867 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa33fe00640 1 -- 172.21.15.3:0/981029042 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56036fb08ae0 con 0x56036fc12410 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa33fe00640 10 client.15867 handle_command_reply: tid=0 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 client.15867 shutdown 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 msgr2=0x56036fc107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fc107b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa358027d10 tx=0x7fa358009e90 comp rx=0 tx=0).stop 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 shutdown_connections 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56036fba6290 0x56036fc107b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56036fc12410 0x56036fc14830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56036fb9dc00 0x56036fba0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 --2- 172.21.15.3:0/981029042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56036fb9cf20 0x56036fb9fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa33fe00640 0 client.15867 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 >> 172.21.15.3:0/981029042 conn(0x56036fb64200 msgr2=0x56036fb96200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 shutdown_connections 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.064+0000 7fa367bbc740 1 -- 172.21.15.3:0/981029042 wait complete. 2024-03-21T08:32:14.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa367bbc740 20 client.15867 trim_cache size 0 max 16384 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: "a10a11d6-5897-4711-a066-007f889dc5b7": { 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "a10a11d6-5897-4711-a066-007f889dc5b7", 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:32:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:32:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa3440a3230 msgr2=0x7fa3440a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa3440a3230 0x7fa3440a56f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3580095f0 tx=0x7fa358009580 comp rx=0 tx=0).stop 2024-03-21T08:32:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360104200 msgr2=0x7fa36019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360104200 0x7fa36019a600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa34c000c00 tx=0x7fa34c0045e0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 shutdown_connections 2024-03-21T08:32:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa3440a3230 0x7fa3440a56f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360109850 0x7fa36019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa360104200 0x7fa36019a600 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 --2- 172.21.15.3:0/3336803196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa360103840 0x7fa36019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.065+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 >> 172.21.15.3:0/3336803196 conn(0x7fa3600ff310 msgr2=0x7fa3600ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.066+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 shutdown_connections 2024-03-21T08:32:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:14.066+0000 7fa364e00640 1 -- 172.21.15.3:0/3336803196 wait complete. 2024-03-21T08:32:14.084 INFO:tasks.cephfs.filesystem:scrub status for tag:a10a11d6-5897-4711-a066-007f889dc5b7 - {'path': '/', 'tag': 'a10a11d6-5897-4711-a066-007f889dc5b7', 'options': 'recursive,force'} 2024-03-21T08:32:14.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:14 smithi084 ceph-mon[35477]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-03-21T08:32:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:14 smithi084 ceph-mon[35477]: scrub queued for path: / 2024-03-21T08:32:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:14 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:14 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:32:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:14 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:14.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:14 smithi171 ceph-mon[35388]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-03-21T08:32:14.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:14 smithi171 ceph-mon[35388]: scrub queued for path: / 2024-03-21T08:32:14.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:14 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:14.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:14 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:32:14.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:14 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:14.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:14 smithi003 ceph-mon[31675]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-03-21T08:32:14.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:14 smithi003 ceph-mon[31675]: scrub queued for path: / 2024-03-21T08:32:14.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:14 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:14.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:14 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:32:14.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:14 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:16 smithi084 ceph-mon[35477]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T08:32:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:16 smithi084 ceph-mon[35477]: scrub summary: idle 2024-03-21T08:32:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:16 smithi171 ceph-mon[35388]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T08:32:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:16 smithi171 ceph-mon[35388]: scrub summary: idle 2024-03-21T08:32:16.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:16 smithi003 ceph-mon[31675]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T08:32:16.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:16 smithi003 ceph-mon[31675]: scrub summary: idle 2024-03-21T08:32:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:18 smithi084 ceph-mon[35477]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T08:32:18.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:18 smithi171 ceph-mon[35388]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T08:32:18.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:18 smithi003 ceph-mon[31675]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T08:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 -- 172.21.15.3:0/2923690004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 msgr2=0x7fa98c069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 --2- 172.21.15.3:0/2923690004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c069310 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa974009900 tx=0x7fa97402bc30 comp rx=0 tx=0).stop 2024-03-21T08:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 -- 172.21.15.3:0/2923690004 shutdown_connections 2024-03-21T08:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 --2- 172.21.15.3:0/2923690004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa98c069850 0x7fa98c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 --2- 172.21.15.3:0/2923690004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c069310 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 --2- 172.21.15.3:0/2923690004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa98c1075b0 0x7fa98c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 -- 172.21.15.3:0/2923690004 >> 172.21.15.3:0/2923690004 conn(0x7fa98c075f00 msgr2=0x7fa98c076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 -- 172.21.15.3:0/2923690004 shutdown_connections 2024-03-21T08:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.992+0000 7fa991200640 1 -- 172.21.15.3:0/2923690004 wait complete. 2024-03-21T08:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.993+0000 7fa991200640 1 Processor -- start 2024-03-21T08:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa991200640 1 -- start start 2024-03-21T08:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa991200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa991200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa98c069850 0x7fa98c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa991200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa98c1075b0 0x7fa98c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa991200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c111730 con 0x7fa98c068e90 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa991200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1115b0 con 0x7fa98c069850 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa991200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1118b0 con 0x7fa98c1075b0 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa98b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa98b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58228/0 (socket says 172.21.15.3:58228) 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.994+0000 7fa98b400640 1 -- 172.21.15.3:0/293642959 learned_addr learned my addr 172.21.15.3:0/293642959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa982a00640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa98c069850 0x7fa98c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa98be00640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa98c1075b0 0x7fa98c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa98b400640 1 -- 172.21.15.3:0/293642959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa98c1075b0 msgr2=0x7fa98c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa98b400640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa98c1075b0 0x7fa98c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa98b400640 1 -- 172.21.15.3:0/293642959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa98c069850 msgr2=0x7fa98c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa98b400640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa98c069850 0x7fa98c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa98b400640 1 -- 172.21.15.3:0/293642959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c19ee10 con 0x7fa98c068e90 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa98b400640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c199e20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00db80 tx=0x7fa97c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa988c00640 1 -- 172.21.15.3:0/293642959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c017070 con 0x7fa98c068e90 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa988c00640 1 -- 172.21.15.3:0/293642959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa97c003b70 con 0x7fa98c068e90 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.995+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c19f100 con 0x7fa98c068e90 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.996+0000 7fa988c00640 1 -- 172.21.15.3:0/293642959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0104c0 con 0x7fa98c068e90 2024-03-21T08:32:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.996+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c1a6990 con 0x7fa98c068e90 2024-03-21T08:32:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.999+0000 7fa9955e5740 1 Processor -- start 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.999+0000 7fa9955e5740 1 -- start start 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.999+0000 7fa988c00640 1 -- 172.21.15.3:0/293642959 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7fa97c004020 con 0x7fa98c068e90 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.999+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.999+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:19.999+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa9955e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623f39703b0 con 0x5623f39aeb00 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa9955e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623f3912830 con 0x5623f39ae1e0 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa9955e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623f38bb5d0 con 0x5623f39b78d0 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58236/0 (socket says 172.21.15.3:58236) 2024-03-21T08:32:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 -- 172.21.15.3:0/2092818624 learned_addr learned my addr 172.21.15.3:0/2092818624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98be00640 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa982a00640 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 -- 172.21.15.3:0/2092818624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 msgr2=0x5623f39b9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 -- 172.21.15.3:0/2092818624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 msgr2=0x5623f39ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa98b400640 1 -- 172.21.15.3:0/2092818624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623f38d32c0 con 0x5623f39aeb00 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.000+0000 7fa982a00640 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.001+0000 7fa98b400640 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b7390 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0cd110 tx=0x7fa97c0d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48be2a4a56c6706 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.001+0000 7fa980c00640 1 -- 172.21.15.3:0/2092818624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0e3070 con 0x5623f39aeb00 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.001+0000 7fa980c00640 1 -- 172.21.15.3:0/2092818624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa97c0d0ac0 con 0x5623f39aeb00 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.001+0000 7fa980c00640 1 -- 172.21.15.3:0/2092818624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0de8a0 con 0x5623f39aeb00 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.001+0000 7fa988c00640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa9700a3210 0x7fa9700a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa988c00640 1 -- 172.21.15.3:0/293642959 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fa97c0cffe0 con 0x7fa98c068e90 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa98be00640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa9700a3210 0x7fa9700a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 -- 172.21.15.3:0/2092818624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 msgr2=0x5623f39b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b7390 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0cd110 tx=0x7fa97c0d8670 comp rx=0 tx=0).stop 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 -- 172.21.15.3:0/2092818624 shutdown_connections 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b7390 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2092818624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 -- 172.21.15.3:0/2092818624 >> 172.21.15.3:0/2092818624 conn(0x5623f39a8b00 msgr2=0x5623f39a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 -- 172.21.15.3:0/2092818624 shutdown_connections 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa9955e5740 1 -- 172.21.15.3:0/2092818624 wait complete. 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.002+0000 7fa98be00640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa9700a3210 0x7fa9700a56d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa984009c40 tx=0x7fa984005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 -- start start 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 Processor -- start 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623f38cea20 con 0x5623f39aeb00 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623f390e8d0 con 0x5623f39ae1e0 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623f39ba380 con 0x5623f39b78d0 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.003+0000 7fa9955e5740 10 client.0 _ll_register_callbacks cb 0x5623f390b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58246/0 (socket says 172.21.15.3:58246) 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 -- 172.21.15.3:0/225081171 learned_addr learned my addr 172.21.15.3:0/225081171 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa98b400640 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa98be00640 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 -- 172.21.15.3:0/225081171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 msgr2=0x5623f39b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 -- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 msgr2=0x5623f39b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.004+0000 7fa982a00640 1 -- 172.21.15.3:0/225081171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623f3627210 con 0x5623f39aeb00 2024-03-21T08:32:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa982a00640 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b4e10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa974005ab0 tx=0x7fa97402c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debeb45bf96e285 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa979600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa979600640 1 -- 172.21.15.3:0/225081171 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974005d50 con 0x5623f39aeb00 2024-03-21T08:32:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa979600640 1 -- 172.21.15.3:0/225081171 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fa974005230 con 0x5623f39aeb00 2024-03-21T08:32:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa97c0d0ac0 con 0x5623f39aeb00 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa979600640 1 -- 172.21.15.3:0/225081171 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa974005430 con 0x5623f39aeb00 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa9955e5740 10 client.15879 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.005+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5623f36f3620 con 0x5623f39aeb00 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa979600640 1 -- 172.21.15.3:0/225081171 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa97c0d0ac0 con 0x5623f39aeb00 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa979600640 1 -- 172.21.15.3:0/225081171 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa974003680 con 0x5623f39aeb00 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa9955e5740 10 client.15879 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa9955e5740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa9955e5740 20 client.15879 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa9955e5740 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f39c7680 0x5623f39c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa9955e5740 4 client.15879 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623f391fd70 con 0x5623f39c7680 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.006+0000 7fa98b400640 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f39c7680 0x5623f39c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.007+0000 7fa98b400640 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f39c7680 0x5623f39c9ab0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a4defd67c7ef93 server_cookie=898d8f60b34c2c08 in_seq=0 out_seq=0 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.007+0000 7fa979600640 10 client.15879 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.009+0000 7fa979600640 1 -- 172.21.15.3:0/225081171 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5623f391fd70 con 0x5623f39c7680 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.009+0000 7fa979600640 10 client.15879 handle_command_reply: tid=0 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.009+0000 7fa9955e5740 1 client.15879 shutdown 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 msgr2=0x5623f39b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b4e10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa974005ab0 tx=0x7fa97402c9b0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 shutdown_connections 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f39c7680 0x5623f39c9ab0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f39b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39aeb00 0x5623f39b4e10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 --2- 172.21.15.3:0/225081171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39ae1e0 0x5623f39b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa979600640 0 client.15879 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 >> 172.21.15.3:0/225081171 conn(0x5623f39a8b00 msgr2=0x5623f39aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 shutdown_connections 2024-03-21T08:32:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 1 -- 172.21.15.3:0/225081171 wait complete. 2024-03-21T08:32:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.010+0000 7fa9955e5740 20 client.15879 trim_cache size 0 max 16384 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.025+0000 7fa9955e5740 1 Processor -- start 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa9955e5740 1 -- start start 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39ae1e0 0x5623f39b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39b78d0 0x5623f39c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa9955e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1b3150 con 0x5623f39b78d0 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa9955e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1b0c50 con 0x5623f39aeb00 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa9955e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1aecf0 con 0x5623f39ae1e0 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa98be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39b78d0 0x5623f39c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa98be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39b78d0 0x5623f39c2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58258/0 (socket says 172.21.15.3:58258) 2024-03-21T08:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa98be00640 1 -- 172.21.15.3:0/2099992157 learned_addr learned my addr 172.21.15.3:0/2099992157 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa982a00640 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.026+0000 7fa98b400640 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39ae1e0 0x5623f39b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa98be00640 1 -- 172.21.15.3:0/2099992157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39ae1e0 msgr2=0x5623f39b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa98be00640 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39ae1e0 0x5623f39b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa98be00640 1 -- 172.21.15.3:0/2099992157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 msgr2=0x5623f39a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa98be00640 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa98be00640 1 -- 172.21.15.3:0/2099992157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623f39af060 con 0x5623f39b78d0 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa98be00640 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39b78d0 0x5623f39c2a90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa984011570 tx=0x7fa984016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3daddd00cc5a27db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa97be00640 1 -- 172.21.15.3:0/2099992157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984004020 con 0x5623f39b78d0 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa97be00640 1 -- 172.21.15.3:0/2099992157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9840041c0 con 0x5623f39b78d0 2024-03-21T08:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.027+0000 7fa97be00640 1 -- 172.21.15.3:0/2099992157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9840286b0 con 0x5623f39b78d0 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.028+0000 7fa9955e5740 1 -- 172.21.15.3:0/2099992157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39b78d0 msgr2=0x5623f39c2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.028+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39b78d0 0x5623f39c2a90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa984011570 tx=0x7fa984016000 comp rx=0 tx=0).stop 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 -- 172.21.15.3:0/2099992157 shutdown_connections 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39b78d0 0x5623f39c2a90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 --2- 172.21.15.3:0/2099992157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39ae1e0 0x5623f39b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 -- 172.21.15.3:0/2099992157 >> 172.21.15.3:0/2099992157 conn(0x5623f39a8b00 msgr2=0x5623f39a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 -- 172.21.15.3:0/2099992157 shutdown_connections 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 -- 172.21.15.3:0/2099992157 wait complete. 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 -- start start 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.029+0000 7fa9955e5740 1 Processor -- start 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39ae1e0 0x5623f39bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa9955e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f3a21f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa9955e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1af2f0 con 0x5623f39ae1e0 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa9955e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1b05f0 con 0x5623f39aeb00 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa9955e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1af930 con 0x5623f39b78d0 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa9955e5740 10 client.0 _ll_register_callbacks cb 0x7fa984005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39ae1e0 0x5623f39bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39ae1e0 0x5623f39bad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58266/0 (socket says 172.21.15.3:58266) 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 -- 172.21.15.3:0/1805979164 learned_addr learned my addr 172.21.15.3:0/1805979164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa982a00640 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 -- 172.21.15.3:0/1805979164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 msgr2=0x5623f3a21f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f3a21f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 -- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 msgr2=0x5623f39bb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39bb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.030+0000 7fa98b400640 1 -- 172.21.15.3:0/1805979164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623f3a224c0 con 0x5623f39ae1e0 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa98b400640 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39ae1e0 0x5623f39bad50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa97c09bcd0 tx=0x7fa97c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb83902086e47ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa979600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa979600640 1 -- 172.21.15.3:0/1805979164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c097fe0 con 0x5623f39ae1e0 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa979600640 1 -- 172.21.15.3:0/1805979164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fa97c099de0 con 0x5623f39ae1e0 2024-03-21T08:32:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9840041c0 con 0x5623f39ae1e0 2024-03-21T08:32:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa979600640 1 -- 172.21.15.3:0/1805979164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa97c09b6a0 con 0x5623f39ae1e0 2024-03-21T08:32:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa9955e5740 10 client.15885 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.031+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5623f3a23600 con 0x5623f39ae1e0 2024-03-21T08:32:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa979600640 1 -- 172.21.15.3:0/1805979164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9840041c0 con 0x5623f39ae1e0 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa979600640 1 -- 172.21.15.3:0/1805979164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa97c023060 con 0x5623f39ae1e0 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa9955e5740 10 client.15885 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa9955e5740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa9955e5740 20 client.15885 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa9955e5740 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f3a239f0 0x5623f3a25e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa9955e5740 4 client.15885 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623f391a0c0 con 0x5623f3a239f0 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.032+0000 7fa982a00640 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f3a239f0 0x5623f3a25e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.033+0000 7fa982a00640 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f3a239f0 0x5623f3a25e10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889c60dd02c76e8e server_cookie=1dc149718274af7b in_seq=0 out_seq=0 2024-03-21T08:32:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.033+0000 7fa979600640 10 client.15885 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa979600640 1 -- 172.21.15.3:0/1805979164 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623f391a0c0 con 0x5623f3a239f0 2024-03-21T08:32:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa979600640 10 client.15885 handle_command_reply: tid=0 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 client.15885 shutdown 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39ae1e0 msgr2=0x5623f39bad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39ae1e0 0x5623f39bad50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa97c09bcd0 tx=0x7fa97c00be60 comp rx=0 tx=0).stop 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 shutdown_connections 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623f3a239f0 0x5623f3a25e10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623f39b78d0 0x5623f3a21f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623f39aeb00 0x5623f39bb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 --2- 172.21.15.3:0/1805979164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623f39ae1e0 0x5623f39bad50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 >> 172.21.15.3:0/1805979164 conn(0x5623f39a8b00 msgr2=0x5623f39a36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.041+0000 7fa979600640 0 client.15885 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 shutdown_connections 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa9955e5740 1 -- 172.21.15.3:0/1805979164 wait complete. 2024-03-21T08:32:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa9955e5740 20 client.15885 trim_cache size 0 max 16384 2024-03-21T08:32:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:32:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:32:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:32:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa9700a3210 msgr2=0x7fa9700a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa9700a3210 0x7fa9700a56d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa984009c40 tx=0x7fa984005f90 comp rx=0 tx=0).stop 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 msgr2=0x7fa98c199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c199e20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00db80 tx=0x7fa97c00b630 comp rx=0 tx=0).stop 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 shutdown_connections 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa9700a3210 0x7fa9700a56d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa98c1075b0 0x7fa98c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa98c069850 0x7fa98c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 --2- 172.21.15.3:0/293642959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa98c068e90 0x7fa98c199e20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.042+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 >> 172.21.15.3:0/293642959 conn(0x7fa98c075f00 msgr2=0x7fa98c10c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.043+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 shutdown_connections 2024-03-21T08:32:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:20.043+0000 7fa991200640 1 -- 172.21.15.3:0/293642959 wait complete. 2024-03-21T08:32:20.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:20 smithi084 ceph-mon[35477]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-21T08:32:20.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:20 smithi171 ceph-mon[35388]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-21T08:32:20.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:20 smithi003 ceph-mon[31675]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-21T08:32:22.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:22 smithi084 ceph-mon[35477]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T08:32:22.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:22 smithi171 ceph-mon[35388]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T08:32:22.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:22 smithi003 ceph-mon[31675]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T08:32:24.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:24 smithi084 ceph-mon[35477]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-03-21T08:32:24.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:24 smithi171 ceph-mon[35388]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-03-21T08:32:24.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:24 smithi003 ceph-mon[31675]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-03-21T08:32:26.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:26 smithi084 ceph-mon[35477]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:26.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:26 smithi171 ceph-mon[35388]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:26.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:26 smithi003 ceph-mon[31675]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:28.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:28 smithi084 ceph-mon[35477]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:28.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:28 smithi171 ceph-mon[35388]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:28.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:28 smithi003 ceph-mon[31675]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:30.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:30 smithi171 ceph-mon[35388]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:32:30.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:30 smithi003 ceph-mon[31675]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:32:30.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:30 smithi084 ceph-mon[35477]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:32:32.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:32 smithi003 ceph-mon[31675]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:32.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:32 smithi084 ceph-mon[35477]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:32.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:32 smithi171 ceph-mon[35388]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.187+0000 7f1165a00640 1 -- 172.21.15.3:0/835422899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11601098a0 msgr2=0x7f1160109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.187+0000 7f1165a00640 1 --2- 172.21.15.3:0/835422899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11601098a0 0x7f1160109c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1150009930 tx=0x7f115002bc30 comp rx=0 tx=0).stop 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.188+0000 7f1165a00640 1 -- 172.21.15.3:0/835422899 shutdown_connections 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.188+0000 7f1165a00640 1 --2- 172.21.15.3:0/835422899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1160104250 0x7f1160110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.188+0000 7f1165a00640 1 --2- 172.21.15.3:0/835422899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1160103890 0x7f1160103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.188+0000 7f1165a00640 1 --2- 172.21.15.3:0/835422899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11601098a0 0x7f1160109c80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.188+0000 7f1165a00640 1 -- 172.21.15.3:0/835422899 >> 172.21.15.3:0/835422899 conn(0x7f11600ff320 msgr2=0x7f1160101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.188+0000 7f1165a00640 1 -- 172.21.15.3:0/835422899 shutdown_connections 2024-03-21T08:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.188+0000 7f1165a00640 1 -- 172.21.15.3:0/835422899 wait complete. 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 Processor -- start 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 -- start start 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1160103890 0x7f116019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1160104250 0x7f116019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11601098a0 0x7f116019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160113bb0 con 0x7f1160104250 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160113a30 con 0x7f1160103890 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.189+0000 7f1165a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160113d30 con 0x7f11601098a0 2024-03-21T08:32:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1160104250 0x7f116019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1160104250 0x7f116019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57984/0 (socket says 172.21.15.3:57984) 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 -- 172.21.15.3:0/2602643839 learned_addr learned my addr 172.21.15.3:0/2602643839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115ea00640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1160103890 0x7f116019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115f400640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11601098a0 0x7f116019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 -- 172.21.15.3:0/2602643839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11601098a0 msgr2=0x7f116019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11601098a0 0x7f116019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 -- 172.21.15.3:0/2602643839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1160103890 msgr2=0x7f116019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1160103890 0x7f116019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 -- 172.21.15.3:0/2602643839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116019f0e0 con 0x7f1160104250 2024-03-21T08:32:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.190+0000 7f115e000640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1160104250 0x7f116019a630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f114c00d950 tx=0x7f114c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.191+0000 7f1157400640 1 -- 172.21.15.3:0/2602643839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c017070 con 0x7f1160104250 2024-03-21T08:32:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.191+0000 7f1157400640 1 -- 172.21.15.3:0/2602643839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f114c009c30 con 0x7f1160104250 2024-03-21T08:32:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.191+0000 7f1157400640 1 -- 172.21.15.3:0/2602643839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c003c60 con 0x7f1160104250 2024-03-21T08:32:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.191+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116019f3d0 con 0x7f1160104250 2024-03-21T08:32:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.191+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11601a6c60 con 0x7f1160104250 2024-03-21T08:32:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 Processor -- start 2024-03-21T08:32:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 -- start start 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe8246fd0 0x55dbe82473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f1157400640 1 -- 172.21.15.3:0/2602643839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f114c003e00 con 0x7f1160104250 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe8250180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe8252ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe82093b0 con 0x55dbe82478f0 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe81ab830 con 0x55dbe82506c0 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f116880f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe81545d0 con 0x55dbe8246fd0 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.195+0000 7f115ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe8250180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe8250180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57998/0 (socket says 172.21.15.3:57998) 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 -- 172.21.15.3:0/1990061053 learned_addr learned my addr 172.21.15.3:0/1990061053 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115e000640 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe8252ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115f400640 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe8246fd0 0x55dbe82473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 -- 172.21.15.3:0/1990061053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe8246fd0 msgr2=0x55dbe82473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe8246fd0 0x55dbe82473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 -- 172.21.15.3:0/1990061053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 msgr2=0x55dbe8252ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe8252ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 -- 172.21.15.3:0/1990061053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbe816c2c0 con 0x55dbe82478f0 2024-03-21T08:32:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.196+0000 7f115ea00640 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe8250180 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1150005ab0 tx=0x7f115002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e2048c2fa9a0ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.197+0000 7f1157400640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11440a3220 0x7f11440a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.197+0000 7f1157400640 1 -- 172.21.15.3:0/2602643839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f114c0cfb30 con 0x7f1160104250 2024-03-21T08:32:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.197+0000 7f114be00640 1 -- 172.21.15.3:0/1990061053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150045020 con 0x55dbe82478f0 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.197+0000 7f114be00640 1 -- 172.21.15.3:0/1990061053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f115002c7e0 con 0x55dbe82478f0 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.197+0000 7f114be00640 1 -- 172.21.15.3:0/1990061053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150005650 con 0x55dbe82478f0 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.197+0000 7f115f400640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11440a3220 0x7f11440a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 -- 172.21.15.3:0/1990061053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 msgr2=0x55dbe8250180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe8250180 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1150005ab0 tx=0x7f115002c650 comp rx=0 tx=0).stop 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f115f400640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11440a3220 0x7f11440a56e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1158005fd0 tx=0x7f1158009500 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 -- 172.21.15.3:0/1990061053 shutdown_connections 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe8252ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe8250180 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 --2- 172.21.15.3:0/1990061053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe8246fd0 0x55dbe82473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 -- 172.21.15.3:0/1990061053 >> 172.21.15.3:0/1990061053 conn(0x55dbe82418f0 msgr2=0x55dbe8241cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 -- 172.21.15.3:0/1990061053 shutdown_connections 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 -- 172.21.15.3:0/1990061053 wait complete. 2024-03-21T08:32:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.198+0000 7f116880f740 1 -- start start 2024-03-21T08:32:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 1 Processor -- start 2024-03-21T08:32:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe824d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 0x55dbe824dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82506c0 0x55dbe824e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe8167a20 con 0x55dbe82506c0 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe81a78d0 con 0x55dbe8246fd0 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe8253290 con 0x55dbe82478f0 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f116880f740 10 client.0 _ll_register_callbacks cb 0x55dbe81a41c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f115f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82506c0 0x55dbe824e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f115f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82506c0 0x55dbe824e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58000/0 (socket says 172.21.15.3:58000) 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f115f400640 1 -- 172.21.15.3:0/1205549185 learned_addr learned my addr 172.21.15.3:0/1205549185 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.199+0000 7f115ea00640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe824d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115e000640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 0x55dbe824dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115ea00640 1 -- 172.21.15.3:0/1205549185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 msgr2=0x55dbe824dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115ea00640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 0x55dbe824dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115ea00640 1 -- 172.21.15.3:0/1205549185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82506c0 msgr2=0x55dbe824e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115ea00640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82506c0 0x55dbe824e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115ea00640 1 -- 172.21.15.3:0/1205549185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbe7ec0210 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115f400640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82506c0 0x55dbe824e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f115ea00640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe824d740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1150006010 tx=0x7f1150005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9db167b560a3de server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.200+0000 7f1148c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.201+0000 7f1148c00640 1 -- 172.21.15.3:0/1205549185 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500035f0 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.201+0000 7f1148c00640 1 -- 172.21.15.3:0/1205549185 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f1150003790 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.201+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f115002c7e0 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.201+0000 7f1148c00640 1 -- 172.21.15.3:0/1205549185 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1150058050 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.201+0000 7f116880f740 10 client.35651 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.201+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dbe7f8c620 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.201+0000 7f1148c00640 1 -- 172.21.15.3:0/1205549185 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f115002c7e0 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f1148c00640 1 -- 172.21.15.3:0/1205549185 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1150004bf0 con 0x55dbe8246fd0 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f116880f740 10 client.35651 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f116880f740 10 client.35651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f116880f740 20 client.35651 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f116880f740 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe8260540 0x55dbe8262930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f116880f740 4 client.35651 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbe81b8d70 con 0x55dbe8260540 2024-03-21T08:32:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.202+0000 7f115e000640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe8260540 0x55dbe8262930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.203+0000 7f115e000640 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe8260540 0x55dbe8262930 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34f72a37ca49979 server_cookie=6d2983394c281357 in_seq=0 out_seq=0 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.203+0000 7f1148c00640 10 client.35651 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.204+0000 7f1148c00640 1 -- 172.21.15.3:0/1205549185 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dbe81b8d70 con 0x55dbe8260540 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.204+0000 7f1148c00640 10 client.35651 handle_command_reply: tid=0 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.204+0000 7f116880f740 1 client.35651 shutdown 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.204+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 msgr2=0x55dbe824d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.204+0000 7f116880f740 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe824d740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1150006010 tx=0x7f1150005780 comp rx=0 tx=0).stop 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 shutdown_connections 2024-03-21T08:32:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82506c0 0x55dbe824e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe8260540 0x55dbe8262930 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 0x55dbe824dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 --2- 172.21.15.3:0/1205549185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe824d740 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f1148c00640 0 client.35651 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 >> 172.21.15.3:0/1205549185 conn(0x55dbe82418f0 msgr2=0x55dbe8243640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 shutdown_connections 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 1 -- 172.21.15.3:0/1205549185 wait complete. 2024-03-21T08:32:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.205+0000 7f116880f740 20 client.35651 trim_cache size 0 max 16384 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.220+0000 7f116880f740 1 Processor -- start 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.220+0000 7f116880f740 1 -- start start 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.220+0000 7f116880f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe8246fd0 0x55dbe824fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.220+0000 7f116880f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 0x55dbe825b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.220+0000 7f116880f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe825bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f116880f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601b31a0 con 0x55dbe8246fd0 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f116880f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601b0ca0 con 0x55dbe82506c0 2024-03-21T08:32:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f116880f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601aed40 con 0x55dbe82478f0 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe8246fd0 0x55dbe824fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe8246fd0 0x55dbe824fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58012/0 (socket says 172.21.15.3:58012) 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 -- 172.21.15.3:0/1809628795 learned_addr learned my addr 172.21.15.3:0/1809628795 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115f400640 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe825bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 -- 172.21.15.3:0/1809628795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 msgr2=0x55dbe825b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 0x55dbe825b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 -- 172.21.15.3:0/1809628795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 msgr2=0x55dbe825bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe825bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 -- 172.21.15.3:0/1809628795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbe8246e00 con 0x55dbe8246fd0 2024-03-21T08:32:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.221+0000 7f115ea00640 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe8246fd0 0x55dbe824fe70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1150004f60 tx=0x7f115002c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0499ed0218f1a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f114b400640 1 -- 172.21.15.3:0/1809628795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150045040 con 0x55dbe8246fd0 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f114b400640 1 -- 172.21.15.3:0/1809628795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f115002c7e0 con 0x55dbe8246fd0 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f114b400640 1 -- 172.21.15.3:0/1809628795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150003880 con 0x55dbe8246fd0 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f116880f740 1 -- 172.21.15.3:0/1809628795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe8246fd0 msgr2=0x55dbe824fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f116880f740 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe8246fd0 0x55dbe824fe70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1150004f60 tx=0x7f115002c740 comp rx=0 tx=0).stop 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f116880f740 1 -- 172.21.15.3:0/1809628795 shutdown_connections 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f116880f740 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe82506c0 0x55dbe825bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f116880f740 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82478f0 0x55dbe825b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f116880f740 1 --2- 172.21.15.3:0/1809628795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe8246fd0 0x55dbe824fe70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.222+0000 7f116880f740 1 -- 172.21.15.3:0/1809628795 >> 172.21.15.3:0/1809628795 conn(0x55dbe82418f0 msgr2=0x55dbe8240950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.223+0000 7f116880f740 1 -- 172.21.15.3:0/1809628795 shutdown_connections 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.223+0000 7f116880f740 1 -- 172.21.15.3:0/1809628795 wait complete. 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.223+0000 7f116880f740 1 -- start start 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.223+0000 7f116880f740 1 Processor -- start 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f116880f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe8249fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f116880f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe824a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f116880f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82506c0 0x55dbe824aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f116880f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601af340 con 0x55dbe82478f0 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f116880f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601b0640 con 0x55dbe8246fd0 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f116880f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601af980 con 0x55dbe82506c0 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f116880f740 10 client.0 _ll_register_callbacks cb 0x7f114c0d3510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe824a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe8249fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe824a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58014/0 (socket says 172.21.15.3:58014) 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 -- 172.21.15.3:0/3839976137 learned_addr learned my addr 172.21.15.3:0/3839976137 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe8249fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:48188/0 (socket says 172.21.15.3:48188) 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 -- 172.21.15.3:0/3839976137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82506c0 msgr2=0x55dbe824aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82506c0 0x55dbe824aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 -- 172.21.15.3:0/3839976137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 msgr2=0x55dbe8249fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe8249fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.224+0000 7f115e000640 1 -- 172.21.15.3:0/3839976137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbe82bac50 con 0x55dbe82478f0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f115e000640 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe824a510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f114c098280 tx=0x7f114c09b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def1e6af3afbf268 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f1148c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f1148c00640 1 -- 172.21.15.3:0/3839976137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c020040 con 0x55dbe82478f0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f1148c00640 1 -- 172.21.15.3:0/3839976137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f114c097f20 con 0x55dbe82478f0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f115002c7e0 con 0x55dbe82478f0 2024-03-21T08:32:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f1148c00640 1 -- 172.21.15.3:0/3839976137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f114c09bfb0 con 0x55dbe82478f0 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f116880f740 10 client.15903 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dbe82bbfd0 con 0x55dbe82478f0 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.225+0000 7f1148c00640 1 -- 172.21.15.3:0/3839976137 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f115002c7e0 con 0x55dbe82478f0 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f1148c00640 1 -- 172.21.15.3:0/3839976137 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f114c0860b0 con 0x55dbe82478f0 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f116880f740 10 client.15903 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f116880f740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f116880f740 20 client.15903 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f116880f740 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe82bc6f0 0x55dbe82beae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f116880f740 4 client.15903 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dbe81b30c0 con 0x55dbe82bc6f0 2024-03-21T08:32:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.226+0000 7f115ea00640 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe82bc6f0 0x55dbe82beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.227+0000 7f115ea00640 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe82bc6f0 0x55dbe82beae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f1f282290bf808 server_cookie=cebc5505a1f5c8a3 in_seq=0 out_seq=0 2024-03-21T08:32:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.227+0000 7f1148c00640 10 client.15903 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f1148c00640 1 -- 172.21.15.3:0/3839976137 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dbe81b30c0 con 0x55dbe82bc6f0 2024-03-21T08:32:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f1148c00640 10 client.15903 handle_command_reply: tid=0 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 client.15903 shutdown 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 msgr2=0x55dbe824a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe824a510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f114c098280 tx=0x7f114c09b740 comp rx=0 tx=0).stop 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 shutdown_connections 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55dbe82bc6f0 0x55dbe82beae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dbe82506c0 0x55dbe824aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dbe82478f0 0x55dbe824a510 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 --2- 172.21.15.3:0/3839976137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbe8246fd0 0x55dbe8249fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 >> 172.21.15.3:0/3839976137 conn(0x55dbe82418f0 msgr2=0x55dbe824bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.235+0000 7f1148c00640 0 client.15903 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 shutdown_connections 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f116880f740 1 -- 172.21.15.3:0/3839976137 wait complete. 2024-03-21T08:32:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f116880f740 20 client.15903 trim_cache size 0 max 16384 2024-03-21T08:32:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:32:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:32:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:32:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11440a3220 msgr2=0x7f11440a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11440a3220 0x7f11440a56e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1158005fd0 tx=0x7f1158009500 comp rx=0 tx=0).stop 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1160104250 msgr2=0x7f116019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1160104250 0x7f116019a630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f114c00d950 tx=0x7f114c00de10 comp rx=0 tx=0).stop 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 shutdown_connections 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11440a3220 0x7f11440a56e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11601098a0 0x7f116019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1160104250 0x7f116019a630 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 --2- 172.21.15.3:0/2602643839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1160103890 0x7f116019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.236+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 >> 172.21.15.3:0/2602643839 conn(0x7f11600ff320 msgr2=0x7f11600ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.237+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 shutdown_connections 2024-03-21T08:32:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:33.237+0000 7f1165a00640 1 -- 172.21.15.3:0/2602643839 wait complete. 2024-03-21T08:32:34.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:34 smithi003 ceph-mon[31675]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:34.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:34 smithi084 ceph-mon[35477]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:34.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:34 smithi171 ceph-mon[35388]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:36.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:36 smithi003 ceph-mon[31675]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:32:36.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:36 smithi084 ceph-mon[35477]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:32:36.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:36 smithi171 ceph-mon[35388]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:32:38.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:38 smithi003 ceph-mon[31675]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:32:38.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:38 smithi084 ceph-mon[35477]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:32:38.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:38 smithi171 ceph-mon[35388]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:32:40.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:40 smithi003 ceph-mon[31675]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:40.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:40 smithi084 ceph-mon[35477]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:40.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:40 smithi171 ceph-mon[35388]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:42.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:42 smithi003 ceph-mon[31675]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:42.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:42 smithi084 ceph-mon[35477]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:42.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:42 smithi171 ceph-mon[35388]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:44.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:32:44.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.295+0000 7fd7bce00640 1 -- 172.21.15.3:0/1878053301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b810de20 msgr2=0x7fd7b8110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.295+0000 7fd7bce00640 1 --2- 172.21.15.3:0/1878053301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b810de20 0x7fd7b8110210 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009ca0 tx=0x7fd7b002c100 comp rx=0 tx=0).stop 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.296+0000 7fd7bce00640 1 -- 172.21.15.3:0/1878053301 shutdown_connections 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.296+0000 7fd7bce00640 1 --2- 172.21.15.3:0/1878053301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b810de20 0x7fd7b8110210 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.296+0000 7fd7bce00640 1 --2- 172.21.15.3:0/1878053301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7b8100c20 0x7fd7b810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.296+0000 7fd7bce00640 1 --2- 172.21.15.3:0/1878053301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7b8100270 0x7fd7b8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.296+0000 7fd7bce00640 1 -- 172.21.15.3:0/1878053301 >> 172.21.15.3:0/1878053301 conn(0x7fd7b80fbe80 msgr2=0x7fd7b80fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.296+0000 7fd7bce00640 1 -- 172.21.15.3:0/1878053301 shutdown_connections 2024-03-21T08:32:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.296+0000 7fd7bce00640 1 -- 172.21.15.3:0/1878053301 wait complete. 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 Processor -- start 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 -- start start 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b8100270 0x7fd7b8199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7b8100c20 0x7fd7b819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7b810de20 0x7fd7b819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8113940 con 0x7fd7b8100270 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81137c0 con 0x7fd7b8100c20 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7bce00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8113ac0 con 0x7fd7b810de20 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.297+0000 7fd7b7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b8100270 0x7fd7b8199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b8100270 0x7fd7b8199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49398/0 (socket says 172.21.15.3:49398) 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7b8100c20 0x7fd7b819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 -- 172.21.15.3:0/2293338137 learned_addr learned my addr 172.21.15.3:0/2293338137 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7b810de20 0x7fd7b819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 -- 172.21.15.3:0/2293338137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7b810de20 msgr2=0x7fd7b819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7b810de20 0x7fd7b819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 -- 172.21.15.3:0/2293338137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7b8100c20 msgr2=0x7fd7b819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7b8100c20 0x7fd7b819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 -- 172.21.15.3:0/2293338137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b819ee90 con 0x7fd7b8100270 2024-03-21T08:32:44.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.298+0000 7fd7b7400640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b8100270 0x7fd7b8199ea0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800e9f0 tx=0x7fd7a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.299+0000 7fd7afe00640 1 -- 172.21.15.3:0/2293338137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800cd80 con 0x7fd7b8100270 2024-03-21T08:32:44.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.299+0000 7fd7afe00640 1 -- 172.21.15.3:0/2293338137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7a80108e0 con 0x7fd7b8100270 2024-03-21T08:32:44.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.299+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b819f180 con 0x7fd7b8100270 2024-03-21T08:32:44.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.299+0000 7fd7afe00640 1 -- 172.21.15.3:0/2293338137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8018920 con 0x7fd7b8100270 2024-03-21T08:32:44.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.299+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b81a6a10 con 0x7fd7b8100270 2024-03-21T08:32:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.303+0000 7fd7c1172740 1 Processor -- start 2024-03-21T08:32:44.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.303+0000 7fd7c1172740 1 -- start start 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7afe00640 1 -- 172.21.15.3:0/2293338137 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7fd7a8010a90 con 0x7fd7b8100270 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a790fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a790fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a79108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7c1172740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a79061250 con 0x556a790fddd0 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7c1172740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a79009a30 con 0x556a79106460 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7c1172740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7905d2f0 con 0x556a790fd0f0 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a790fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a790fcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49412/0 (socket says 172.21.15.3:49412) 2024-03-21T08:32:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.304+0000 7fd7b7e00640 1 -- 172.21.15.3:0/1560159037 learned_addr learned my addr 172.21.15.3:0/1560159037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7b7400640 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a790fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7b7e00640 1 -- 172.21.15.3:0/1560159037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 msgr2=0x556a790fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a790fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7b7e00640 1 -- 172.21.15.3:0/1560159037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 msgr2=0x556a79108850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a79108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7b7e00640 1 -- 172.21.15.3:0/1560159037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a78ff4690 con 0x556a790fddd0 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a790fcb20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd7b002c640 tx=0x7fd7b0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45176ed7767f9e43 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.305+0000 7fd7afe00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd79c0a31b0 0x7fd79c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.306+0000 7fd7acc00640 1 -- 172.21.15.3:0/1560159037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0043020 con 0x556a790fddd0 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.306+0000 7fd7afe00640 1 -- 172.21.15.3:0/2293338137 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fd7a80d0260 con 0x7fd7b8100270 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.306+0000 7fd7acc00640 1 -- 172.21.15.3:0/1560159037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7b0032970 con 0x556a790fddd0 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.306+0000 7fd7acc00640 1 -- 172.21.15.3:0/1560159037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0039040 con 0x556a790fddd0 2024-03-21T08:32:44.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.306+0000 7fd7b7400640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd79c0a31b0 0x7fd79c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 -- 172.21.15.3:0/1560159037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 msgr2=0x556a790fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7b7400640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd79c0a31b0 0x7fd79c0a5670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80cd470 tx=0x7fd7a8000bc0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a790fcb20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd7b002c640 tx=0x7fd7b0005ef0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 -- 172.21.15.3:0/1560159037 shutdown_connections 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a79108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a790fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 --2- 172.21.15.3:0/1560159037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a790fcb20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 -- 172.21.15.3:0/1560159037 >> 172.21.15.3:0/1560159037 conn(0x556a790c4200 msgr2=0x556a790f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 -- 172.21.15.3:0/1560159037 shutdown_connections 2024-03-21T08:32:44.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 -- 172.21.15.3:0/1560159037 wait complete. 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.307+0000 7fd7c1172740 1 -- start start 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 1 Processor -- start 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fd0f0 0x556a79105be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79102f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a791034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a79109270 con 0x556a790fd0f0 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7901ce80 con 0x556a79106460 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a791093f0 con 0x556a790fddd0 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7c1172740 10 client.0 _ll_register_callbacks cb 0x556a7902f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fd0f0 0x556a79105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fd0f0 0x556a79105be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49428/0 (socket says 172.21.15.3:49428) 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.308+0000 7fd7b7e00640 1 -- 172.21.15.3:0/2530087815 learned_addr learned my addr 172.21.15.3:0/2530087815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a791034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b7e00640 1 -- 172.21.15.3:0/2530087815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 msgr2=0x556a79102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b7e00640 1 -- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 msgr2=0x556a791034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a791034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b7e00640 1 -- 172.21.15.3:0/2530087815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a78e3f7d0 con 0x556a790fd0f0 2024-03-21T08:32:44.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:44.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fd0f0 0x556a79105be0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009600 tx=0x7fd7b0003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489c8116134ce370 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.309+0000 7fd7a1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.310+0000 7fd7a1600640 1 -- 172.21.15.3:0/2530087815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0043050 con 0x556a790fd0f0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.310+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7b0032970 con 0x556a790fd0f0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.310+0000 7fd7a1600640 1 -- 172.21.15.3:0/2530087815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fd7b00326c0 con 0x556a790fd0f0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.310+0000 7fd7a1600640 1 -- 172.21.15.3:0/2530087815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b0057050 con 0x556a790fd0f0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.310+0000 7fd7c1172740 10 client.15915 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.310+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556a78d753f0 con 0x556a790fd0f0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.310+0000 7fd7a1600640 1 -- 172.21.15.3:0/2530087815 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7b0032970 con 0x556a790fd0f0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7a1600640 1 -- 172.21.15.3:0/2530087815 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd7b005a020 con 0x556a790fd0f0 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7c1172740 10 client.15915 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7c1172740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7c1172740 20 client.15915 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7c1172740 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a79116260 0x556a79118650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7c1172740 4 client.15915 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a7906e790 con 0x556a79116260 2024-03-21T08:32:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.311+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a79116260 0x556a79118650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.312+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a79116260 0x556a79118650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c42846d3500bbfb server_cookie=9d7e8c17c8473408 in_seq=0 out_seq=0 2024-03-21T08:32:44.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.312+0000 7fd7a1600640 10 client.15915 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.314+0000 7fd7a1600640 1 -- 172.21.15.3:0/2530087815 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556a7906e790 con 0x556a79116260 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.314+0000 7fd7a1600640 10 client.15915 handle_command_reply: tid=0 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.314+0000 7fd7c1172740 1 client.15915 shutdown 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.314+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fd0f0 msgr2=0x556a79105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.314+0000 7fd7c1172740 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fd0f0 0x556a79105be0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009600 tx=0x7fd7b0003db0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 shutdown_connections 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a791034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a79116260 0x556a79118650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 --2- 172.21.15.3:0/2530087815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fd0f0 0x556a79105be0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7a1600640 0 client.15915 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 >> 172.21.15.3:0/2530087815 conn(0x556a790c4200 msgr2=0x556a790f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 shutdown_connections 2024-03-21T08:32:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 1 -- 172.21.15.3:0/2530087815 wait complete. 2024-03-21T08:32:44.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.315+0000 7fd7c1172740 20 client.15915 trim_cache size 0 max 16384 2024-03-21T08:32:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.335+0000 7fd7c1172740 1 Processor -- start 2024-03-21T08:32:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.335+0000 7fd7c1172740 1 -- start start 2024-03-21T08:32:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.335+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a79111d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.335+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a791056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a79105be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7c1172740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81b3130 con 0x556a790fddd0 2024-03-21T08:32:44.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7c1172740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81b0c30 con 0x556a79106460 2024-03-21T08:32:44.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7c1172740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81aecd0 con 0x556a790fd0f0 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a791056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a791056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49440/0 (socket says 172.21.15.3:49440) 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 -- 172.21.15.3:0/120963706 learned_addr learned my addr 172.21.15.3:0/120963706 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a79105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a79111d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 -- 172.21.15.3:0/120963706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 msgr2=0x556a79111d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a79111d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 -- 172.21.15.3:0/120963706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 msgr2=0x556a79105be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a79105be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b6a00640 1 -- 172.21.15.3:0/120963706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a78d350f0 con 0x556a790fddd0 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.336+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a79111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:44.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.337+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a791056a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x556a791094c0 tx=0x7fd7a4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519c9d71bbd83344 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.337+0000 7fd7a3e00640 1 -- 172.21.15.3:0/120963706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4003ac0 con 0x556a790fddd0 2024-03-21T08:32:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.337+0000 7fd7a3e00640 1 -- 172.21.15.3:0/120963706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7a40054d0 con 0x556a790fddd0 2024-03-21T08:32:44.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.337+0000 7fd7a3e00640 1 -- 172.21.15.3:0/120963706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4004c80 con 0x556a790fddd0 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 -- 172.21.15.3:0/120963706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 msgr2=0x556a791056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a791056a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x556a791094c0 tx=0x7fd7a4004410 comp rx=0 tx=0).stop 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 -- 172.21.15.3:0/120963706 shutdown_connections 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a79106460 0x556a79105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a790fddd0 0x556a791056a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 --2- 172.21.15.3:0/120963706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fd0f0 0x556a79111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 -- 172.21.15.3:0/120963706 >> 172.21.15.3:0/120963706 conn(0x556a790c4200 msgr2=0x556a79111680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 -- 172.21.15.3:0/120963706 shutdown_connections 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 -- 172.21.15.3:0/120963706 wait complete. 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.338+0000 7fd7c1172740 1 -- start start 2024-03-21T08:32:44.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 1 Processor -- start 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a790fd0f0 0x556a79116cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79117200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a79106460 0x556a79117740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81af2d0 con 0x556a79106460 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81b05d0 con 0x556a790fd0f0 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81af910 con 0x556a790fddd0 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7c1172740 10 client.0 _ll_register_callbacks cb 0x7fd7a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.339+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a79106460 0x556a79117740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a79106460 0x556a79117740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49456/0 (socket says 172.21.15.3:49456) 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 -- 172.21.15.3:0/527394879 learned_addr learned my addr 172.21.15.3:0/527394879 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a790fd0f0 0x556a79116cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79117200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 -- 172.21.15.3:0/527394879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 msgr2=0x556a79117200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79117200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 -- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a790fd0f0 msgr2=0x556a79116cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:44.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a790fd0f0 0x556a79116cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.340+0000 7fd7b7e00640 1 -- 172.21.15.3:0/527394879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a791709d0 con 0x556a79106460 2024-03-21T08:32:44.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7b7e00640 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a79106460 0x556a79117740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0003d80 tx=0x7fd7b0003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57ba11af930ce46 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7a1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7a1600640 1 -- 172.21.15.3:0/527394879 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0043040 con 0x556a79106460 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7a1600640 1 -- 172.21.15.3:0/527394879 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fd7b003fd70 con 0x556a79106460 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a40054d0 con 0x556a79106460 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7a1600640 1 -- 172.21.15.3:0/527394879 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b0066050 con 0x556a79106460 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7c1172740 10 client.15921 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.341+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556a79171c50 con 0x556a79106460 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7a1600640 1 -- 172.21.15.3:0/527394879 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7a40054d0 con 0x556a79106460 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7a1600640 1 -- 172.21.15.3:0/527394879 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd7b005a050 con 0x556a79106460 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7c1172740 10 client.15921 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7c1172740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7c1172740 20 client.15921 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7c1172740 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a791722c0 0x556a791746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7c1172740 4 client.15921 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:32:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.342+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556a79068ae0 con 0x556a791722c0 2024-03-21T08:32:44.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.343+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a791722c0 0x556a791746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7b6a00640 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a791722c0 0x556a791746e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152959ad46d80c03 server_cookie=b510a5f8dbc12be5 in_seq=0 out_seq=0 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7a1600640 10 client.15921 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7a1600640 1 -- 172.21.15.3:0/527394879 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556a79068ae0 con 0x556a791722c0 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7a1600640 10 client.15921 handle_command_reply: tid=0 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 client.15921 shutdown 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a79106460 msgr2=0x556a79117740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a79106460 0x556a79117740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0003d80 tx=0x7fd7b0003db0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 shutdown_connections 2024-03-21T08:32:44.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a79106460 0x556a79117740 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556a791722c0 0x556a791746e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a790fddd0 0x556a79117200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 --2- 172.21.15.3:0/527394879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a790fd0f0 0x556a79116cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7a1600640 0 client.15921 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 >> 172.21.15.3:0/527394879 conn(0x556a790c4200 msgr2=0x556a79101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.344+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 shutdown_connections 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7c1172740 1 -- 172.21.15.3:0/527394879 wait complete. 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7c1172740 20 client.15921 trim_cache size 0 max 16384 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-03-21T08:32:44.349 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-03-21T08:32:44.350 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:32:44.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd79c0a31b0 msgr2=0x7fd79c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd79c0a31b0 0x7fd79c0a5670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80cd470 tx=0x7fd7a8000bc0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b8100270 msgr2=0x7fd7b8199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b8100270 0x7fd7b8199ea0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800e9f0 tx=0x7fd7a800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 shutdown_connections 2024-03-21T08:32:44.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd79c0a31b0 0x7fd79c0a5670 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7b810de20 0x7fd7b819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7b8100c20 0x7fd7b819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 --2- 172.21.15.3:0/2293338137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7b8100270 0x7fd7b8199ea0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.345+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 >> 172.21.15.3:0/2293338137 conn(0x7fd7b80fbe80 msgr2=0x7fd7b80fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.346+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 shutdown_connections 2024-03-21T08:32:44.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.346+0000 7fd7bce00640 1 -- 172.21.15.3:0/2293338137 wait complete. 2024-03-21T08:32:44.367 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T08:32:44.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.562+0000 7f389a200640 1 -- 172.21.15.3:0/653284444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894102700 msgr2=0x7f3894102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.562+0000 7f389a200640 1 --2- 172.21.15.3:0/653284444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894102700 0x7f3894102ae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3888009900 tx=0x7f388802bcd0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.562+0000 7f389a200640 1 -- 172.21.15.3:0/653284444 shutdown_connections 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.562+0000 7f389a200640 1 --2- 172.21.15.3:0/653284444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 0x7f389410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.563+0000 7f389a200640 1 --2- 172.21.15.3:0/653284444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3894103020 0x7f389410b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.563+0000 7f389a200640 1 --2- 172.21.15.3:0/653284444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894102700 0x7f3894102ae0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.563+0000 7f389a200640 1 -- 172.21.15.3:0/653284444 >> 172.21.15.3:0/653284444 conn(0x7f38940fbf00 msgr2=0x7f38940fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.563+0000 7f389a200640 1 -- 172.21.15.3:0/653284444 shutdown_connections 2024-03-21T08:32:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.563+0000 7f389a200640 1 -- 172.21.15.3:0/653284444 wait complete. 2024-03-21T08:32:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.564+0000 7f389a200640 1 Processor -- start 2024-03-21T08:32:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.564+0000 7f389a200640 1 -- start start 2024-03-21T08:32:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.564+0000 7f389a200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3894102700 0x7f3894100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.564+0000 7f389a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894103020 0x7f3894100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.564+0000 7f389a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 0x7f3894100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f389a200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894111790 con 0x7f389410bb20 2024-03-21T08:32:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f389a200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894111610 con 0x7f3894103020 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f389a200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894111910 con 0x7f3894102700 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 0x7f3894100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 0x7f3894100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49474/0 (socket says 172.21.15.3:49474) 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 -- 172.21.15.3:0/2132827601 learned_addr learned my addr 172.21.15.3:0/2132827601 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3893e00640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894103020 0x7f3894100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3898e00640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3894102700 0x7f3894100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 -- 172.21.15.3:0/2132827601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3894102700 msgr2=0x7f3894100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3894102700 0x7f3894100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 -- 172.21.15.3:0/2132827601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894103020 msgr2=0x7f3894100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894103020 0x7f3894100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.565+0000 7f3899800640 1 -- 172.21.15.3:0/2132827601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38941a62b0 con 0x7f389410bb20 2024-03-21T08:32:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.566+0000 7f3899800640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 0x7f3894100dd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f388c00efd0 tx=0x7f388c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.566+0000 7f3891600640 1 -- 172.21.15.3:0/2132827601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c003b70 con 0x7f389410bb20 2024-03-21T08:32:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.566+0000 7f3891600640 1 -- 172.21.15.3:0/2132827601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f388c003d10 con 0x7f389410bb20 2024-03-21T08:32:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.566+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38941a65a0 con 0x7f389410bb20 2024-03-21T08:32:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.566+0000 7f3891600640 1 -- 172.21.15.3:0/2132827601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c010980 con 0x7f389410bb20 2024-03-21T08:32:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.566+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941a6aa0 con 0x7f389410bb20 2024-03-21T08:32:44.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.567+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389404f470 con 0x7f389410bb20 2024-03-21T08:32:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.571+0000 7f3891600640 1 -- 172.21.15.3:0/2132827601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f388c004020 con 0x7f389410bb20 2024-03-21T08:32:44.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.573+0000 7f3891600640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f387c0a31a0 0x7f387c0a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:44.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.573+0000 7f3891600640 1 -- 172.21.15.3:0/2132827601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f388c0d0440 con 0x7f389410bb20 2024-03-21T08:32:44.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.573+0000 7f3898e00640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f387c0a31a0 0x7f387c0a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:44.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.574+0000 7f3898e00640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f387c0a31a0 0x7f387c0a5660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f388802c750 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:44.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.578+0000 7f3891600640 1 -- 172.21.15.3:0/2132827601 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f388c09c860 con 0x7f389410bb20 2024-03-21T08:32:44.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:44 smithi003 ceph-mon[31675]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:44.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:44 smithi084 ceph-mon[35477]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:44.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.881+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3894106bc0 con 0x7f389410bb20 2024-03-21T08:32:44.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.882+0000 7f3891600640 1 -- 172.21.15.3:0/2132827601 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f388c09c160 con 0x7f389410bb20 2024-03-21T08:32:44.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:44 smithi171 ceph-mon[35388]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T08:32:44.887 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:32:44.887 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:31:29.351404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34472,34409]},"id":1}]} 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f387c0a31a0 msgr2=0x7f387c0a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f387c0a31a0 0x7f387c0a5660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f388802c750 comp rx=0 tx=0).stop 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 msgr2=0x7f3894100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 0x7f3894100dd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f388c00efd0 tx=0x7f388c00c6a0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 shutdown_connections 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f389410bb20 0x7f3894100dd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3894103020 0x7f3894100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f387c0a31a0 0x7f387c0a5660 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 --2- 172.21.15.3:0/2132827601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3894102700 0x7f3894100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:44.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 >> 172.21.15.3:0/2132827601 conn(0x7f38940fbf00 msgr2=0x7f38940fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:44.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.884+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 shutdown_connections 2024-03-21T08:32:44.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:44.885+0000 7f389a200640 1 -- 172.21.15.3:0/2132827601 wait complete. 2024-03-21T08:32:44.904 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:32:44.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T08:32:45.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 -- 172.21.15.3:0/572738166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 msgr2=0x7faaa810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 --2- 172.21.15.3:0/572738166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa810ded0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009ca0 tx=0x7faaa002c180 comp rx=0 tx=0).stop 2024-03-21T08:32:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 -- 172.21.15.3:0/572738166 shutdown_connections 2024-03-21T08:32:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 --2- 172.21.15.3:0/572738166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa810ded0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 --2- 172.21.15.3:0/572738166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa8102f50 0x7faaa810b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 --2- 172.21.15.3:0/572738166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaa8102630 0x7faaa8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 -- 172.21.15.3:0/572738166 >> 172.21.15.3:0/572738166 conn(0x7faaa8077130 msgr2=0x7faaa80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 -- 172.21.15.3:0/572738166 shutdown_connections 2024-03-21T08:32:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.149+0000 7faaaee00640 1 -- 172.21.15.3:0/572738166 wait complete. 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.150+0000 7faaaee00640 1 Processor -- start 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaaee00640 1 -- start start 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaaee00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa8102630 0x7faaa8199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaaee00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaa8102f50 0x7faaa819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaaee00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaaee00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81116c0 con 0x7faaa810ba50 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaaee00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8111540 con 0x7faaa8102630 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaaee00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8111840 con 0x7faaa8102f50 2024-03-21T08:32:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaae400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaae400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49486/0 (socket says 172.21.15.3:49486) 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.151+0000 7faaae400640 1 -- 172.21.15.3:0/1451260465 learned_addr learned my addr 172.21.15.3:0/1451260465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaada00640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa8102630 0x7faaa8199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaad000640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaa8102f50 0x7faaa819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaae400640 1 -- 172.21.15.3:0/1451260465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaa8102f50 msgr2=0x7faaa819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaae400640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaa8102f50 0x7faaa819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaae400640 1 -- 172.21.15.3:0/1451260465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa8102630 msgr2=0x7faaa8199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaae400640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa8102630 0x7faaa8199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaae400640 1 -- 172.21.15.3:0/1451260465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa819ee10 con 0x7faaa810ba50 2024-03-21T08:32:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.152+0000 7faaae400640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa819e6e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faaa002c790 tx=0x7faaa0035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.153+0000 7faaa6000640 1 -- 172.21.15.3:0/1451260465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0039040 con 0x7faaa810ba50 2024-03-21T08:32:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.153+0000 7faaa6000640 1 -- 172.21.15.3:0/1451260465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faaa0044070 con 0x7faaa810ba50 2024-03-21T08:32:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.153+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa819f0a0 con 0x7faaa810ba50 2024-03-21T08:32:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.153+0000 7faaa6000640 1 -- 172.21.15.3:0/1451260465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0032850 con 0x7faaa810ba50 2024-03-21T08:32:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.153+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa81a6990 con 0x7faaa810ba50 2024-03-21T08:32:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.157+0000 7faab31dc740 1 Processor -- start 2024-03-21T08:32:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.158+0000 7faaa6000640 1 -- 172.21.15.3:0/1451260465 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7faaa00329f0 con 0x7faaa810ba50 2024-03-21T08:32:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.158+0000 7faab31dc740 1 -- start start 2024-03-21T08:32:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faab31dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ffec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faab31dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5007810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faab31dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d500a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faab31dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d4f622c0 con 0x55d0d4fff260 2024-03-21T08:32:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faab31dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d4f0ac90 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faab31dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d4f5e360 con 0x55d0d5007de0 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5007810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5007810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49492/0 (socket says 172.21.15.3:49492) 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 -- 172.21.15.3:0/2025804597 learned_addr learned my addr 172.21.15.3:0/2025804597 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaad000640 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ffec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaad000640 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d500a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 -- 172.21.15.3:0/2025804597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 msgr2=0x55d0d500a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d500a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 -- 172.21.15.3:0/2025804597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4ffe8b0 msgr2=0x55d0d4ffec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ffec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaada00640 1 -- 172.21.15.3:0/2025804597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d4f2f080 con 0x55d0d4fff260 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.159+0000 7faaad000640 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d500a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.160+0000 7faaa6000640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faa900a31b0 0x7faa900a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.160+0000 7faaada00640 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5007810 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009d20 tx=0x7faa9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09f693500ee40dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.160+0000 7faaa6000640 1 -- 172.21.15.3:0/1451260465 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7faaa00f21c0 con 0x7faaa810ba50 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.160+0000 7faaad000640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faa900a31b0 0x7faa900a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.160+0000 7faa9aa00640 1 -- 172.21.15.3:0/2025804597 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c01e070 con 0x55d0d4fff260 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.160+0000 7faa9aa00640 1 -- 172.21.15.3:0/2025804597 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa9c019ae0 con 0x55d0d4fff260 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.160+0000 7faa9aa00640 1 -- 172.21.15.3:0/2025804597 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00eac0 con 0x55d0d4fff260 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.161+0000 7faab31dc740 1 -- 172.21.15.3:0/2025804597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 msgr2=0x55d0d5007810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.161+0000 7faab31dc740 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5007810 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009d20 tx=0x7faa9c011810 comp rx=0 tx=0).stop 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.161+0000 7faaad000640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faa900a31b0 0x7faa900a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faaa810dd80 tx=0x7faa94014040 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 -- 172.21.15.3:0/2025804597 shutdown_connections 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d500a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5007810 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 --2- 172.21.15.3:0/2025804597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ffec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 -- 172.21.15.3:0/2025804597 >> 172.21.15.3:0/2025804597 conn(0x55d0d4ff8f80 msgr2=0x55d0d4ff9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 -- 172.21.15.3:0/2025804597 shutdown_connections 2024-03-21T08:32:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 -- 172.21.15.3:0/2025804597 wait complete. 2024-03-21T08:32:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 -- start start 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.162+0000 7faab31dc740 1 Processor -- start 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faab31dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4ffe8b0 0x55d0d5007710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faab31dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5004aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faab31dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d50050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faab31dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d500ad50 con 0x55d0d4fff260 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faab31dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d4f1e0d0 con 0x55d0d5007de0 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faab31dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d500aed0 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faab31dc740 10 client.0 _ll_register_callbacks cb 0x55d0d4f22e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faaad000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5004aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faaad000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5004aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49506/0 (socket says 172.21.15.3:49506) 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faaad000640 1 -- 172.21.15.3:0/4147064057 learned_addr learned my addr 172.21.15.3:0/4147064057 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.163+0000 7faaae400640 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d50050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faaada00640 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4ffe8b0 0x55d0d5007710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faaad000640 1 -- 172.21.15.3:0/4147064057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4ffe8b0 msgr2=0x55d0d5007710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faaad000640 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4ffe8b0 0x55d0d5007710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faaad000640 1 -- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 msgr2=0x55d0d50050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faaad000640 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d50050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faaad000640 1 -- 172.21.15.3:0/4147064057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0d4c763f0 con 0x55d0d4fff260 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faaad000640 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5004aa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7faa940038a0 tx=0x7faa940095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32fbd96d9d429b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faa8f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.164+0000 7faa8f400640 1 -- 172.21.15.3:0/4147064057 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9402b060 con 0x55d0d4fff260 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.165+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa9c019ae0 con 0x55d0d4fff260 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.165+0000 7faa8f400640 1 -- 172.21.15.3:0/4147064057 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7faa9401c430 con 0x55d0d4fff260 2024-03-21T08:32:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.165+0000 7faa8f400640 1 -- 172.21.15.3:0/4147064057 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa94032050 con 0x55d0d4fff260 2024-03-21T08:32:45.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.165+0000 7faab31dc740 10 client.15939 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:45.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.165+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d0d4d407d0 con 0x55d0d4fff260 2024-03-21T08:32:45.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.165+0000 7faa8f400640 1 -- 172.21.15.3:0/4147064057 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa9c019ae0 con 0x55d0d4fff260 2024-03-21T08:32:45.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.165+0000 7faa8f400640 1 -- 172.21.15.3:0/4147064057 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faa94025030 con 0x55d0d4fff260 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.166+0000 7faab31dc740 10 client.15939 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.166+0000 7faab31dc740 10 client.15939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.166+0000 7faab31dc740 20 client.15939 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.166+0000 7faab31dc740 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5017c00 0x55d0d501a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.166+0000 7faab31dc740 4 client.15939 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.166+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0d4f6f800 con 0x55d0d5017c00 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.166+0000 7faaada00640 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5017c00 0x55d0d501a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.167+0000 7faaada00640 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5017c00 0x55d0d501a010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=545ee2bc71755d21 server_cookie=a61e2f1276d2ca6f in_seq=0 out_seq=0 2024-03-21T08:32:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.167+0000 7faa8f400640 10 client.15939 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.168+0000 7faa8f400640 1 -- 172.21.15.3:0/4147064057 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d0d4f6f800 con 0x55d0d5017c00 2024-03-21T08:32:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.168+0000 7faa8f400640 10 client.15939 handle_command_reply: tid=0 2024-03-21T08:32:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 client.15939 shutdown 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 msgr2=0x55d0d5004aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5004aa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7faa940038a0 tx=0x7faa940095e0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 shutdown_connections 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5017c00 0x55d0d501a010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faa8f400640 0 client.15939 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d50050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4fff260 0x55d0d5004aa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 --2- 172.21.15.3:0/4147064057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4ffe8b0 0x55d0d5007710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 >> 172.21.15.3:0/4147064057 conn(0x55d0d4ff8f80 msgr2=0x55d0d4ffacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 shutdown_connections 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 1 -- 172.21.15.3:0/4147064057 wait complete. 2024-03-21T08:32:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.169+0000 7faab31dc740 20 client.15939 trim_cache size 0 max 16384 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.184+0000 7faab31dc740 1 Processor -- start 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faab31dc740 1 -- start start 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faab31dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ff83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faab31dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4fff260 0x55d0d50071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faab31dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d5007710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faab31dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81b3110 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faab31dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81b0c10 con 0x55d0d4fff260 2024-03-21T08:32:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faab31dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81aecb0 con 0x55d0d5007de0 2024-03-21T08:32:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faaada00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ff83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faaada00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ff83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49522/0 (socket says 172.21.15.3:49522) 2024-03-21T08:32:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.185+0000 7faaada00640 1 -- 172.21.15.3:0/3620895109 learned_addr learned my addr 172.21.15.3:0/3620895109 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaad000640 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4fff260 0x55d0d50071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaae400640 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d5007710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaada00640 1 -- 172.21.15.3:0/3620895109 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 msgr2=0x55d0d5007710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaada00640 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d5007710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaada00640 1 -- 172.21.15.3:0/3620895109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4fff260 msgr2=0x55d0d50071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaada00640 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4fff260 0x55d0d50071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaada00640 1 -- 172.21.15.3:0/3620895109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d500ac30 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaae400640 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d5007710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faaada00640 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ff83a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faa9c01c040 tx=0x7faa9c009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b027a410b46ca026 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faa9a000640 1 -- 172.21.15.3:0/3620895109 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c01e040 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faa9a000640 1 -- 172.21.15.3:0/3620895109 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa9c02a270 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.186+0000 7faa9a000640 1 -- 172.21.15.3:0/3620895109 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00eaf0 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 -- 172.21.15.3:0/3620895109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 msgr2=0x55d0d4ff83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ff83a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faa9c01c040 tx=0x7faa9c009910 comp rx=0 tx=0).stop 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 -- 172.21.15.3:0/3620895109 shutdown_connections 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d5007de0 0x55d0d5007710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d4fff260 0x55d0d50071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 --2- 172.21.15.3:0/3620895109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d4ff83a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 -- 172.21.15.3:0/3620895109 >> 172.21.15.3:0/3620895109 conn(0x55d0d4ff8f80 msgr2=0x55d0d4ffff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 -- 172.21.15.3:0/3620895109 shutdown_connections 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.187+0000 7faab31dc740 1 -- 172.21.15.3:0/3620895109 wait complete. 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 -- start start 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 Processor -- start 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d500fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4fff260 0x55d0d500e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d500e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81af2b0 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81b05b0 con 0x55d0d5007de0 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81af8f0 con 0x55d0d4fff260 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.188+0000 7faab31dc740 10 client.0 _ll_register_callbacks cb 0x7faa940099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d500fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d500fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49528/0 (socket says 172.21.15.3:49528) 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 -- 172.21.15.3:0/3980345854 learned_addr learned my addr 172.21.15.3:0/3980345854 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaae400640 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d500e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaad000640 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4fff260 0x55d0d500e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 -- 172.21.15.3:0/3980345854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4fff260 msgr2=0x55d0d500e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4fff260 0x55d0d500e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 -- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 msgr2=0x55d0d500e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d500e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 -- 172.21.15.3:0/3980345854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0d500edd0 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faaada00640 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d500fe00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7faa9c005bc0 tx=0x7faa9c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbebfda3550a4d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.189+0000 7faa8f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.190+0000 7faa8f400640 1 -- 172.21.15.3:0/3980345854 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c030020 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.190+0000 7faa8f400640 1 -- 172.21.15.3:0/3980345854 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7faa9c0050e0 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.190+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa9c02a270 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.190+0000 7faa8f400640 1 -- 172.21.15.3:0/3980345854 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa9c01e040 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.190+0000 7faab31dc740 10 client.15945 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.190+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d0d5074350 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.190+0000 7faa8f400640 1 -- 172.21.15.3:0/3980345854 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa9c02a270 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faa8f400640 1 -- 172.21.15.3:0/3980345854 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faa9c00e8e0 con 0x55d0d4ffe8b0 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faab31dc740 10 client.15945 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faab31dc740 10 client.15945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faab31dc740 20 client.15945 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faab31dc740 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5074820 0x55d0d5076c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faab31dc740 4 client.15945 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d0d4f69b50 con 0x55d0d5074820 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.191+0000 7faaad000640 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5074820 0x55d0d5076c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faaad000640 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5074820 0x55d0d5076c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b48096d3fa5cdb9f server_cookie=6dd38ada7f35ab8f in_seq=0 out_seq=0 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faa8f400640 10 client.15945 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faa8f400640 1 -- 172.21.15.3:0/3980345854 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d0d4f69b50 con 0x55d0d5074820 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faa8f400640 10 client.15945 handle_command_reply: tid=0 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 client.15945 shutdown 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 msgr2=0x55d0d500fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d500fe00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7faa9c005bc0 tx=0x7faa9c009780 comp rx=0 tx=0).stop 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 shutdown_connections 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x55d0d5074820 0x55d0d5076c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d0d5007de0 0x55d0d500e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0d4fff260 0x55d0d500e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 --2- 172.21.15.3:0/3980345854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d0d4ffe8b0 0x55d0d500fe00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faa8f400640 0 client.15945 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.192+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 >> 172.21.15.3:0/3980345854 conn(0x55d0d4ff8f80 msgr2=0x55d0d501e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.193+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 shutdown_connections 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.193+0000 7faab31dc740 1 -- 172.21.15.3:0/3980345854 wait complete. 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.193+0000 7faab31dc740 20 client.15945 trim_cache size 0 max 16384 2024-03-21T08:32:45.198 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:32:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.193+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faa900a31b0 msgr2=0x7faa900a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.193+0000 7faaaee00640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faa900a31b0 0x7faa900a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faaa810dd80 tx=0x7faa94014040 comp rx=0 tx=0).stop 2024-03-21T08:32:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.193+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 msgr2=0x7faaa819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.193+0000 7faaaee00640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa819e6e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faaa002c790 tx=0x7faaa0035db0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 shutdown_connections 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa810ba50 0x7faaa819e6e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faa900a31b0 0x7faa900a5670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaa8102f50 0x7faaa819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 --2- 172.21.15.3:0/1451260465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaa8102630 0x7faaa8199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 >> 172.21.15.3:0/1451260465 conn(0x7faaa8077130 msgr2=0x7faaa8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 shutdown_connections 2024-03-21T08:32:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.194+0000 7faaaee00640 1 -- 172.21.15.3:0/1451260465 wait complete. 2024-03-21T08:32:45.212 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:32:45.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T08:32:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 -- 172.21.15.3:0/4173114956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 msgr2=0x7f1a08103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 --2- 172.21.15.3:0/4173114956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08103b00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009930 tx=0x7f19f402bc20 comp rx=0 tx=0).stop 2024-03-21T08:32:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 -- 172.21.15.3:0/4173114956 shutdown_connections 2024-03-21T08:32:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 --2- 172.21.15.3:0/4173114956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a08104040 0x7f1a08110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 --2- 172.21.15.3:0/4173114956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08103b00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 --2- 172.21.15.3:0/4173114956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a08109690 0x7f1a08109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 -- 172.21.15.3:0/4173114956 >> 172.21.15.3:0/4173114956 conn(0x7f1a080ff110 msgr2=0x7f1a08101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 -- 172.21.15.3:0/4173114956 shutdown_connections 2024-03-21T08:32:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.408+0000 7f1a0d400640 1 -- 172.21.15.3:0/4173114956 wait complete. 2024-03-21T08:32:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.409+0000 7f1a0d400640 1 Processor -- start 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.409+0000 7f1a0d400640 1 -- start start 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a0d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a0d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a08104040 0x7f1a0819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a0d400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a08109690 0x7f1a0819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a0d400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08113910 con 0x7f1a08103680 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a0d400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08113790 con 0x7f1a08104040 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a0d400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08113a90 con 0x7f1a08109690 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49534/0 (socket says 172.21.15.3:49534) 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a07e00640 1 -- 172.21.15.3:0/3379351925 learned_addr learned my addr 172.21.15.3:0/3379351925 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.410+0000 7f1a07400640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a08104040 0x7f1a0819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a08109690 0x7f1a0819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a07e00640 1 -- 172.21.15.3:0/3379351925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a08109690 msgr2=0x7f1a0819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a08109690 0x7f1a0819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a07e00640 1 -- 172.21.15.3:0/3379351925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a08104040 msgr2=0x7f1a0819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a08104040 0x7f1a0819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a07e00640 1 -- 172.21.15.3:0/3379351925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0819ee90 con 0x7f1a08103680 2024-03-21T08:32:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08199ea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e9f0 tx=0x7f19f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a04c00640 1 -- 172.21.15.3:0/3379351925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800cd80 con 0x7f1a08103680 2024-03-21T08:32:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a04c00640 1 -- 172.21.15.3:0/3379351925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19f80108e0 con 0x7f1a08103680 2024-03-21T08:32:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a04c00640 1 -- 172.21.15.3:0/3379351925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8018920 con 0x7f1a08103680 2024-03-21T08:32:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.411+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0819f180 con 0x7f1a08103680 2024-03-21T08:32:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.412+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a081a6a10 con 0x7f1a08103680 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 Processor -- start 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 -- start start 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a04c00640 1 -- 172.21.15.3:0/3379351925 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f19f8010a90 con 0x7f1a08103680 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f1294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f132020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8f0842c0 con 0x55ae8f129a70 2024-03-21T08:32:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8f02cc90 con 0x55ae8f1325f0 2024-03-21T08:32:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.416+0000 7f1a1182b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8f080360 con 0x55ae8f1290c0 2024-03-21T08:32:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f132020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f132020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49546/0 (socket says 172.21.15.3:49546) 2024-03-21T08:32:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 -- 172.21.15.3:0/3214984710 learned_addr learned my addr 172.21.15.3:0/3214984710 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f1294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 -- 172.21.15.3:0/3214984710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 msgr2=0x55ae8f1294a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f1294a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 -- 172.21.15.3:0/3214984710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 msgr2=0x55ae8f134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.417+0000 7f1a07e00640 1 -- 172.21.15.3:0/3214984710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8f051080 con 0x55ae8f129a70 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.418+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f132020 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f19f80cd360 tx=0x7f19f80d88c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1196c4f1403ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.418+0000 7f1a04c00640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f19ec0a31b0 0x7f19ec0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.418+0000 7f19fd200640 1 -- 172.21.15.3:0/3214984710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80e3070 con 0x55ae8f129a70 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.418+0000 7f19fd200640 1 -- 172.21.15.3:0/3214984710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19f8010de0 con 0x55ae8f129a70 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.418+0000 7f19fd200640 1 -- 172.21.15.3:0/3214984710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f809b8f0 con 0x55ae8f129a70 2024-03-21T08:32:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a04c00640 1 -- 172.21.15.3:0/3379351925 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f19f80d0270 con 0x7f1a08103680 2024-03-21T08:32:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f19ec0a31b0 0x7f19ec0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a1182b740 1 -- 172.21.15.3:0/3214984710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 msgr2=0x55ae8f132020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f132020 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f19f80cd360 tx=0x7f19f80d88c0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a1182b740 1 -- 172.21.15.3:0/3214984710 shutdown_connections 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f132020 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3214984710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f1294a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.419+0000 7f1a1182b740 1 -- 172.21.15.3:0/3214984710 >> 172.21.15.3:0/3214984710 conn(0x55ae8f123790 msgr2=0x55ae8f123b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.420+0000 7f1a1182b740 1 -- 172.21.15.3:0/3214984710 shutdown_connections 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.420+0000 7f1a1182b740 1 -- 172.21.15.3:0/3214984710 wait complete. 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.420+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f19ec0a31b0 0x7f19ec0a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a00005fd0 tx=0x7f1a00009500 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.420+0000 7f1a1182b740 1 -- start start 2024-03-21T08:32:45.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.421+0000 7f1a1182b740 1 Processor -- start 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.421+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1290c0 0x55ae8f131f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.421+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f12f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1325f0 0x55ae8f12f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a1182b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8f135560 con 0x55ae8f129a70 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a1182b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8f0400d0 con 0x55ae8f1290c0 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a1182b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8f1356e0 con 0x55ae8f1325f0 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a1182b740 10 client.0 _ll_register_callbacks cb 0x55ae8f044e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f12f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f12f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49556/0 (socket says 172.21.15.3:49556) 2024-03-21T08:32:45.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 -- 172.21.15.3:0/826436528 learned_addr learned my addr 172.21.15.3:0/826436528 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07e00640 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1290c0 0x55ae8f131f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1325f0 0x55ae8f12f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 -- 172.21.15.3:0/826436528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1325f0 msgr2=0x55ae8f12f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1325f0 0x55ae8f12f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 -- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1290c0 msgr2=0x55ae8f131f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1290c0 0x55ae8f131f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.422+0000 7f1a07400640 1 -- 172.21.15.3:0/826436528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae8ed983f0 con 0x55ae8f129a70 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.423+0000 7f1a07400640 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f12f2b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f19f4036040 tx=0x7f19f402c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc99c5c9b4ad5fa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.423+0000 7f19f1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.424+0000 7f19f1600640 1 -- 172.21.15.3:0/826436528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f402c5c0 con 0x55ae8f129a70 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.424+0000 7f19f1600640 1 -- 172.21.15.3:0/826436528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f19f4005030 con 0x55ae8f129a70 2024-03-21T08:32:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.424+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae8f091800 con 0x55ae8f129a70 2024-03-21T08:32:45.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.424+0000 7f19f1600640 1 -- 172.21.15.3:0/826436528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19f4005230 con 0x55ae8f129a70 2024-03-21T08:32:45.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.424+0000 7f1a1182b740 10 client.15957 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:45.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.424+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ae8ee627d0 con 0x55ae8f129a70 2024-03-21T08:32:45.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f19f1600640 1 -- 172.21.15.3:0/826436528 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ae8f091800 con 0x55ae8f129a70 2024-03-21T08:32:45.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f19f1600640 1 -- 172.21.15.3:0/826436528 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f19f40035f0 con 0x55ae8f129a70 2024-03-21T08:32:45.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f1a1182b740 10 client.15957 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:45.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f1a1182b740 10 client.15957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:32:45.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f1a1182b740 20 client.15957 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:45.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f1a1182b740 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f142470 0x55ae8f144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f1a1182b740 4 client.15957 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:45.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.425+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae8f08bb50 con 0x55ae8f142470 2024-03-21T08:32:45.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.426+0000 7f1a07e00640 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f142470 0x55ae8f144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.426+0000 7f19f1600640 10 client.15957 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:45.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.426+0000 7f1a07e00640 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f142470 0x55ae8f144860 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e86b345c0ad0533 server_cookie=188ffb70c749dc83 in_seq=0 out_seq=0 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.428+0000 7f19f1600640 1 -- 172.21.15.3:0/826436528 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae8f08bb50 con 0x55ae8f142470 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.428+0000 7f19f1600640 10 client.15957 handle_command_reply: tid=0 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.428+0000 7f1a1182b740 1 client.15957 shutdown 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 msgr2=0x55ae8f12f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f12f2b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f19f4036040 tx=0x7f19f402c9e0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 shutdown_connections 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f142470 0x55ae8f144860 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1325f0 0x55ae8f12f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f129a70 0x55ae8f12f2b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 --2- 172.21.15.3:0/826436528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1290c0 0x55ae8f131f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f19f1600640 0 client.15957 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:45.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 >> 172.21.15.3:0/826436528 conn(0x55ae8f123790 msgr2=0x55ae8f124c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 shutdown_connections 2024-03-21T08:32:45.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 1 -- 172.21.15.3:0/826436528 wait complete. 2024-03-21T08:32:45.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.429+0000 7f1a1182b740 20 client.15957 trim_cache size 0 max 16384 2024-03-21T08:32:45.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.444+0000 7f1a1182b740 1 Processor -- start 2024-03-21T08:32:45.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.444+0000 7f1a1182b740 1 -- start start 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1290c0 0x55ae8f125370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f129a70 0x55ae8f119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f1319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a1182b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081aee40 con 0x55ae8f1290c0 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a1182b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081ac940 con 0x55ae8f1325f0 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a1182b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081aa9e0 con 0x55ae8f129a70 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1290c0 0x55ae8f125370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1290c0 0x55ae8f125370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49570/0 (socket says 172.21.15.3:49570) 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 -- 172.21.15.3:0/3225464513 learned_addr learned my addr 172.21.15.3:0/3225464513 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f1319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07400640 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f129a70 0x55ae8f119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 -- 172.21.15.3:0/3225464513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f129a70 msgr2=0x55ae8f119fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f129a70 0x55ae8f119fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 -- 172.21.15.3:0/3225464513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 msgr2=0x55ae8f1319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f1319e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07e00640 1 -- 172.21.15.3:0/3225464513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8f12fc30 con 0x55ae8f1290c0 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f1319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.445+0000 7f1a07400640 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f129a70 0x55ae8f119fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.446+0000 7f1a07e00640 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1290c0 0x55ae8f125370 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f19f809bf20 tx=0x7f19f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3c31c34ae03d56 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.446+0000 7f19f3e00640 1 -- 172.21.15.3:0/3225464513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8026060 con 0x55ae8f1290c0 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.446+0000 7f19f3e00640 1 -- 172.21.15.3:0/3225464513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19f80cd300 con 0x55ae8f1290c0 2024-03-21T08:32:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.446+0000 7f19f3e00640 1 -- 172.21.15.3:0/3225464513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f809c690 con 0x55ae8f1290c0 2024-03-21T08:32:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 -- 172.21.15.3:0/3225464513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1290c0 msgr2=0x55ae8f125370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1290c0 0x55ae8f125370 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f19f809bf20 tx=0x7f19f8000bc0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 -- 172.21.15.3:0/3225464513 shutdown_connections 2024-03-21T08:32:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f1325f0 0x55ae8f1319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f129a70 0x55ae8f119fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 --2- 172.21.15.3:0/3225464513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1290c0 0x55ae8f125370 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 -- 172.21.15.3:0/3225464513 >> 172.21.15.3:0/3225464513 conn(0x55ae8f123790 msgr2=0x55ae8f1198d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 -- 172.21.15.3:0/3225464513 shutdown_connections 2024-03-21T08:32:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 -- 172.21.15.3:0/3225464513 wait complete. 2024-03-21T08:32:45.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.447+0000 7f1a1182b740 1 -- start start 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 1 Processor -- start 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f13a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f129a70 0x55ae8f1389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1325f0 0x55ae8f138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081aafe0 con 0x55ae8f1325f0 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081ac2e0 con 0x55ae8f129a70 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081ab620 con 0x55ae8f1290c0 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.448+0000 7f1a1182b740 10 client.0 _ll_register_callbacks cb 0x7f19f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1325f0 0x55ae8f138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a07400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f129a70 0x55ae8f1389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1325f0 0x55ae8f138ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49584/0 (socket says 172.21.15.3:49584) 2024-03-21T08:32:45.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 -- 172.21.15.3:0/2647046579 learned_addr learned my addr 172.21.15.3:0/2647046579 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a07e00640 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f13a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 -- 172.21.15.3:0/2647046579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 msgr2=0x55ae8f13a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f13a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 -- 172.21.15.3:0/2647046579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f129a70 msgr2=0x55ae8f1389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f129a70 0x55ae8f1389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.449+0000 7f1a0ca00640 1 -- 172.21.15.3:0/2647046579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae8f129810 con 0x55ae8f1325f0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f1a0ca00640 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1325f0 0x55ae8f138ef0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1a000206e0 tx=0x7f1a00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ed47bd964d71f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f19f1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f19f1600640 1 -- 172.21.15.3:0/2647046579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0001c5c0 con 0x55ae8f1325f0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f19f1600640 1 -- 172.21.15.3:0/2647046579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f1a00017030 con 0x55ae8f1325f0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19f80cd300 con 0x55ae8f1325f0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f19f1600640 1 -- 172.21.15.3:0/2647046579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a0001c760 con 0x55ae8f1325f0 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f1a1182b740 10 client.15963 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:45.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.450+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ae8f19ea40 con 0x55ae8f1325f0 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f19f1600640 1 -- 172.21.15.3:0/2647046579 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19f80cd300 con 0x55ae8f1325f0 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f19f1600640 1 -- 172.21.15.3:0/2647046579 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1a0002aa30 con 0x55ae8f1325f0 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f1a1182b740 10 client.15963 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f1a1182b740 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f1a1182b740 20 client.15963 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f1a1182b740 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f19f110 0x55ae8f1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f1a1182b740 4 client.15963 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:32:45.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.451+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae8f009500 con 0x55ae8f19f110 2024-03-21T08:32:45.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.452+0000 7f1a07e00640 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f19f110 0x55ae8f1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.452+0000 7f1a07e00640 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f19f110 0x55ae8f1a1530 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bf21b98b81f5b8 server_cookie=2fae20cde31f99c1 in_seq=0 out_seq=0 2024-03-21T08:32:45.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.452+0000 7f19f1600640 10 client.15963 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:45.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f19f1600640 1 -- 172.21.15.3:0/2647046579 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae8f009500 con 0x55ae8f19f110 2024-03-21T08:32:45.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f19f1600640 10 client.15963 handle_command_reply: tid=0 2024-03-21T08:32:45.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 client.15963 shutdown 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1325f0 msgr2=0x55ae8f138ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1325f0 0x55ae8f138ef0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1a000206e0 tx=0x7f1a00004ab0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 shutdown_connections 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae8f1325f0 0x55ae8f138ef0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae8f129a70 0x55ae8f1389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55ae8f19f110 0x55ae8f1a1530 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 --2- 172.21.15.3:0/2647046579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae8f1290c0 0x55ae8f13a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 >> 172.21.15.3:0/2647046579 conn(0x55ae8f123790 msgr2=0x55ae8f1251f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f19f1600640 0 client.15963 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 shutdown_connections 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.453+0000 7f1a1182b740 1 -- 172.21.15.3:0/2647046579 wait complete. 2024-03-21T08:32:45.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a1182b740 20 client.15963 trim_cache size 0 max 16384 2024-03-21T08:32:45.461 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:32:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f19ec0a31b0 msgr2=0x7f19ec0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f19ec0a31b0 0x7f19ec0a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a00005fd0 tx=0x7f1a00009500 comp rx=0 tx=0).stop 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 msgr2=0x7f1a08199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08199ea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e9f0 tx=0x7f19f800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 shutdown_connections 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f19ec0a31b0 0x7f19ec0a5670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a08109690 0x7f1a0819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a08104040 0x7f1a0819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 --2- 172.21.15.3:0/3379351925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a08103680 0x7f1a08199ea0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 >> 172.21.15.3:0/3379351925 conn(0x7f1a080ff110 msgr2=0x7f1a080ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.454+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 shutdown_connections 2024-03-21T08:32:45.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.455+0000 7f1a0d400640 1 -- 172.21.15.3:0/3379351925 wait complete. 2024-03-21T08:32:45.469 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:45 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2132827601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:32:45.473 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:32:45.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T08:32:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 -- 172.21.15.3:0/3360824049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c074370 msgr2=0x7f920c10d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 --2- 172.21.15.3:0/3360824049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c074370 0x7f920c10d800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009900 tx=0x7f91f402bcc0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 -- 172.21.15.3:0/3360824049 shutdown_connections 2024-03-21T08:32:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 --2- 172.21.15.3:0/3360824049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f920c10dd40 0x7f920c110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 --2- 172.21.15.3:0/3360824049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c074370 0x7f920c10d800 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 --2- 172.21.15.3:0/3360824049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c073a50 0x7f920c073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 -- 172.21.15.3:0/3360824049 >> 172.21.15.3:0/3360824049 conn(0x7f920c0fcfb0 msgr2=0x7f920c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 -- 172.21.15.3:0/3360824049 shutdown_connections 2024-03-21T08:32:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.661+0000 7f9210a00640 1 -- 172.21.15.3:0/3360824049 wait complete. 2024-03-21T08:32:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.662+0000 7f9210a00640 1 Processor -- start 2024-03-21T08:32:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.662+0000 7f9210a00640 1 -- start start 2024-03-21T08:32:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f9210a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c073a50 0x7f920c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f9210a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f920c074370 0x7f920c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f9210a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c10dd40 0x7f920c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f9210a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c113910 con 0x7f920c10dd40 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f9210a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c113790 con 0x7f920c074370 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f9210a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c113a90 con 0x7f920c073a50 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f920be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c10dd40 0x7f920c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f920be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c10dd40 0x7f920c19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49600/0 (socket says 172.21.15.3:49600) 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f920be00640 1 -- 172.21.15.3:0/1812371133 learned_addr learned my addr 172.21.15.3:0/1812371133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f9203e00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f920c074370 0x7f920c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.663+0000 7f920b400640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c073a50 0x7f920c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f920be00640 1 -- 172.21.15.3:0/1812371133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c073a50 msgr2=0x7f920c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f920be00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c073a50 0x7f920c199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f920be00640 1 -- 172.21.15.3:0/1812371133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f920c074370 msgr2=0x7f920c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f920be00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f920c074370 0x7f920c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f920be00640 1 -- 172.21.15.3:0/1812371133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c19ef20 con 0x7f920c10dd40 2024-03-21T08:32:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f920be00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c10dd40 0x7f920c19e7f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fd0 tx=0x7f920400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f9208c00640 1 -- 172.21.15.3:0/1812371133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204017070 con 0x7f920c10dd40 2024-03-21T08:32:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f9208c00640 1 -- 172.21.15.3:0/1812371133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9204003e20 con 0x7f920c10dd40 2024-03-21T08:32:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c19f210 con 0x7f920c10dd40 2024-03-21T08:32:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.664+0000 7f9208c00640 1 -- 172.21.15.3:0/1812371133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204010460 con 0x7f920c10dd40 2024-03-21T08:32:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.665+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c1a6aa0 con 0x7f920c10dd40 2024-03-21T08:32:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.668+0000 7f9208c00640 1 -- 172.21.15.3:0/1812371133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f9204003970 con 0x7f920c10dd40 2024-03-21T08:32:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.668+0000 7f9214c1a740 1 Processor -- start 2024-03-21T08:32:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9214c1a740 1 -- start start 2024-03-21T08:32:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e68b0 0x55a7451e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e7260 0x55a7451ef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9214c1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74514a2c0 con 0x55a7451e68b0 2024-03-21T08:32:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9214c1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7450f2c90 con 0x55a7451e7260 2024-03-21T08:32:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9214c1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a745146360 con 0x55a7451efde0 2024-03-21T08:32:45.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e68b0 0x55a7451e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e68b0 0x55a7451e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49608/0 (socket says 172.21.15.3:49608) 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f9203e00640 1 -- 172.21.15.3:0/3924226247 learned_addr learned my addr 172.21.15.3:0/3924226247 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.669+0000 7f920b400640 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e7260 0x55a7451ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f9203e00640 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f9208c00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f91f00a31b0 0x7f91f00a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f920b400640 1 -- 172.21.15.3:0/3924226247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 msgr2=0x55a7451f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f920b400640 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f920b400640 1 -- 172.21.15.3:0/3924226247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e68b0 msgr2=0x55a7451e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f920b400640 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e68b0 0x55a7451e6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f920b400640 1 -- 172.21.15.3:0/3924226247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a745117080 con 0x55a7451e7260 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f9203e00640 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e68b0 0x55a7451e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f9208c00640 1 -- 172.21.15.3:0/1812371133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f92040cfca0 con 0x7f920c10dd40 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f920be00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f91f00a31b0 0x7f91f00a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.670+0000 7f920b400640 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e7260 0x55a7451ef810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00bbe0 tx=0x7f91fc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9005f05dabb9b2cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.671+0000 7f91fbe00640 1 -- 172.21.15.3:0/3924226247 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc015070 con 0x55a7451e7260 2024-03-21T08:32:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.671+0000 7f91fbe00640 1 -- 172.21.15.3:0/3924226247 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91fc0054d0 con 0x55a7451e7260 2024-03-21T08:32:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.671+0000 7f91fbe00640 1 -- 172.21.15.3:0/3924226247 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc011440 con 0x55a7451e7260 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 -- 172.21.15.3:0/3924226247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e7260 msgr2=0x55a7451ef810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e7260 0x55a7451ef810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00bbe0 tx=0x7f91fc00f730 comp rx=0 tx=0).stop 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f920be00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f91f00a31b0 0x7f91f00a5670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92040cceb0 tx=0x7f920400bea0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 -- 172.21.15.3:0/3924226247 shutdown_connections 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e7260 0x55a7451ef810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3924226247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e68b0 0x55a7451e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 -- 172.21.15.3:0/3924226247 >> 172.21.15.3:0/3924226247 conn(0x55a7451e0f80 msgr2=0x55a7451e1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 -- 172.21.15.3:0/3924226247 shutdown_connections 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 -- 172.21.15.3:0/3924226247 wait complete. 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.672+0000 7f9214c1a740 1 -- start start 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 1 Processor -- start 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7451f2d50 con 0x55a7451e7260 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7451060d0 con 0x55a7451e68b0 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7451f2ed0 con 0x55a7451efde0 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.673+0000 7f9214c1a740 10 client.0 _ll_register_callbacks cb 0x55a74510ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f9203e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ecad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49610/0 (socket says 172.21.15.3:49610) 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 -- 172.21.15.3:0/3523925941 learned_addr learned my addr 172.21.15.3:0/3523925941 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f9203e00640 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 -- 172.21.15.3:0/3523925941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 msgr2=0x55a7451ed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451ed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 -- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 msgr2=0x55a7451ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 -- 172.21.15.3:0/3523925941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a744e5e3f0 con 0x55a7451e7260 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f9203e00640 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.674+0000 7f920b400640 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ecad0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f91fc005fa0 tx=0x7f91fc00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e29c9b71a63a3f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.675+0000 7f91f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.675+0000 7f91f8c00640 1 -- 172.21.15.3:0/3523925941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc028060 con 0x55a7451e7260 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.675+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91fc0054d0 con 0x55a7451e7260 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.675+0000 7f91f8c00640 1 -- 172.21.15.3:0/3523925941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f91fc004d00 con 0x55a7451e7260 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.675+0000 7f91f8c00640 1 -- 172.21.15.3:0/3523925941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91fc02f050 con 0x55a7451e7260 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.675+0000 7f9214c1a740 10 client.15975 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.675+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a744f287d0 con 0x55a7451e7260 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f91f8c00640 1 -- 172.21.15.3:0/3523925941 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91fc0054d0 con 0x55a7451e7260 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f91f8c00640 1 -- 172.21.15.3:0/3523925941 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f91fc0118c0 con 0x55a7451e7260 2024-03-21T08:32:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f9214c1a740 10 client.15975 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f9214c1a740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f9214c1a740 20 client.15975 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7451ffbc0 0x55a745201fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f9214c1a740 4 client.15975 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.676+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a745157800 con 0x55a7451ffbc0 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.677+0000 7f9203e00640 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7451ffbc0 0x55a745201fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.677+0000 7f9203e00640 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7451ffbc0 0x55a745201fb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f1a03f06d94ade server_cookie=bb4dfe3d0da781f3 in_seq=0 out_seq=0 2024-03-21T08:32:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.677+0000 7f91f8c00640 10 client.15975 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:45.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f91f8c00640 1 -- 172.21.15.3:0/3523925941 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a745157800 con 0x55a7451ffbc0 2024-03-21T08:32:45.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f91f8c00640 10 client.15975 handle_command_reply: tid=0 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 client.15975 shutdown 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 msgr2=0x55a7451ecad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ecad0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f91fc005fa0 tx=0x7f91fc00b920 comp rx=0 tx=0).stop 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 shutdown_connections 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7451ffbc0 0x55a745201fb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ecad0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 --2- 172.21.15.3:0/3523925941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f91f8c00640 0 client.15975 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.679+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 >> 172.21.15.3:0/3523925941 conn(0x55a7451e0f80 msgr2=0x55a7451e2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.680+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 shutdown_connections 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.680+0000 7f9214c1a740 1 -- 172.21.15.3:0/3523925941 wait complete. 2024-03-21T08:32:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.680+0000 7f9214c1a740 20 client.15975 trim_cache size 0 max 16384 2024-03-21T08:32:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.696+0000 7f9214c1a740 1 Processor -- start 2024-03-21T08:32:45.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.696+0000 7f9214c1a740 1 -- start start 2024-03-21T08:32:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9214c1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c1b3180 con 0x55a7451e7260 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9214c1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c1b0c80 con 0x55a7451e68b0 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9214c1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c1aed20 con 0x55a7451efde0 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f920b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f920b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49626/0 (socket says 172.21.15.3:49626) 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f920b400640 1 -- 172.21.15.3:0/504756156 learned_addr learned my addr 172.21.15.3:0/504756156 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9203e00640 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.697+0000 7f9203e00640 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f920b400640 1 -- 172.21.15.3:0/504756156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 msgr2=0x55a7451e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f920b400640 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f920b400640 1 -- 172.21.15.3:0/504756156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 msgr2=0x55a7451ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f920b400640 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f920b400640 1 -- 172.21.15.3:0/504756156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7451e5bf0 con 0x55a7451e7260 2024-03-21T08:32:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f920b400640 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ef740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0059f0 tx=0x7f91fc004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc7dc8aeda39e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f91fb400640 1 -- 172.21.15.3:0/504756156 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc028070 con 0x55a7451e7260 2024-03-21T08:32:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f91fb400640 1 -- 172.21.15.3:0/504756156 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91fc005a70 con 0x55a7451e7260 2024-03-21T08:32:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.698+0000 7f91fb400640 1 -- 172.21.15.3:0/504756156 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc015070 con 0x55a7451e7260 2024-03-21T08:32:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 -- 172.21.15.3:0/504756156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 msgr2=0x55a7451ef740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ef740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0059f0 tx=0x7f91fc004850 comp rx=0 tx=0).stop 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 -- 172.21.15.3:0/504756156 shutdown_connections 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a7451e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a7451ef740 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 --2- 172.21.15.3:0/504756156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a7451ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 -- 172.21.15.3:0/504756156 >> 172.21.15.3:0/504756156 conn(0x55a7451e0f80 msgr2=0x55a7451e7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 -- 172.21.15.3:0/504756156 shutdown_connections 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.699+0000 7f9214c1a740 1 -- 172.21.15.3:0/504756156 wait complete. 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 -- start start 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 Processor -- start 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a745202550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a745202a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a745202fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c1af320 con 0x55a7451e7260 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c1b0620 con 0x55a7451e68b0 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c1af960 con 0x55a7451efde0 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f9214c1a740 10 client.0 _ll_register_callbacks cb 0x7f91f4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.700+0000 7f920b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a745202a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a745202a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49632/0 (socket says 172.21.15.3:49632) 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 -- 172.21.15.3:0/276383587 learned_addr learned my addr 172.21.15.3:0/276383587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f9203e00640 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a745202550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 -- 172.21.15.3:0/276383587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 msgr2=0x55a745202fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a745202fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 -- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 msgr2=0x55a745202550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a745202550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 -- 172.21.15.3:0/276383587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7452629d0 con 0x55a7451e7260 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f920b400640 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a745202a90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f91fc023000 tx=0x7f91fc00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daed00ec0ffe7945 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.701+0000 7f91f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.702+0000 7f91f8c00640 1 -- 172.21.15.3:0/276383587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc015070 con 0x55a7451e7260 2024-03-21T08:32:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.702+0000 7f91f8c00640 1 -- 172.21.15.3:0/276383587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f91fc011650 con 0x55a7451e7260 2024-03-21T08:32:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.702+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91fc005a70 con 0x55a7451e7260 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.702+0000 7f91f8c00640 1 -- 172.21.15.3:0/276383587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91fc044050 con 0x55a7451e7260 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.702+0000 7f9214c1a740 10 client.15981 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.702+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a745263cb0 con 0x55a7451e7260 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.702+0000 7f91f8c00640 1 -- 172.21.15.3:0/276383587 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f91fc005a70 con 0x55a7451e7260 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f91f8c00640 1 -- 172.21.15.3:0/276383587 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f91fc039020 con 0x55a7451e7260 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f9214c1a740 10 client.15981 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f9214c1a740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f9214c1a740 20 client.15981 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f9214c1a740 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7452642c0 0x55a7452666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:45.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f9214c1a740 4 client.15981 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a745151b50 con 0x55a7452642c0 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.703+0000 7f9203e00640 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7452642c0 0x55a7452666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.704+0000 7f9203e00640 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7452642c0 0x55a7452666e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5805683c376f07 server_cookie=d9ee7634056e3710 in_seq=0 out_seq=0 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.704+0000 7f91f8c00640 10 client.15981 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f91f8c00640 1 -- 172.21.15.3:0/276383587 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a745151b50 con 0x55a7452642c0 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f91f8c00640 10 client.15981 handle_command_reply: tid=0 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 client.15981 shutdown 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 msgr2=0x55a745202a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a745202a90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f91fc023000 tx=0x7f91fc00bee0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 shutdown_connections 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55a7452642c0 0x55a7452666e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7451efde0 0x55a745202fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a7451e7260 0x55a745202a90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 --2- 172.21.15.3:0/276383587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7451e68b0 0x55a745202550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 >> 172.21.15.3:0/276383587 conn(0x55a7451e0f80 msgr2=0x55a7451e7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f91f8c00640 0 client.15981 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:45.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 shutdown_connections 2024-03-21T08:32:45.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.705+0000 7f9214c1a740 1 -- 172.21.15.3:0/276383587 wait complete. 2024-03-21T08:32:45.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9214c1a740 20 client.15981 trim_cache size 0 max 16384 2024-03-21T08:32:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f91f00a31b0 msgr2=0x7f91f00a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f91f00a31b0 0x7f91f00a5670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92040cceb0 tx=0x7f920400bea0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c10dd40 msgr2=0x7f920c19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c10dd40 0x7f920c19e7f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fd0 tx=0x7f920400b260 comp rx=0 tx=0).stop 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 shutdown_connections 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f920c10dd40 0x7f920c19e7f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f920c074370 0x7f920c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f91f00a31b0 0x7f91f00a5670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 --2- 172.21.15.3:0/1812371133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c073a50 0x7f920c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.706+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 >> 172.21.15.3:0/1812371133 conn(0x7f920c0fcfb0 msgr2=0x7f920c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.707+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 shutdown_connections 2024-03-21T08:32:45.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:45.707+0000 7f9210a00640 1 -- 172.21.15.3:0/1812371133 wait complete. 2024-03-21T08:32:45.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:45 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2132827601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:32:45.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:45 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2132827601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:32:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 -- 172.21.15.3:0/3926160032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c068e90 msgr2=0x7f6f9c069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3926160032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c068e90 0x7f6f9c069310 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009900 tx=0x7f6f8802bc40 comp rx=0 tx=0).stop 2024-03-21T08:32:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 -- 172.21.15.3:0/3926160032 shutdown_connections 2024-03-21T08:32:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3926160032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f9c069850 0x7f6f9c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3926160032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c068e90 0x7f6f9c069310 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3926160032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f9c1075b0 0x7f6f9c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 -- 172.21.15.3:0/3926160032 >> 172.21.15.3:0/3926160032 conn(0x7f6f9c075f00 msgr2=0x7f6f9c076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 -- 172.21.15.3:0/3926160032 shutdown_connections 2024-03-21T08:32:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.375+0000 7f6fa2200640 1 -- 172.21.15.3:0/3926160032 wait complete. 2024-03-21T08:32:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 Processor -- start 2024-03-21T08:32:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 -- start start 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f9c068e90 0x7f6f9c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f9c069850 0x7f6f9c100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c1075b0 0x7f6f9c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c111700 con 0x7f6f9c1075b0 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c111580 con 0x7f6f9c069850 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.377+0000 7f6fa2200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c111880 con 0x7f6f9c068e90 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c1075b0 0x7f6f9c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c1075b0 0x7f6f9c101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49646/0 (socket says 172.21.15.3:49646) 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 -- 172.21.15.3:0/3510152516 learned_addr learned my addr 172.21.15.3:0/3510152516 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f9c068e90 0x7f6f9c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6f9be00640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f9c069850 0x7f6f9c100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 -- 172.21.15.3:0/3510152516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f9c068e90 msgr2=0x7f6f9c1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f9c068e90 0x7f6f9c1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 -- 172.21.15.3:0/3510152516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f9c069850 msgr2=0x7f6f9c100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f9c069850 0x7f6f9c100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.378+0000 7f6fa1800640 1 -- 172.21.15.3:0/3510152516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c1a6140 con 0x7f6f9c1075b0 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.379+0000 7f6fa1800640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c1075b0 0x7f6f9c101170 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400eaa0 tx=0x7f6f9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.379+0000 7f6f99600640 1 -- 172.21.15.3:0/3510152516 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400cdb0 con 0x7f6f9c1075b0 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.379+0000 7f6f99600640 1 -- 172.21.15.3:0/3510152516 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f94005020 con 0x7f6f9c1075b0 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.379+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c1a6430 con 0x7f6f9c1075b0 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.379+0000 7f6f99600640 1 -- 172.21.15.3:0/3510152516 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940116d0 con 0x7f6f9c1075b0 2024-03-21T08:32:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.379+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c1a6980 con 0x7f6f9c1075b0 2024-03-21T08:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.383+0000 7f6fa6616740 1 Processor -- start 2024-03-21T08:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6f99600640 1 -- 172.21.15.3:0/3510152516 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f6f940051d0 con 0x7f6f9c1075b0 2024-03-21T08:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6fa6616740 1 -- start start 2024-03-21T08:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadd25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aaddb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6fa6616740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558aad943b0 con 0x5558aadd2b00 2024-03-21T08:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6fa6616740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558aad36830 con 0x5558aaddb8d0 2024-03-21T08:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.384+0000 7f6fa6616740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558aacdf5d0 con 0x5558aadd21e0 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aaddb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aaddb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49656/0 (socket says 172.21.15.3:49656) 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 -- 172.21.15.3:0/3081861402 learned_addr learned my addr 172.21.15.3:0/3081861402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6f9be00640 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadd25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 -- 172.21.15.3:0/3081861402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 msgr2=0x5558aadd25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadd25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6f9be00640 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 -- 172.21.15.3:0/3081861402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 msgr2=0x5558aadddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 -- 172.21.15.3:0/3081861402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558aacf72c0 con 0x5558aadd2b00 2024-03-21T08:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6f9be00640 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6f99600640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f6f840a3190 0x7f6f840a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.385+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aaddb390 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000bc60 tx=0x7f6f9000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae7792ab9eb6b46 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.386+0000 7f6f99600640 1 -- 172.21.15.3:0/3510152516 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f6f940d0130 con 0x7f6f9c1075b0 2024-03-21T08:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.386+0000 7f6f9be00640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f6f840a3190 0x7f6f840a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.386+0000 7f6f8e000640 1 -- 172.21.15.3:0/3081861402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90015070 con 0x5558aadd2b00 2024-03-21T08:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.386+0000 7f6f8e000640 1 -- 172.21.15.3:0/3081861402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f900118f0 con 0x5558aadd2b00 2024-03-21T08:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.386+0000 7f6f8e000640 1 -- 172.21.15.3:0/3081861402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90010710 con 0x5558aadd2b00 2024-03-21T08:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 -- 172.21.15.3:0/3081861402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 msgr2=0x5558aaddb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aaddb390 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000bc60 tx=0x7f6f9000f730 comp rx=0 tx=0).stop 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6f9be00640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f6f840a3190 0x7f6f840a5650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c102180 tx=0x7f6f88000bc0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 -- 172.21.15.3:0/3081861402 shutdown_connections 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aaddb390 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 --2- 172.21.15.3:0/3081861402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadd25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 -- 172.21.15.3:0/3081861402 >> 172.21.15.3:0/3081861402 conn(0x5558aadccb00 msgr2=0x5558aadccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 -- 172.21.15.3:0/3081861402 shutdown_connections 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 -- 172.21.15.3:0/3081861402 wait complete. 2024-03-21T08:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 -- start start 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.387+0000 7f6fa6616740 1 Processor -- start 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd21e0 0x5558aadd89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadd8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aaddb8d0 0x5558aadd9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa6616740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558aacf2a20 con 0x5558aadd21e0 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa6616740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558aad328d0 con 0x5558aadd2b00 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa6616740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558aadde500 con 0x5558aaddb8d0 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa6616740 10 client.0 _ll_register_callbacks cb 0x5558aad2f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd21e0 0x5558aadd89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd21e0 0x5558aadd89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49672/0 (socket says 172.21.15.3:49672) 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6fa0e00640 1 -- 172.21.15.3:0/659252306 learned_addr learned my addr 172.21.15.3:0/659252306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.388+0000 7f6f9be00640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadd8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa1800640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aaddb8d0 0x5558aadd9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa0e00640 1 -- 172.21.15.3:0/659252306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aaddb8d0 msgr2=0x5558aadd9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aaddb8d0 0x5558aadd9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa0e00640 1 -- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 msgr2=0x5558aadd8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadd8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa0e00640 1 -- 172.21.15.3:0/659252306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558aaa4b210 con 0x5558aadd21e0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa1800640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aaddb8d0 0x5558aadd9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd21e0 0x5558aadd89b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005f70 tx=0x7f6f90011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e3fe477f066f46 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6f82a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6f82a00640 1 -- 172.21.15.3:0/659252306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90015070 con 0x5558aadd21e0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6f82a00640 1 -- 172.21.15.3:0/659252306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f6f90011cb0 con 0x5558aadd21e0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.389+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f900118f0 con 0x5558aadd21e0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6f82a00640 1 -- 172.21.15.3:0/659252306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f9000be90 con 0x5558aadd21e0 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 10 client.15993 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5558aab17620 con 0x5558aadd21e0 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6f82a00640 1 -- 172.21.15.3:0/659252306 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f900118f0 con 0x5558aadd21e0 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6f82a00640 1 -- 172.21.15.3:0/659252306 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6f900046c0 con 0x5558aadd21e0 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 10 client.15993 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 20 client.15993 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aadeb840 0x5558aadedc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 4 client.15993 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.390+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558aad43d70 con 0x5558aadeb840 2024-03-21T08:32:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.391+0000 7f6f9be00640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aadeb840 0x5558aadedc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.392+0000 7f6f9be00640 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aadeb840 0x5558aadedc30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e0a1e185c7f76f server_cookie=f9fb142c3dea6ceb in_seq=0 out_seq=0 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.392+0000 7f6f82a00640 10 client.15993 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.393+0000 7f6f82a00640 1 -- 172.21.15.3:0/659252306 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5558aad43d70 con 0x5558aadeb840 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.393+0000 7f6f82a00640 10 client.15993 handle_command_reply: tid=0 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.393+0000 7f6fa6616740 1 client.15993 shutdown 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd21e0 msgr2=0x5558aadd89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd21e0 0x5558aadd89b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005f70 tx=0x7f6f90011670 comp rx=0 tx=0).stop 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 shutdown_connections 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aadeb840 0x5558aadedc30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aaddb8d0 0x5558aadd9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadd8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6f82a00640 0 client.15993 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 --2- 172.21.15.3:0/659252306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd21e0 0x5558aadd89b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 >> 172.21.15.3:0/659252306 conn(0x5558aadccb00 msgr2=0x5558aadce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 shutdown_connections 2024-03-21T08:32:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 1 -- 172.21.15.3:0/659252306 wait complete. 2024-03-21T08:32:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.394+0000 7f6fa6616740 20 client.15993 trim_cache size 0 max 16384 2024-03-21T08:32:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 Processor -- start 2024-03-21T08:32:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 -- start start 2024-03-21T08:32:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aaddb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadc76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aaddb8d0 0x5558aadd5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1b31d0 con 0x5558aaddb8d0 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1b0cd0 con 0x5558aadd2b00 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.410+0000 7f6fa6616740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1aed70 con 0x5558aadd21e0 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aaddb8d0 0x5558aadd5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aaddb8d0 0x5558aadd5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49682/0 (socket says 172.21.15.3:49682) 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 -- 172.21.15.3:0/4143612656 learned_addr learned my addr 172.21.15.3:0/4143612656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aaddb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6f9be00640 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadc76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 -- 172.21.15.3:0/4143612656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 msgr2=0x5558aaddb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aaddb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 -- 172.21.15.3:0/4143612656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 msgr2=0x5558aadc76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadc76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 -- 172.21.15.3:0/4143612656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558aadd6f00 con 0x5558aaddb8d0 2024-03-21T08:32:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.411+0000 7f6fa1800640 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aaddb8d0 0x5558aadd5c30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6f94098310 tx=0x7f6f9409c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5141ef66ba920113 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6f8d600640 1 -- 172.21.15.3:0/4143612656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9401f070 con 0x5558aaddb8d0 2024-03-21T08:32:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6f8d600640 1 -- 172.21.15.3:0/4143612656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f940bc0c0 con 0x5558aaddb8d0 2024-03-21T08:32:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6f8d600640 1 -- 172.21.15.3:0/4143612656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940cdbb0 con 0x5558aaddb8d0 2024-03-21T08:32:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6fa6616740 1 -- 172.21.15.3:0/4143612656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aaddb8d0 msgr2=0x5558aadd5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6fa6616740 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aaddb8d0 0x5558aadd5c30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6f94098310 tx=0x7f6f9409c670 comp rx=0 tx=0).stop 2024-03-21T08:32:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6fa6616740 1 -- 172.21.15.3:0/4143612656 shutdown_connections 2024-03-21T08:32:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6fa6616740 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aaddb8d0 0x5558aadd5c30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6fa6616740 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aadd2b00 0x5558aadc76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6fa6616740 1 --2- 172.21.15.3:0/4143612656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aaddb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.412+0000 7f6fa6616740 1 -- 172.21.15.3:0/4143612656 >> 172.21.15.3:0/4143612656 conn(0x5558aadccb00 msgr2=0x5558aadccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.413+0000 7f6fa6616740 1 -- 172.21.15.3:0/4143612656 shutdown_connections 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.413+0000 7f6fa6616740 1 -- 172.21.15.3:0/4143612656 wait complete. 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.413+0000 7f6fa6616740 1 -- start start 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.413+0000 7f6fa6616740 1 Processor -- start 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aadee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa6616740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadeed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa6616740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1af370 con 0x5558aadd2b00 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa6616740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1b0670 con 0x5558aaddb8d0 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa6616740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1af9b0 con 0x5558aadd21e0 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa6616740 10 client.0 _ll_register_callbacks cb 0x7f6f9000bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aadee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aadee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49684/0 (socket says 172.21.15.3:49684) 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 -- 172.21.15.3:0/2026053772 learned_addr learned my addr 172.21.15.3:0/2026053772 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa1800640 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadeed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 -- 172.21.15.3:0/2026053772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 msgr2=0x5558aadee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 -- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 msgr2=0x5558aadeed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadeed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.414+0000 7f6f9be00640 1 -- 172.21.15.3:0/2026053772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558aae45fc0 con 0x5558aadd2b00 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.415+0000 7f6f9be00640 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aadee850 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802cc70 tx=0x7f6f880058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8fcb1b8a807b82 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.415+0000 7f6f82a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.415+0000 7f6f82a00640 1 -- 172.21.15.3:0/2026053772 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880044e0 con 0x5558aadd2b00 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.415+0000 7f6f82a00640 1 -- 172.21.15.3:0/2026053772 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f6f88005410 con 0x5558aadd2b00 2024-03-21T08:32:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.415+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558aad3e0c0 con 0x5558aadd2b00 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.415+0000 7f6f82a00640 1 -- 172.21.15.3:0/2026053772 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f88065050 con 0x5558aadd2b00 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 10 client.15999 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5558aae472d0 con 0x5558aadd2b00 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6f82a00640 1 -- 172.21.15.3:0/2026053772 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5558aad3e0c0 con 0x5558aadd2b00 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6f82a00640 1 -- 172.21.15.3:0/2026053772 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6f88003710 con 0x5558aadd2b00 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 10 client.15999 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 20 client.15999 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aae47950 0x5558aae49d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 4 client.15999 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.416+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5558aacbc5d0 con 0x5558aae47950 2024-03-21T08:32:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.417+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aae47950 0x5558aae49d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.418+0000 7f6fa0e00640 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aae47950 0x5558aae49d70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531d534088156dff server_cookie=8406bc0c9c2f1415 in_seq=0 out_seq=0 2024-03-21T08:32:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.418+0000 7f6f82a00640 10 client.15999 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.425+0000 7f6f82a00640 1 -- 172.21.15.3:0/2026053772 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5558aacbc5d0 con 0x5558aae47950 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.425+0000 7f6f82a00640 10 client.15999 handle_command_reply: tid=0 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.425+0000 7f6fa6616740 1 client.15999 shutdown 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 msgr2=0x5558aadee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aadee850 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802cc70 tx=0x7f6f880058b0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 shutdown_connections 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558aaddb8d0 0x5558aadeed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558aadd2b00 0x5558aadee850 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5558aae47950 0x5558aae49d70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 --2- 172.21.15.3:0/2026053772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558aadd21e0 0x5558aadee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 >> 172.21.15.3:0/2026053772 conn(0x5558aadccb00 msgr2=0x5558aadc76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6f82a00640 0 client.15999 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 shutdown_connections 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 1 -- 172.21.15.3:0/2026053772 wait complete. 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.426+0000 7f6fa6616740 20 client.15999 trim_cache size 0 max 16384 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:32:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:32:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f6f840a3190 msgr2=0x7f6f840a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f6f840a3190 0x7f6f840a5650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c102180 tx=0x7f6f88000bc0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c1075b0 msgr2=0x7f6f9c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c1075b0 0x7f6f9c101170 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400eaa0 tx=0x7f6f9400ef60 comp rx=0 tx=0).stop 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 shutdown_connections 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f9c1075b0 0x7f6f9c101170 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f9c069850 0x7f6f9c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f6f840a3190 0x7f6f840a5650 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 --2- 172.21.15.3:0/3510152516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f9c068e90 0x7f6f9c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 >> 172.21.15.3:0/3510152516 conn(0x7f6f9c075f00 msgr2=0x7f6f9c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 shutdown_connections 2024-03-21T08:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:46.427+0000 7f6fa2200640 1 -- 172.21.15.3:0/3510152516 wait complete. 2024-03-21T08:32:46.727 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T08:32:46.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:32:46.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T08:32:46.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:46 smithi003 ceph-mon[31675]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:46.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:46 smithi084 ceph-mon[35477]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:46.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:46 smithi171 ceph-mon[35388]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.950+0000 7f3af3200640 1 -- 172.21.15.3:0/3424213960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 msgr2=0x7f3af4101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.950+0000 7f3af3200640 1 --2- 172.21.15.3:0/3424213960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af4101920 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009900 tx=0x7f3ae002bc30 comp rx=0 tx=0).stop 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.950+0000 7f3af3200640 1 -- 172.21.15.3:0/3424213960 shutdown_connections 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.950+0000 7f3af3200640 1 --2- 172.21.15.3:0/3424213960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af4101e60 0x7f3af410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.950+0000 7f3af3200640 1 --2- 172.21.15.3:0/3424213960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af4101920 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.950+0000 7f3af3200640 1 --2- 172.21.15.3:0/3424213960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3af4107560 0x7f3af4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.950+0000 7f3af3200640 1 -- 172.21.15.3:0/3424213960 >> 172.21.15.3:0/3424213960 conn(0x7f3af40fcfb0 msgr2=0x7f3af40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.951+0000 7f3af3200640 1 -- 172.21.15.3:0/3424213960 shutdown_connections 2024-03-21T08:32:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.951+0000 7f3af3200640 1 -- 172.21.15.3:0/3424213960 wait complete. 2024-03-21T08:32:46.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.952+0000 7f3af3200640 1 Processor -- start 2024-03-21T08:32:46.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af3200640 1 -- start start 2024-03-21T08:32:46.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3af4101e60 0x7f3af419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af4107560 0x7f3af4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4111740 con 0x7f3af41014a0 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41115c0 con 0x7f3af4101e60 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41118c0 con 0x7f3af4107560 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49704/0 (socket says 172.21.15.3:49704) 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af1e00640 1 -- 172.21.15.3:0/2574558191 learned_addr learned my addr 172.21.15.3:0/2574558191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af1400640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3af4101e60 0x7f3af419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.953+0000 7f3af2800640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af4107560 0x7f3af4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.954+0000 7f3af1e00640 1 -- 172.21.15.3:0/2574558191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af4107560 msgr2=0x7f3af4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.954+0000 7f3af1e00640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af4107560 0x7f3af4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.954+0000 7f3af1e00640 1 -- 172.21.15.3:0/2574558191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3af4101e60 msgr2=0x7f3af419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.954+0000 7f3af1e00640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3af4101e60 0x7f3af419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.954+0000 7f3af1e00640 1 -- 172.21.15.3:0/2574558191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af41981a0 con 0x7f3af41014a0 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.954+0000 7f3af1e00640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af419d870 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d950 tx=0x7f3ae400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.955+0000 7f3aeaa00640 1 -- 172.21.15.3:0/2574558191 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4017070 con 0x7f3af41014a0 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.955+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af4198490 con 0x7f3af41014a0 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.955+0000 7f3aeaa00640 1 -- 172.21.15.3:0/2574558191 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ae4009c30 con 0x7f3af41014a0 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.955+0000 7f3aeaa00640 1 -- 172.21.15.3:0/2574558191 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4020060 con 0x7f3af41014a0 2024-03-21T08:32:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.955+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af41a4730 con 0x7f3af41014a0 2024-03-21T08:32:46.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.959+0000 7f3afba2b740 1 Processor -- start 2024-03-21T08:32:46.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.959+0000 7f3afba2b740 1 -- start start 2024-03-21T08:32:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.959+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.959+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 0x563b490b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.959+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3afba2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4900eeb0 con 0x563b490b3790 2024-03-21T08:32:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3afba2b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b48fb7a60 con 0x563b490aabb0 2024-03-21T08:32:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3afba2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4900af50 con 0x563b490aa290 2024-03-21T08:32:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3aeaa00640 1 -- 172.21.15.3:0/2574558191 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f3ae40109a0 con 0x7f3af41014a0 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49718/0 (socket says 172.21.15.3:49718) 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 -- 172.21.15.3:0/1806082044 learned_addr learned my addr 172.21.15.3:0/1806082044 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af2800640 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 -- 172.21.15.3:0/1806082044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 msgr2=0x563b490aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490aa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 -- 172.21.15.3:0/1806082044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 msgr2=0x563b490b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 0x563b490b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.960+0000 7f3af1400640 1 -- 172.21.15.3:0/1806082044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b48deb7e0 con 0x563b490b3790 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.961+0000 7f3af1400640 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b5b80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000c00 tx=0x7f3ae00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6f88cb813e0b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.961+0000 7f3aeaa00640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3ad80a31b0 0x7f3ad80a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.961+0000 7f3adf400640 1 -- 172.21.15.3:0/1806082044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0045020 con 0x563b490b3790 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.961+0000 7f3adf400640 1 -- 172.21.15.3:0/1806082044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ae002c780 con 0x563b490b3790 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.961+0000 7f3adf400640 1 -- 172.21.15.3:0/1806082044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00046c0 con 0x563b490b3790 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.961+0000 7f3aeaa00640 1 -- 172.21.15.3:0/2574558191 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3ae40cfb60 con 0x7f3af41014a0 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3af2800640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3ad80a31b0 0x7f3ad80a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 -- 172.21.15.3:0/1806082044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 msgr2=0x563b490b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b5b80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000c00 tx=0x7f3ae00095e0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3af2800640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3ad80a31b0 0x7f3ad80a5670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3aec005fd0 tx=0x7f3aec009600 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 -- 172.21.15.3:0/1806082044 shutdown_connections 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b5b80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 0x563b490b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 --2- 172.21.15.3:0/1806082044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 -- 172.21.15.3:0/1806082044 >> 172.21.15.3:0/1806082044 conn(0x563b490a49c0 msgr2=0x563b490a4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.962+0000 7f3afba2b740 1 -- 172.21.15.3:0/1806082044 shutdown_connections 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.963+0000 7f3afba2b740 1 -- 172.21.15.3:0/1806082044 wait complete. 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.963+0000 7f3afba2b740 1 -- start start 2024-03-21T08:32:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.963+0000 7f3afba2b740 1 Processor -- start 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.963+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 0x563b490b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3afba2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b490b6520 con 0x563b490b3790 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3afba2b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b48fcaeb0 con 0x563b490aabb0 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3afba2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b490b66a0 con 0x563b490aa290 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3afba2b740 10 client.0 _ll_register_callbacks cb 0x563b48fbf150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49734/0 (socket says 172.21.15.3:49734) 2024-03-21T08:32:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 -- 172.21.15.3:0/3174081601 learned_addr learned my addr 172.21.15.3:0/3174081601 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af1400640 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 0x563b490b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af1e00640 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 -- 172.21.15.3:0/3174081601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 msgr2=0x563b490b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 -- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 msgr2=0x563b490b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 0x563b490b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.964+0000 7f3af2800640 1 -- 172.21.15.3:0/3174081601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b48e4bf70 con 0x563b490b3790 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3af2800640 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b0af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01d6f0 tx=0x7f3aec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da857e8efa5d32f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3ad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3ad3e00640 1 -- 172.21.15.3:0/3174081601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec003b70 con 0x563b490b3790 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3ad3e00640 1 -- 172.21.15.3:0/3174081601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3aec003d10 con 0x563b490b3790 2024-03-21T08:32:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ae002c780 con 0x563b490b3790 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3ad3e00640 1 -- 172.21.15.3:0/3174081601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aec004030 con 0x563b490b3790 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3afba2b740 10 client.16011 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.965+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563b48ce3170 con 0x563b490b3790 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3ad3e00640 1 -- 172.21.15.3:0/3174081601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ae002c780 con 0x563b490b3790 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3ad3e00640 1 -- 172.21.15.3:0/3174081601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3aec00f040 con 0x563b490b3790 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3afba2b740 10 client.16011 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3afba2b740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3afba2b740 20 client.16011 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b490c7c50 0x563b490ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3afba2b740 4 client.16011 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b4901c3f0 con 0x563b490c7c50 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.966+0000 7f3af1e00640 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b490c7c50 0x563b490ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.967+0000 7f3af1e00640 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b490c7c50 0x563b490ca040 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6d0327aa2175ef server_cookie=4342aa840e8821bf in_seq=0 out_seq=0 2024-03-21T08:32:46.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.967+0000 7f3ad3e00640 10 client.16011 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3ad3e00640 1 -- 172.21.15.3:0/3174081601 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563b4901c3f0 con 0x563b490c7c50 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3ad3e00640 10 client.16011 handle_command_reply: tid=0 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 client.16011 shutdown 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 msgr2=0x563b490b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b0af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01d6f0 tx=0x7f3aec004a40 comp rx=0 tx=0).stop 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 shutdown_connections 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490b0af0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aabb0 0x563b490b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b490c7c50 0x563b490ca040 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3174081601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aa290 0x563b490b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.969+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 >> 172.21.15.3:0/3174081601 conn(0x563b490a49c0 msgr2=0x563b490a6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.970+0000 7f3ad3e00640 0 client.16011 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.970+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 shutdown_connections 2024-03-21T08:32:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.970+0000 7f3afba2b740 1 -- 172.21.15.3:0/3174081601 wait complete. 2024-03-21T08:32:46.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:46.970+0000 7f3afba2b740 20 client.16011 trim_cache size 0 max 16384 2024-03-21T08:32:47.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.003+0000 7f3afba2b740 1 Processor -- start 2024-03-21T08:32:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.003+0000 7f3afba2b740 1 -- start start 2024-03-21T08:32:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3afba2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41aec50 con 0x563b490b3790 2024-03-21T08:32:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3afba2b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41ac750 con 0x563b490aa290 2024-03-21T08:32:47.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3afba2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41aa7f0 con 0x563b490aabb0 2024-03-21T08:32:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3af2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3af2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49750/0 (socket says 172.21.15.3:49750) 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3af2800640 1 -- 172.21.15.3:0/3627017751 learned_addr learned my addr 172.21.15.3:0/3627017751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3af1400640 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.004+0000 7f3af1e00640 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3af2800640 1 -- 172.21.15.3:0/3627017751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 msgr2=0x563b490b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3af2800640 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3af2800640 1 -- 172.21.15.3:0/3627017751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 msgr2=0x563b490b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3af2800640 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3af2800640 1 -- 172.21.15.3:0/3627017751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b490a40a0 con 0x563b490b3790 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3af1e00640 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3af2800640 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490c3240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01d540 tx=0x7f3aec009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd65d30dda71fb83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.005+0000 7f3adea00640 1 -- 172.21.15.3:0/3627017751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0110c0 con 0x563b490b3790 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.006+0000 7f3adea00640 1 -- 172.21.15.3:0/3627017751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3aec010670 con 0x563b490b3790 2024-03-21T08:32:47.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.006+0000 7f3adea00640 1 -- 172.21.15.3:0/3627017751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0041d0 con 0x563b490b3790 2024-03-21T08:32:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.006+0000 7f3afba2b740 1 -- 172.21.15.3:0/3627017751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 msgr2=0x563b490c3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.006+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490c3240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01d540 tx=0x7f3aec009770 comp rx=0 tx=0).stop 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 -- 172.21.15.3:0/3627017751 shutdown_connections 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b490c3240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 --2- 172.21.15.3:0/3627017751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 -- 172.21.15.3:0/3627017751 >> 172.21.15.3:0/3627017751 conn(0x563b490a49c0 msgr2=0x563b490af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 -- 172.21.15.3:0/3627017751 shutdown_connections 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 -- 172.21.15.3:0/3627017751 wait complete. 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.007+0000 7f3afba2b740 1 -- start start 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 1 Processor -- start 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b49122480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41aadf0 con 0x563b490b3790 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41ac0f0 con 0x563b490aa290 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af41ab430 con 0x563b490aabb0 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3afba2b740 10 client.0 _ll_register_callbacks cb 0x7f3aec009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.008+0000 7f3af2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b49122480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b49122480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49764/0 (socket says 172.21.15.3:49764) 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 -- 172.21.15.3:0/2805939124 learned_addr learned my addr 172.21.15.3:0/2805939124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af1e00640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af1400640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 -- 172.21.15.3:0/2805939124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 msgr2=0x563b490ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 -- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 msgr2=0x563b490ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 -- 172.21.15.3:0/2805939124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b491229c0 con 0x563b490b3790 2024-03-21T08:32:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af1e00640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3af2800640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b49122480 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01d6f0 tx=0x7f3aec010a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ea4bb35c791eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.009+0000 7f3ad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3ad3e00640 1 -- 172.21.15.3:0/2805939124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec043030 con 0x563b490b3790 2024-03-21T08:32:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3ad3e00640 1 -- 172.21.15.3:0/2805939124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3aec011080 con 0x563b490b3790 2024-03-21T08:32:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aec010670 con 0x563b490b3790 2024-03-21T08:32:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3ad3e00640 1 -- 172.21.15.3:0/2805939124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aec003970 con 0x563b490b3790 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3afba2b740 10 client.16017 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563b49123ad0 con 0x563b490b3790 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3ad3e00640 1 -- 172.21.15.3:0/2805939124 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3aec010670 con 0x563b490b3790 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.010+0000 7f3ad3e00640 1 -- 172.21.15.3:0/2805939124 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3aec005330 con 0x563b490b3790 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.011+0000 7f3afba2b740 10 client.16017 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.011+0000 7f3afba2b740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.011+0000 7f3afba2b740 20 client.16017 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.011+0000 7f3afba2b740 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b49123f00 0x563b491262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.011+0000 7f3afba2b740 4 client.16017 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.011+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563b49016740 con 0x563b49123f00 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.011+0000 7f3af1e00640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b49123f00 0x563b491262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.012+0000 7f3af1e00640 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b49123f00 0x563b491262f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb2eceffafa0bb6 server_cookie=a6b117994e46d566 in_seq=0 out_seq=0 2024-03-21T08:32:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.012+0000 7f3ad3e00640 10 client.16017 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:47.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.014+0000 7f3ad3e00640 1 -- 172.21.15.3:0/2805939124 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563b49016740 con 0x563b49123f00 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.014+0000 7f3ad3e00640 10 client.16017 handle_command_reply: tid=0 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.014+0000 7f3afba2b740 1 client.16017 shutdown 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.014+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 msgr2=0x563b49122480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.014+0000 7f3afba2b740 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b49122480 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01d6f0 tx=0x7f3aec010a30 comp rx=0 tx=0).stop 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 shutdown_connections 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563b490b3790 0x563b49122480 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563b49123f00 0x563b491262f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b490aabb0 0x563b490ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 --2- 172.21.15.3:0/2805939124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b490aa290 0x563b490ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 >> 172.21.15.3:0/2805939124 conn(0x563b490a49c0 msgr2=0x563b490ab0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3ad3e00640 0 client.16017 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 shutdown_connections 2024-03-21T08:32:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 1 -- 172.21.15.3:0/2805939124 wait complete. 2024-03-21T08:32:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.015+0000 7f3afba2b740 20 client.16017 trim_cache size 0 max 16384 2024-03-21T08:32:47.018 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:32:47.018 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-03-21T08:32:47.018 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "99158e08-be83-428a-83c1-a7a1fefb204b", 2024-03-21T08:32:47.018 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-03-21T08:32:47.018 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3ad80a31b0 msgr2=0x7f3ad80a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3ad80a31b0 0x7f3ad80a5670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3aec005fd0 tx=0x7f3aec009600 comp rx=0 tx=0).stop 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 msgr2=0x7f3af419d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af419d870 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d950 tx=0x7f3ae400de10 comp rx=0 tx=0).stop 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 shutdown_connections 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3ad80a31b0 0x7f3ad80a5670 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af4107560 0x7f3af4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3af4101e60 0x7f3af419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 --2- 172.21.15.3:0/2574558191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3af41014a0 0x7f3af419d870 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 >> 172.21.15.3:0/2574558191 conn(0x7f3af40fcfb0 msgr2=0x7f3af40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 shutdown_connections 2024-03-21T08:32:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.016+0000 7f3af3200640 1 -- 172.21.15.3:0/2574558191 wait complete. 2024-03-21T08:32:47.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:32:47.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:32:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 -- 172.21.15.3:0/4220478349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee4101e00 msgr2=0x7f9ee410e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 --2- 172.21.15.3:0/4220478349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee4101e00 0x7f9ee410e310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009cd0 tx=0x7f9edc02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 -- 172.21.15.3:0/4220478349 shutdown_connections 2024-03-21T08:32:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 --2- 172.21.15.3:0/4220478349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee4101e00 0x7f9ee410e310 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 --2- 172.21.15.3:0/4220478349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ee4101440 0x7f9ee41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 --2- 172.21.15.3:0/4220478349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee41074e0 0x7f9ee41078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 -- 172.21.15.3:0/4220478349 >> 172.21.15.3:0/4220478349 conn(0x7f9ee40fcf30 msgr2=0x7f9ee40ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 -- 172.21.15.3:0/4220478349 shutdown_connections 2024-03-21T08:32:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.251+0000 7f9ee9600640 1 -- 172.21.15.3:0/4220478349 wait complete. 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 Processor -- start 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 -- start start 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ee4101440 0x7f9ee419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4101e00 0x7f9ee419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee41074e0 0x7f9ee419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4073c60 con 0x7f9ee41074e0 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4073ae0 con 0x7f9ee4101e00 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.253+0000 7f9ee9600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4073de0 con 0x7f9ee4101440 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee41074e0 0x7f9ee419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee41074e0 0x7f9ee419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49782/0 (socket says 172.21.15.3:49782) 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 -- 172.21.15.3:0/1339217485 learned_addr learned my addr 172.21.15.3:0/1339217485 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9edb400640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4101e00 0x7f9ee419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ee4101440 0x7f9ee419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 -- 172.21.15.3:0/1339217485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ee4101440 msgr2=0x7f9ee419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ee4101440 0x7f9ee419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 -- 172.21.15.3:0/1339217485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4101e00 msgr2=0x7f9ee419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4101e00 0x7f9ee419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 -- 172.21.15.3:0/1339217485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee419ee90 con 0x7f9ee41074e0 2024-03-21T08:32:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.254+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee41074e0 0x7f9ee419e760 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9edc004af0 tx=0x7f9edc035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.255+0000 7f9ee1600640 1 -- 172.21.15.3:0/1339217485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc039040 con 0x7f9ee41074e0 2024-03-21T08:32:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.255+0000 7f9ee1600640 1 -- 172.21.15.3:0/1339217485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9edc044070 con 0x7f9ee41074e0 2024-03-21T08:32:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.255+0000 7f9ee1600640 1 -- 172.21.15.3:0/1339217485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc004310 con 0x7f9ee41074e0 2024-03-21T08:32:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.255+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee419f120 con 0x7f9ee41074e0 2024-03-21T08:32:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.255+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee41a6a10 con 0x7f9ee41074e0 2024-03-21T08:32:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.258+0000 7f9eed9c5740 1 Processor -- start 2024-03-21T08:32:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.260+0000 7f9ee1600640 1 -- 172.21.15.3:0/1339217485 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f9edc0044b0 con 0x7f9ee41074e0 2024-03-21T08:32:47.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.260+0000 7f9eed9c5740 1 -- start start 2024-03-21T08:32:47.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.260+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa02eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.260+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa02f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.260+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa038460 0x55e8aa03a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.260+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a9f93250 con 0x55e8aa02f0f0 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a9f3ba30 con 0x55e8aa02fdd0 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a9f8f2f0 con 0x55e8aa038460 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa02f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa02f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49784/0 (socket says 172.21.15.3:49784) 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 -- 172.21.15.3:0/1870628371 learned_addr learned my addr 172.21.15.3:0/1870628371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9edb400640 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa02eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9edb400640 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa038460 0x55e8aa03a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 -- 172.21.15.3:0/1870628371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa038460 msgr2=0x55e8aa03a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa038460 0x55e8aa03a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 -- 172.21.15.3:0/1870628371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 msgr2=0x55e8aa02eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa02eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee3e00640 1 -- 172.21.15.3:0/1870628371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8a9f26690 con 0x55e8aa02f0f0 2024-03-21T08:32:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9edb400640 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa038460 0x55e8aa03a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.261+0000 7f9ee1600640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9ec80a31b0 0x7f9ec80a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.262+0000 7f9ee1600640 1 -- 172.21.15.3:0/1339217485 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f9edc0f2040 con 0x7f9ee41074e0 2024-03-21T08:32:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.262+0000 7f9edb400640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9ec80a31b0 0x7f9ec80a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.262+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa02f570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400dc30 tx=0x7f9ed400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a3ca07e9bad20f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.262+0000 7f9ed9200640 1 -- 172.21.15.3:0/1870628371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4011440 con 0x55e8aa02f0f0 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.262+0000 7f9ed9200640 1 -- 172.21.15.3:0/1870628371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ed4017070 con 0x55e8aa02f0f0 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.262+0000 7f9ed9200640 1 -- 172.21.15.3:0/1870628371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4010830 con 0x55e8aa02f0f0 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1870628371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 msgr2=0x55e8aa02f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9edb400640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9ec80a31b0 0x7f9ec80a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40fec30 tx=0x7f9ecc004730 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa02f570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400dc30 tx=0x7f9ed400f730 comp rx=0 tx=0).stop 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1870628371 shutdown_connections 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa038460 0x55e8aa03a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa02f570 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1870628371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa02eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1870628371 >> 172.21.15.3:0/1870628371 conn(0x55e8a9ff6200 msgr2=0x55e8aa029b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1870628371 shutdown_connections 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.263+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1870628371 wait complete. 2024-03-21T08:32:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.264+0000 7f9eed9c5740 1 -- start start 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.264+0000 7f9eed9c5740 1 Processor -- start 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.264+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa037b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02fdd0 0x55e8aa034e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8aa03b2d0 con 0x55e8aa02f0f0 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a9f4ee80 con 0x55e8aa038460 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8aa03b450 con 0x55e8aa02fdd0 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9eed9c5740 10 client.0 _ll_register_callbacks cb 0x55e8a9f61130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa037b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa037b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49796/0 (socket says 172.21.15.3:49796) 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 -- 172.21.15.3:0/999470054 learned_addr learned my addr 172.21.15.3:0/999470054 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9edb400640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02fdd0 0x55e8aa034e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 -- 172.21.15.3:0/999470054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02fdd0 msgr2=0x55e8aa034e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02fdd0 0x55e8aa034e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 -- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 msgr2=0x55e8aa0353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9ee3e00640 1 -- 172.21.15.3:0/999470054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8a9d717d0 con 0x55e8aa02f0f0 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.265+0000 7f9edb400640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02fdd0 0x55e8aa034e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa037b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4010570 tx=0x7f9ed400d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3010d9a88440d8cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9ed1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9ed1600640 1 -- 172.21.15.3:0/999470054 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed402b060 con 0x55e8aa02f0f0 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ed4017070 con 0x55e8aa02f0f0 2024-03-21T08:32:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9ed1600640 1 -- 172.21.15.3:0/999470054 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f9ed4004030 con 0x55e8aa02f0f0 2024-03-21T08:32:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9ed1600640 1 -- 172.21.15.3:0/999470054 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ed4032050 con 0x55e8aa02f0f0 2024-03-21T08:32:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9eed9c5740 10 client.16029 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.266+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e8a9ca73f0 con 0x55e8aa02f0f0 2024-03-21T08:32:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9ed1600640 1 -- 172.21.15.3:0/999470054 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ed4017070 con 0x55e8aa02f0f0 2024-03-21T08:32:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9ed1600640 1 -- 172.21.15.3:0/999470054 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9ed40047e0 con 0x55e8aa02f0f0 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9eed9c5740 10 client.16029 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9eed9c5740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9eed9c5740 20 client.16029 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa048290 0x55e8aa04a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9eed9c5740 4 client.16029 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8a9fa0790 con 0x55e8aa048290 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.267+0000 7f9edb400640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa048290 0x55e8aa04a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.268+0000 7f9edb400640 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa048290 0x55e8aa04a680 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9586dd2dbc2cde38 server_cookie=c729840dd096712e in_seq=0 out_seq=0 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.268+0000 7f9ed1600640 10 client.16029 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9ed1600640 1 -- 172.21.15.3:0/999470054 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e8a9fa0790 con 0x55e8aa048290 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9ed1600640 10 client.16029 handle_command_reply: tid=0 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 client.16029 shutdown 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 msgr2=0x55e8aa037b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa037b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4010570 tx=0x7f9ed400d970 comp rx=0 tx=0).stop 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 shutdown_connections 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa048290 0x55e8aa04a680 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02fdd0 0x55e8aa034e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/999470054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02f0f0 0x55e8aa037b00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 >> 172.21.15.3:0/999470054 conn(0x55e8a9ff6200 msgr2=0x55e8aa02b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9ed1600640 0 client.16029 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 shutdown_connections 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 1 -- 172.21.15.3:0/999470054 wait complete. 2024-03-21T08:32:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.270+0000 7f9eed9c5740 20 client.16029 trim_cache size 0 max 16384 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 Processor -- start 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 -- start start 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa0354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa033e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa038460 0x55e8aa0375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41b30b0 con 0x55e8aa038460 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41b0bb0 con 0x55e8aa02fdd0 2024-03-21T08:32:47.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.288+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41aec50 con 0x55e8aa02f0f0 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa038460 0x55e8aa0375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa038460 0x55e8aa0375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49800/0 (socket says 172.21.15.3:49800) 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 -- 172.21.15.3:0/3432715571 learned_addr learned my addr 172.21.15.3:0/3432715571 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9edb400640 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa033e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa0354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 -- 172.21.15.3:0/3432715571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 msgr2=0x55e8aa0354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa0354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 -- 172.21.15.3:0/3432715571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 msgr2=0x55e8aa033e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa033e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 -- 172.21.15.3:0/3432715571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8aa04a9f0 con 0x55e8aa038460 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.289+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa038460 0x55e8aa0375c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0bdbd0 tx=0x7f9edc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf59c988e22f52e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.290+0000 7f9ed3e00640 1 -- 172.21.15.3:0/3432715571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0bded0 con 0x55e8aa038460 2024-03-21T08:32:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.290+0000 7f9ed3e00640 1 -- 172.21.15.3:0/3432715571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9edc0c2020 con 0x55e8aa038460 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.290+0000 7f9ed3e00640 1 -- 172.21.15.3:0/3432715571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0bc260 con 0x55e8aa038460 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.290+0000 7f9eed9c5740 1 -- 172.21.15.3:0/3432715571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa038460 msgr2=0x55e8aa0375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.290+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa038460 0x55e8aa0375c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0bdbd0 tx=0x7f9edc005ef0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 -- 172.21.15.3:0/3432715571 shutdown_connections 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa038460 0x55e8aa0375c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa02fdd0 0x55e8aa033e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/3432715571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa0354c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 -- 172.21.15.3:0/3432715571 >> 172.21.15.3:0/3432715571 conn(0x55e8a9ff6200 msgr2=0x55e8aa033780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 -- 172.21.15.3:0/3432715571 shutdown_connections 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 -- 172.21.15.3:0/3432715571 wait complete. 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 -- start start 2024-03-21T08:32:47.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.291+0000 7f9eed9c5740 1 Processor -- start 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa031720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02fdd0 0x55e8aa031c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9eed9c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41af250 con 0x55e8aa02fdd0 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41b0550 con 0x55e8aa038460 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9eed9c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41af890 con 0x55e8aa02f0f0 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9eed9c5740 10 client.0 _ll_register_callbacks cb 0x7f9ed4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02fdd0 0x55e8aa031c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02fdd0 0x55e8aa031c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49804/0 (socket says 172.21.15.3:49804) 2024-03-21T08:32:47.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 -- 172.21.15.3:0/1541257420 learned_addr learned my addr 172.21.15.3:0/1541257420 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9ee8c00640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa031720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 -- 172.21.15.3:0/1541257420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 msgr2=0x55e8aa031720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa031720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 -- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 msgr2=0x55e8aa0321a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0321a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9edb400640 1 -- 172.21.15.3:0/1541257420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8a9c670f0 con 0x55e8aa02fdd0 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.292+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa031720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9edb400640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02fdd0 0x55e8aa031c60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55e8aa04abf0 tx=0x7f9ecc00c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5081bf7eaa806f3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9ed1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9ed1600640 1 -- 172.21.15.3:0/1541257420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc005220 con 0x55e8aa02fdd0 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9edc0c2020 con 0x55e8aa02fdd0 2024-03-21T08:32:47.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9ed1600640 1 -- 172.21.15.3:0/1541257420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f9ecc0053c0 con 0x55e8aa02fdd0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9ed1600640 1 -- 172.21.15.3:0/1541257420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ecc039050 con 0x55e8aa02fdd0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9eed9c5740 10 client.16035 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.293+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e8aa0a3bc0 con 0x55e8aa02fdd0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9ed1600640 1 -- 172.21.15.3:0/1541257420 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9edc0c2020 con 0x55e8aa02fdd0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9ed1600640 1 -- 172.21.15.3:0/1541257420 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9ecc029620 con 0x55e8aa02fdd0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9eed9c5740 10 client.16035 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9eed9c5740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9eed9c5740 20 client.16035 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa0a42d0 0x55e8aa0a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9eed9c5740 4 client.16035 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e8a9f9aae0 con 0x55e8aa0a42d0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.294+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa0a42d0 0x55e8aa0a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.295+0000 7f9ee3e00640 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa0a42d0 0x55e8aa0a66f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d027ee729ab9204 server_cookie=66a3cf067e4bdce8 in_seq=0 out_seq=0 2024-03-21T08:32:47.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.295+0000 7f9ed1600640 10 client.16035 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.295+0000 7f9ed1600640 1 -- 172.21.15.3:0/1541257420 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e8a9f9aae0 con 0x55e8aa0a42d0 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.295+0000 7f9ed1600640 10 client.16035 handle_command_reply: tid=0 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.295+0000 7f9eed9c5740 1 client.16035 shutdown 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02fdd0 msgr2=0x55e8aa031c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02fdd0 0x55e8aa031c60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55e8aa04abf0 tx=0x7f9ecc00c610 comp rx=0 tx=0).stop 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 shutdown_connections 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8aa038460 0x55e8aa0321a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e8aa02fdd0 0x55e8aa031c60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e8aa0a42d0 0x55e8aa0a66f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 --2- 172.21.15.3:0/1541257420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e8aa02f0f0 0x55e8aa031720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 >> 172.21.15.3:0/1541257420 conn(0x55e8a9ff6200 msgr2=0x55e8aa028330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9ed1600640 0 client.16035 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 shutdown_connections 2024-03-21T08:32:47.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 1 -- 172.21.15.3:0/1541257420 wait complete. 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.296+0000 7f9eed9c5740 20 client.16035 trim_cache size 0 max 16384 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: "99158e08-be83-428a-83c1-a7a1fefb204b": { 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "99158e08-be83-428a-83c1-a7a1fefb204b", 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:32:47.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:32:47.303 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:32:47.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9ec80a31b0 msgr2=0x7f9ec80a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9ec80a31b0 0x7f9ec80a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40fec30 tx=0x7f9ecc004730 comp rx=0 tx=0).stop 2024-03-21T08:32:47.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee41074e0 msgr2=0x7f9ee419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:47.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee41074e0 0x7f9ee419e760 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9edc004af0 tx=0x7f9edc035ce0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 shutdown_connections 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ee41074e0 0x7f9ee419e760 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4101e00 0x7f9ee419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f9ec80a31b0 0x7f9ec80a5670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 --2- 172.21.15.3:0/1339217485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ee4101440 0x7f9ee419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 >> 172.21.15.3:0/1339217485 conn(0x7f9ee40fcf30 msgr2=0x7f9ee4074630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 shutdown_connections 2024-03-21T08:32:47.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:32:47.297+0000 7f9ee9600640 1 -- 172.21.15.3:0/1339217485 wait complete. 2024-03-21T08:32:47.315 INFO:tasks.cephfs.filesystem:scrub status for tag:99158e08-be83-428a-83c1-a7a1fefb204b - {'path': '/', 'tag': '99158e08-be83-428a-83c1-a7a1fefb204b', 'options': 'recursive,force'} 2024-03-21T08:32:48.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:48 smithi084 ceph-mon[35477]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:48.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:48 smithi084 ceph-mon[35477]: scrub queued for path: / 2024-03-21T08:32:48.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:48 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:48.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:48 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:32:48.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:48 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:48.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:48 smithi171 ceph-mon[35388]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:48.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:48 smithi171 ceph-mon[35388]: scrub queued for path: / 2024-03-21T08:32:48.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:48 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:48.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:48 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:32:48.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:48 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:48.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:48 smithi003 ceph-mon[31675]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:48.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:48 smithi003 ceph-mon[31675]: scrub queued for path: / 2024-03-21T08:32:48.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:48 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:48.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:48 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:32:48.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:48 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:32:50.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:50 smithi084 ceph-mon[35477]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-21T08:32:50.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:50 smithi171 ceph-mon[35388]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-21T08:32:50.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:50 smithi003 ceph-mon[31675]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-21T08:32:51.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:51 smithi084 ceph-mon[35477]: scrub summary: idle 2024-03-21T08:32:51.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:51 smithi171 ceph-mon[35388]: scrub summary: idle 2024-03-21T08:32:51.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:51 smithi003 ceph-mon[31675]: scrub summary: idle 2024-03-21T08:32:52.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:52 smithi003 ceph-mon[31675]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:52.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:52 smithi171 ceph-mon[35388]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:52.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:52 smithi084 ceph-mon[35477]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T08:32:54.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:54 smithi084 ceph-mon[35477]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:32:54.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:54 smithi171 ceph-mon[35388]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:32:54.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:54 smithi003 ceph-mon[31675]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:32:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:56 smithi084 ceph-mon[35477]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:56.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:56 smithi171 ceph-mon[35388]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:56.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:56 smithi003 ceph-mon[31675]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:58.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:32:58 smithi084 ceph-mon[35477]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:58.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:32:58 smithi171 ceph-mon[35388]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:58.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:32:58 smithi003 ceph-mon[31675]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.617+0000 7f4069200640 1 -- 172.21.15.3:0/2741057538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40640693d0 msgr2=0x7f4064069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.617+0000 7f4069200640 1 --2- 172.21.15.3:0/2741057538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40640693d0 0x7f4064069810 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4050009900 tx=0x7f405002bcd0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.618+0000 7f4069200640 1 -- 172.21.15.3:0/2741057538 shutdown_connections 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.618+0000 7f4069200640 1 --2- 172.21.15.3:0/2741057538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406410dc90 0x7f4064110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.618+0000 7f4069200640 1 --2- 172.21.15.3:0/2741057538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40640693d0 0x7f4064069810 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.618+0000 7f4069200640 1 --2- 172.21.15.3:0/2741057538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064102430 0x7f4064068e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.618+0000 7f4069200640 1 -- 172.21.15.3:0/2741057538 >> 172.21.15.3:0/2741057538 conn(0x7f4064075f00 msgr2=0x7f4064076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.618+0000 7f4069200640 1 -- 172.21.15.3:0/2741057538 shutdown_connections 2024-03-21T08:32:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.618+0000 7f4069200640 1 -- 172.21.15.3:0/2741057538 wait complete. 2024-03-21T08:32:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.619+0000 7f4069200640 1 Processor -- start 2024-03-21T08:32:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.619+0000 7f4069200640 1 -- start start 2024-03-21T08:32:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.620+0000 7f4069200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40640693d0 0x7f406419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.620+0000 7f4069200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4064102430 0x7f406419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.620+0000 7f4069200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406410dc90 0x7f4064199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.620+0000 7f4069200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4064113960 con 0x7f4064102430 2024-03-21T08:32:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.620+0000 7f4069200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641137e0 con 0x7f40640693d0 2024-03-21T08:32:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.620+0000 7f4069200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4064113ae0 con 0x7f406410dc90 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.620+0000 7f4062a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4064102430 0x7f406419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4062a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4064102430 0x7f406419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39402/0 (socket says 172.21.15.3:39402) 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4062a00640 1 -- 172.21.15.3:0/237880966 learned_addr learned my addr 172.21.15.3:0/237880966 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4063400640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40640693d0 0x7f406419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4063e00640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406410dc90 0x7f4064199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4062a00640 1 -- 172.21.15.3:0/237880966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406410dc90 msgr2=0x7f4064199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4062a00640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406410dc90 0x7f4064199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4062a00640 1 -- 172.21.15.3:0/237880966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40640693d0 msgr2=0x7f406419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4062a00640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40640693d0 0x7f406419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.621+0000 7f4062a00640 1 -- 172.21.15.3:0/237880966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406419a1e0 con 0x7f4064102430 2024-03-21T08:32:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.622+0000 7f4062a00640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4064102430 0x7f406419fdf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f405002c210 tx=0x7f405002db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.622+0000 7f405be00640 1 -- 172.21.15.3:0/237880966 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050042020 con 0x7f4064102430 2024-03-21T08:32:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.622+0000 7f405be00640 1 -- 172.21.15.3:0/237880966 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f405002c670 con 0x7f4064102430 2024-03-21T08:32:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.622+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406419a470 con 0x7f4064102430 2024-03-21T08:32:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.622+0000 7f405be00640 1 -- 172.21.15.3:0/237880966 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050003b80 con 0x7f4064102430 2024-03-21T08:32:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.622+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40641a6770 con 0x7f4064102430 2024-03-21T08:32:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.627+0000 7f406d3f4740 1 Processor -- start 2024-03-21T08:32:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.627+0000 7f406d3f4740 1 -- start start 2024-03-21T08:32:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.627+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.627+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f406d3f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620582ac3b0 con 0x5620582eab00 2024-03-21T08:32:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f406d3f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205824e830 con 0x5620582ea1e0 2024-03-21T08:32:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f406d3f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620581f75d0 con 0x5620582f38d0 2024-03-21T08:32:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f405be00640 1 -- 172.21.15.3:0/237880966 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f4050045020 con 0x7f4064102430 2024-03-21T08:32:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f4063e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f4063e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582ea5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55880/0 (socket says 172.21.15.3:55880) 2024-03-21T08:32:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f4063e00640 1 -- 172.21.15.3:0/977450474 learned_addr learned my addr 172.21.15.3:0/977450474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.628+0000 7f4063400640 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4062a00640 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4063e00640 1 -- 172.21.15.3:0/977450474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 msgr2=0x5620582f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4063e00640 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4063e00640 1 -- 172.21.15.3:0/977450474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 msgr2=0x5620582f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4063e00640 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4063e00640 1 -- 172.21.15.3:0/977450474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56205820f2c0 con 0x5620582ea1e0 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4063400640 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.629+0000 7f4062a00640 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.630+0000 7f4063e00640 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582ea5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f405c009fd0 tx=0x7f405c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2363886a2b647c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.630+0000 7f4058c00640 1 -- 172.21.15.3:0/977450474 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c004d20 con 0x5620582ea1e0 2024-03-21T08:32:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.630+0000 7f405be00640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f40480a3210 0x7f40480a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.630+0000 7f4058c00640 1 -- 172.21.15.3:0/977450474 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f405c004050 con 0x5620582ea1e0 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.630+0000 7f4058c00640 1 -- 172.21.15.3:0/977450474 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c005240 con 0x5620582ea1e0 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.631+0000 7f405be00640 1 -- 172.21.15.3:0/237880966 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f40500f1eb0 con 0x7f4064102430 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.631+0000 7f4063400640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f40480a3210 0x7f40480a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.631+0000 7f406d3f4740 1 -- 172.21.15.3:0/977450474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 msgr2=0x5620582ea5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.631+0000 7f406d3f4740 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582ea5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f405c009fd0 tx=0x7f405c0118d0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f4063400640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f40480a3210 0x7f40480a56d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5620582e9c90 tx=0x7f4054005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f406d3f4740 1 -- 172.21.15.3:0/977450474 shutdown_connections 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f406d3f4740 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f406d3f4740 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f406d3f4740 1 --2- 172.21.15.3:0/977450474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582ea5c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f406d3f4740 1 -- 172.21.15.3:0/977450474 >> 172.21.15.3:0/977450474 conn(0x5620582e4b00 msgr2=0x5620582e4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f406d3f4740 1 -- 172.21.15.3:0/977450474 shutdown_connections 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.632+0000 7f406d3f4740 1 -- 172.21.15.3:0/977450474 wait complete. 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.633+0000 7f406d3f4740 1 -- start start 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.633+0000 7f406d3f4740 1 Processor -- start 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f406d3f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205820aa20 con 0x5620582eab00 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f406d3f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205824a8d0 con 0x5620582ea1e0 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f406d3f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620582f6500 con 0x5620582f38d0 2024-03-21T08:32:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f406d3f4740 10 client.0 _ll_register_callbacks cb 0x5620582471c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f4062a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f4063400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f4062a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39428/0 (socket says 172.21.15.3:39428) 2024-03-21T08:32:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.634+0000 7f4062a00640 1 -- 172.21.15.3:0/1627757047 learned_addr learned my addr 172.21.15.3:0/1627757047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.635+0000 7f4063e00640 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.635+0000 7f4062a00640 1 -- 172.21.15.3:0/1627757047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 msgr2=0x5620582f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.635+0000 7f4062a00640 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.635+0000 7f4062a00640 1 -- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 msgr2=0x5620582f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.635+0000 7f4062a00640 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.635+0000 7f4062a00640 1 -- 172.21.15.3:0/1627757047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562057f63210 con 0x5620582eab00 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.635+0000 7f4063400640 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.636+0000 7f4062a00640 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f0f20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40500bda40 tx=0x7f40500bdbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506960d82b05df84 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.636+0000 7f404d600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.636+0000 7f404d600640 1 -- 172.21.15.3:0/1627757047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40500bdd40 con 0x5620582eab00 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.636+0000 7f404d600640 1 -- 172.21.15.3:0/1627757047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f40500c62c0 con 0x5620582eab00 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.636+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f405c004050 con 0x5620582eab00 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.636+0000 7f404d600640 1 -- 172.21.15.3:0/1627757047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40500c1e90 con 0x5620582eab00 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.636+0000 7f406d3f4740 10 client.16047 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.637+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56205802f620 con 0x5620582eab00 2024-03-21T08:32:59.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.637+0000 7f404d600640 1 -- 172.21.15.3:0/1627757047 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f405c004050 con 0x5620582eab00 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.637+0000 7f404d600640 1 -- 172.21.15.3:0/1627757047 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f40500b9a90 con 0x5620582eab00 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.637+0000 7f406d3f4740 10 client.16047 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.637+0000 7f406d3f4740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.637+0000 7f406d3f4740 20 client.16047 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.637+0000 7f406d3f4740 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562058303820 0x562058305c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.638+0000 7f406d3f4740 4 client.16047 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.638+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56205825bd70 con 0x562058303820 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.638+0000 7f4063400640 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562058303820 0x562058305c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.639+0000 7f4063400640 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562058303820 0x562058305c30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be8b1aa642e5b89 server_cookie=fcf1aeacca42da9a in_seq=0 out_seq=0 2024-03-21T08:32:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.639+0000 7f404d600640 10 client.16047 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.641+0000 7f404d600640 1 -- 172.21.15.3:0/1627757047 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56205825bd70 con 0x562058303820 2024-03-21T08:32:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.641+0000 7f404d600640 10 client.16047 handle_command_reply: tid=0 2024-03-21T08:32:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.641+0000 7f406d3f4740 1 client.16047 shutdown 2024-03-21T08:32:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 msgr2=0x5620582f0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f0f20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40500bda40 tx=0x7f40500bdbd0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 shutdown_connections 2024-03-21T08:32:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562058303820 0x562058305c30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f38d0 0x5620582f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582eab00 0x5620582f0f20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 --2- 172.21.15.3:0/1627757047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620582ea1e0 0x5620582f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f404d600640 0 client.16047 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 >> 172.21.15.3:0/1627757047 conn(0x5620582e4b00 msgr2=0x5620582e6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 shutdown_connections 2024-03-21T08:32:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.642+0000 7f406d3f4740 1 -- 172.21.15.3:0/1627757047 wait complete. 2024-03-21T08:32:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.643+0000 7f406d3f4740 20 client.16047 trim_cache size 0 max 16384 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 Processor -- start 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 -- start start 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582ea1e0 0x5620582f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582f38d0 0x5620582fec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641b2f50 con 0x5620582f38d0 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641b0a50 con 0x56205830b590 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.671+0000 7f406d3f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641aeaf0 con 0x5620582ea1e0 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582f38d0 0x5620582fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582f38d0 0x5620582fec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39438/0 (socket says 172.21.15.3:39438) 2024-03-21T08:32:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 -- 172.21.15.3:0/3001360995 learned_addr learned my addr 172.21.15.3:0/3001360995 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4063e00640 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4063400640 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582ea1e0 0x5620582f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 -- 172.21.15.3:0/3001360995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582ea1e0 msgr2=0x5620582f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582ea1e0 0x5620582f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 -- 172.21.15.3:0/3001360995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 msgr2=0x5620582ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582ff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.672+0000 7f4062a00640 1 -- 172.21.15.3:0/3001360995 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620583062b0 con 0x5620582f38d0 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.673+0000 7f4062a00640 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582f38d0 0x5620582fec10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f40500c6220 tx=0x7f4050005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528e722a419a1b11 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.673+0000 7f404fe00640 1 -- 172.21.15.3:0/3001360995 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40500bdd40 con 0x5620582f38d0 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.673+0000 7f404fe00640 1 -- 172.21.15.3:0/3001360995 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40500c6750 con 0x5620582f38d0 2024-03-21T08:32:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.673+0000 7f404fe00640 1 -- 172.21.15.3:0/3001360995 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40500bd6a0 con 0x5620582f38d0 2024-03-21T08:32:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.674+0000 7f406d3f4740 1 -- 172.21.15.3:0/3001360995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582f38d0 msgr2=0x5620582fec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.674+0000 7f406d3f4740 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582f38d0 0x5620582fec10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f40500c6220 tx=0x7f4050005fd0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.674+0000 7f406d3f4740 1 -- 172.21.15.3:0/3001360995 shutdown_connections 2024-03-21T08:32:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.674+0000 7f406d3f4740 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.674+0000 7f406d3f4740 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582f38d0 0x5620582fec10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.674+0000 7f406d3f4740 1 --2- 172.21.15.3:0/3001360995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582ea1e0 0x5620582f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.674+0000 7f406d3f4740 1 -- 172.21.15.3:0/3001360995 >> 172.21.15.3:0/3001360995 conn(0x5620582e4b00 msgr2=0x5620582ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.675+0000 7f406d3f4740 1 -- 172.21.15.3:0/3001360995 shutdown_connections 2024-03-21T08:32:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.675+0000 7f406d3f4740 1 -- 172.21.15.3:0/3001360995 wait complete. 2024-03-21T08:32:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.675+0000 7f406d3f4740 1 -- start start 2024-03-21T08:32:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.675+0000 7f406d3f4740 1 Processor -- start 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582ea1e0 0x5620582fb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f406d3f4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f71c0 0x562058366960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f406d3f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641af0f0 con 0x5620582ea1e0 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f406d3f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641b03f0 con 0x56205830b590 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f406d3f4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641af730 con 0x5620582f71c0 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f406d3f4740 10 client.0 _ll_register_callbacks cb 0x7f40500b9fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.676+0000 7f4063400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582ea1e0 0x5620582fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582ea1e0 0x5620582fb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39452/0 (socket says 172.21.15.3:39452) 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 -- 172.21.15.3:0/775978738 learned_addr learned my addr 172.21.15.3:0/775978738 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:32:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4062a00640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063e00640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f71c0 0x562058366960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 -- 172.21.15.3:0/775978738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f71c0 msgr2=0x562058366960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f71c0 0x562058366960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 -- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 msgr2=0x5620582f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 -- 172.21.15.3:0/775978738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620582f7900 con 0x5620582ea1e0 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4062a00640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f4063400640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582ea1e0 0x5620582fb620 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4054009900 tx=0x7f40540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5201a59f8c3bd952 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.677+0000 7f404d600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.678+0000 7f404d600640 1 -- 172.21.15.3:0/775978738 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405401d8d0 con 0x5620582ea1e0 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.678+0000 7f404d600640 1 -- 172.21.15.3:0/775978738 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f405401da70 con 0x5620582ea1e0 2024-03-21T08:32:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.678+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40500c6750 con 0x5620582ea1e0 2024-03-21T08:32:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.679+0000 7f404d600640 1 -- 172.21.15.3:0/775978738 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f405401dc70 con 0x5620582ea1e0 2024-03-21T08:32:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.679+0000 7f406d3f4740 10 client.16053 fetch_fsmap learned FSMap version 23 2024-03-21T08:32:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.679+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562058367f50 con 0x5620582ea1e0 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.679+0000 7f404d600640 1 -- 172.21.15.3:0/775978738 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40500c6750 con 0x5620582ea1e0 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.680+0000 7f404d600640 1 -- 172.21.15.3:0/775978738 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f405403f670 con 0x5620582ea1e0 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.680+0000 7f406d3f4740 10 client.16053 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.680+0000 7f406d3f4740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.680+0000 7f406d3f4740 20 client.16053 populate_metadata read hostname 'smithi003' 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.680+0000 7f406d3f4740 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5620583686b0 0x56205836aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.680+0000 7f406d3f4740 4 client.16053 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.680+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620582560c0 con 0x5620583686b0 2024-03-21T08:32:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.681+0000 7f4062a00640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5620583686b0 0x56205836aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:32:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.682+0000 7f4062a00640 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5620583686b0 0x56205836aad0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba57a7b6b835aae server_cookie=1f35398341d14c2d in_seq=0 out_seq=0 2024-03-21T08:32:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.682+0000 7f404d600640 10 client.16053 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f404d600640 1 -- 172.21.15.3:0/775978738 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620582560c0 con 0x5620583686b0 2024-03-21T08:32:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f404d600640 10 client.16053 handle_command_reply: tid=0 2024-03-21T08:32:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 client.16053 shutdown 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582ea1e0 msgr2=0x5620582fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582ea1e0 0x5620582fb620 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4054009900 tx=0x7f40540046c0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 shutdown_connections 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5620583686b0 0x56205836aad0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5620582f71c0 0x562058366960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56205830b590 0x5620582f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 --2- 172.21.15.3:0/775978738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620582ea1e0 0x5620582fb620 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 >> 172.21.15.3:0/775978738 conn(0x5620582e4b00 msgr2=0x5620582ef240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.691+0000 7f404d600640 0 client.16053 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.692+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 shutdown_connections 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.692+0000 7f406d3f4740 1 -- 172.21.15.3:0/775978738 wait complete. 2024-03-21T08:32:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.692+0000 7f406d3f4740 20 client.16053 trim_cache size 0 max 16384 2024-03-21T08:32:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:32:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:32:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:32:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:32:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f40480a3210 msgr2=0x7f40480a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f40480a3210 0x7f40480a56d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5620582e9c90 tx=0x7f4054005f90 comp rx=0 tx=0).stop 2024-03-21T08:32:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4064102430 msgr2=0x7f406419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:32:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4064102430 0x7f406419fdf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f405002c210 tx=0x7f405002db40 comp rx=0 tx=0).stop 2024-03-21T08:32:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 shutdown_connections 2024-03-21T08:32:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f40480a3210 0x7f40480a56d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406410dc90 0x7f4064199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4064102430 0x7f406419fdf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.693+0000 7f4069200640 1 --2- 172.21.15.3:0/237880966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40640693d0 0x7f406419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:32:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.694+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 >> 172.21.15.3:0/237880966 conn(0x7f4064075f00 msgr2=0x7f40640ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:32:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.694+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 shutdown_connections 2024-03-21T08:32:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:32:59.694+0000 7f4069200640 1 -- 172.21.15.3:0/237880966 wait complete. 2024-03-21T08:33:00.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:00 smithi084 ceph-mon[35477]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:33:00.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:00 smithi171 ceph-mon[35388]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:33:00.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:00 smithi003 ceph-mon[31675]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T08:33:01.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:01 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:33:01.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:01 smithi084 ceph-mon[35477]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-21T08:33:01.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:01 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:33:01.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:01 smithi171 ceph-mon[35388]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-21T08:33:01.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:01 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:33:01.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:01 smithi003 ceph-mon[31675]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-21T08:33:02.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:02 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:33:02.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:02 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:33:02.777 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:02 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:33:02.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:02 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:33:02.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:02 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:33:02.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:02 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:33:02.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:02 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:33:02.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:02 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:33:02.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:02 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:33:03.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:03 smithi084 ceph-mon[35477]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T08:33:03.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:03 smithi171 ceph-mon[35388]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T08:33:03.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:03 smithi003 ceph-mon[31675]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T08:33:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:05 smithi084 ceph-mon[35477]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:06.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:05 smithi171 ceph-mon[35388]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:06.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:05 smithi003 ceph-mon[31675]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:08.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:07 smithi084 ceph-mon[35477]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:08.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:07 smithi171 ceph-mon[35388]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:08.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:07 smithi003 ceph-mon[31675]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:10.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:09 smithi084 ceph-mon[35477]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T08:33:10.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:09 smithi171 ceph-mon[35388]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T08:33:10.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:09 smithi003 ceph-mon[31675]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T08:33:11.978 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:11 smithi003 ceph-mon[31675]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:12.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:11 smithi084 ceph-mon[35477]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:12.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:11 smithi171 ceph-mon[35388]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T08:33:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.826+0000 7f951ca00640 1 -- 172.21.15.3:0/899499527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9518069850 msgr2=0x7f951810e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 --2- 172.21.15.3:0/899499527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9518069850 0x7f951810e330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9510009ca0 tx=0x7f951002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 -- 172.21.15.3:0/899499527 shutdown_connections 2024-03-21T08:33:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 --2- 172.21.15.3:0/899499527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9518069850 0x7f951810e330 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 --2- 172.21.15.3:0/899499527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518068e90 0x7f9518069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 --2- 172.21.15.3:0/899499527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95181075b0 0x7f9518107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 -- 172.21.15.3:0/899499527 >> 172.21.15.3:0/899499527 conn(0x7f9518075f00 msgr2=0x7f9518076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 -- 172.21.15.3:0/899499527 shutdown_connections 2024-03-21T08:33:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.827+0000 7f951ca00640 1 -- 172.21.15.3:0/899499527 wait complete. 2024-03-21T08:33:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 Processor -- start 2024-03-21T08:33:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 -- start start 2024-03-21T08:33:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9518068e90 0x7f9518199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518069850 0x7f951819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95181075b0 0x7f951819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518111700 con 0x7f95181075b0 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518111580 con 0x7f9518068e90 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.828+0000 7f951ca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518111880 con 0x7f9518069850 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95181075b0 0x7f951819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95181075b0 0x7f951819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39938/0 (socket says 172.21.15.3:39938) 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 -- 172.21.15.3:0/2485733097 learned_addr learned my addr 172.21.15.3:0/2485733097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517400640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9518068e90 0x7f9518199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9516a00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518069850 0x7f951819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 -- 172.21.15.3:0/2485733097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518069850 msgr2=0x7f951819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518069850 0x7f951819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 -- 172.21.15.3:0/2485733097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9518068e90 msgr2=0x7f9518199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9518068e90 0x7f9518199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.829+0000 7f9517e00640 1 -- 172.21.15.3:0/2485733097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951819eda0 con 0x7f95181075b0 2024-03-21T08:33:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.830+0000 7f9517e00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95181075b0 0x7f951819e670 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9510037040 tx=0x7f9510035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.830+0000 7f950fe00640 1 -- 172.21.15.3:0/2485733097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510039040 con 0x7f95181075b0 2024-03-21T08:33:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.830+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951819f030 con 0x7f95181075b0 2024-03-21T08:33:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.830+0000 7f950fe00640 1 -- 172.21.15.3:0/2485733097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9510044070 con 0x7f95181075b0 2024-03-21T08:33:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.830+0000 7f950fe00640 1 -- 172.21.15.3:0/2485733097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510004330 con 0x7f95181075b0 2024-03-21T08:33:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.830+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95181a6920 con 0x7f95181075b0 2024-03-21T08:33:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.835+0000 7f9520e17740 1 Processor -- start 2024-03-21T08:33:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.835+0000 7f950fe00640 1 -- 172.21.15.3:0/2485733097 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f95100044d0 con 0x7f95181075b0 2024-03-21T08:33:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9520e17740 1 -- start start 2024-03-21T08:33:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9520e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89901e0 0x55efe89905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9520e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8999390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9520e17740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89998d0 0x55efe899bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9520e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe89523b0 con 0x55efe89901e0 2024-03-21T08:33:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9520e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe88f4830 con 0x55efe8990b00 2024-03-21T08:33:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9520e17740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe889d5d0 con 0x55efe89998d0 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9517400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8999390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9517400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8999390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:41318/0 (socket says 172.21.15.3:41318) 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.836+0000 7f9517400640 1 -- 172.21.15.3:0/1346721977 learned_addr learned my addr 172.21.15.3:0/1346721977 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f9516a00640 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89901e0 0x55efe89905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f9517400640 1 -- 172.21.15.3:0/1346721977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89998d0 msgr2=0x55efe899bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f9517400640 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89998d0 0x55efe899bcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f9517400640 1 -- 172.21.15.3:0/1346721977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89901e0 msgr2=0x55efe89905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f9517400640 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89901e0 0x55efe89905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f9517400640 1 -- 172.21.15.3:0/1346721977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efe88b52c0 con 0x55efe8990b00 2024-03-21T08:33:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f9517400640 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8999390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f950800bd10 tx=0x7f95080107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b001ca97cb4bb7f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.837+0000 7f950fe00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f94fc0a3190 0x7f94fc0a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.838+0000 7f9503e00640 1 -- 172.21.15.3:0/1346721977 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508004030 con 0x55efe8990b00 2024-03-21T08:33:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.838+0000 7f950fe00640 1 -- 172.21.15.3:0/2485733097 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f95100f21a0 con 0x7f95181075b0 2024-03-21T08:33:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.838+0000 7f9503e00640 1 -- 172.21.15.3:0/1346721977 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f95080035f0 con 0x55efe8990b00 2024-03-21T08:33:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.838+0000 7f9503e00640 1 -- 172.21.15.3:0/1346721977 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508011770 con 0x55efe8990b00 2024-03-21T08:33:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.838+0000 7f9516a00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f94fc0a3190 0x7f94fc0a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 -- 172.21.15.3:0/1346721977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 msgr2=0x55efe8999390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8999390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f950800bd10 tx=0x7f95080107a0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 -- 172.21.15.3:0/1346721977 shutdown_connections 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89998d0 0x55efe899bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8999390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 --2- 172.21.15.3:0/1346721977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89901e0 0x55efe89905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 -- 172.21.15.3:0/1346721977 >> 172.21.15.3:0/1346721977 conn(0x55efe898ab00 msgr2=0x55efe898aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 -- 172.21.15.3:0/1346721977 shutdown_connections 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9516a00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f94fc0a3190 0x7f94fc0a5650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9504005fd0 tx=0x7f9504005f40 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.839+0000 7f9520e17740 1 -- 172.21.15.3:0/1346721977 wait complete. 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 -- start start 2024-03-21T08:33:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 Processor -- start 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe8996990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8996ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe8997410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe88b0a20 con 0x55efe89998d0 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe88f08d0 con 0x55efe8990b00 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe899c500 con 0x55efe89901e0 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.840+0000 7f9520e17740 10 client.0 _ll_register_callbacks cb 0x55efe88ed1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe8997410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe8997410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39958/0 (socket says 172.21.15.3:39958) 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 -- 172.21.15.3:0/1775982912 learned_addr learned my addr 172.21.15.3:0/1775982912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9516a00640 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8996ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517400640 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe8996990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 -- 172.21.15.3:0/1775982912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 msgr2=0x55efe8996990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe8996990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 -- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 msgr2=0x55efe8996ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8996ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.841+0000 7f9517e00640 1 -- 172.21.15.3:0/1775982912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efe8609210 con 0x55efe89998d0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9517e00640 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe8997410 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9510005e30 tx=0x7f95100bdb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f4c4a10160fee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9500c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9500c00640 1 -- 172.21.15.3:0/1775982912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100be030 con 0x55efe89998d0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9500c00640 1 -- 172.21.15.3:0/1775982912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f95100de070 con 0x55efe89998d0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95080035f0 con 0x55efe89998d0 2024-03-21T08:33:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9500c00640 1 -- 172.21.15.3:0/1775982912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95100c2180 con 0x55efe89998d0 2024-03-21T08:33:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9520e17740 10 client.16065 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.842+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55efe86d5620 con 0x55efe89998d0 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9500c00640 1 -- 172.21.15.3:0/1775982912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f95080035f0 con 0x55efe89998d0 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9500c00640 1 -- 172.21.15.3:0/1775982912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f951003c030 con 0x55efe89998d0 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9520e17740 10 client.16065 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9520e17740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9520e17740 20 client.16065 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9520e17740 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe89c3160 0x55efe89c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9520e17740 4 client.16065 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.843+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efe8901d70 con 0x55efe89c3160 2024-03-21T08:33:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.844+0000 7f9517400640 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe89c3160 0x55efe89c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.844+0000 7f9517400640 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe89c3160 0x55efe89c5550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f69f8858ee019f server_cookie=2bb32442cd5732d8 in_seq=0 out_seq=0 2024-03-21T08:33:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.844+0000 7f9500c00640 10 client.16065 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.846+0000 7f9500c00640 1 -- 172.21.15.3:0/1775982912 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55efe8901d70 con 0x55efe89c3160 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.846+0000 7f9500c00640 10 client.16065 handle_command_reply: tid=0 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.846+0000 7f9520e17740 1 client.16065 shutdown 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.846+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 msgr2=0x55efe8997410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.846+0000 7f9520e17740 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe8997410 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9510005e30 tx=0x7f95100bdb70 comp rx=0 tx=0).stop 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 shutdown_connections 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe8997410 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe8990b00 0x55efe8996ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe89c3160 0x55efe89c5550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 --2- 172.21.15.3:0/1775982912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe8996990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 >> 172.21.15.3:0/1775982912 conn(0x55efe898ab00 msgr2=0x55efe898c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9500c00640 0 client.16065 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 shutdown_connections 2024-03-21T08:33:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 1 -- 172.21.15.3:0/1775982912 wait complete. 2024-03-21T08:33:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.847+0000 7f9520e17740 20 client.16065 trim_cache size 0 max 16384 2024-03-21T08:33:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 Processor -- start 2024-03-21T08:33:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 -- start start 2024-03-21T08:33:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89901e0 0x55efe89990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe8990b00 0x55efe89ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe89ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518073200 con 0x55efe89998d0 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518070d00 con 0x55efe89901e0 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9520e17740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951806edc0 con 0x55efe8990b00 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.862+0000 7f9517e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe89ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe89ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39962/0 (socket says 172.21.15.3:39962) 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 -- 172.21.15.3:0/3986070746 learned_addr learned my addr 172.21.15.3:0/3986070746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517400640 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89901e0 0x55efe89990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9516a00640 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe8990b00 0x55efe89ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 -- 172.21.15.3:0/3986070746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe8990b00 msgr2=0x55efe89ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe8990b00 0x55efe89ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 -- 172.21.15.3:0/3986070746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89901e0 msgr2=0x55efe89990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89901e0 0x55efe89990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 -- 172.21.15.3:0/3986070746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efe899c030 con 0x55efe89998d0 2024-03-21T08:33:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9517e00640 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe89ad9e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95100efad0 tx=0x7f9510046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d1073ffd242934 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.863+0000 7f9503400640 1 -- 172.21.15.3:0/3986070746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100bd990 con 0x55efe89998d0 2024-03-21T08:33:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.864+0000 7f9503400640 1 -- 172.21.15.3:0/3986070746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f95100be030 con 0x55efe89998d0 2024-03-21T08:33:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.864+0000 7f9503400640 1 -- 172.21.15.3:0/3986070746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100de0e0 con 0x55efe89998d0 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.864+0000 7f9520e17740 1 -- 172.21.15.3:0/3986070746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 msgr2=0x55efe89ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.864+0000 7f9520e17740 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe89ad9e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95100efad0 tx=0x7f9510046000 comp rx=0 tx=0).stop 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 -- 172.21.15.3:0/3986070746 shutdown_connections 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe89998d0 0x55efe89ad9e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe8990b00 0x55efe89ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 --2- 172.21.15.3:0/3986070746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89901e0 0x55efe89990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 -- 172.21.15.3:0/3986070746 >> 172.21.15.3:0/3986070746 conn(0x55efe898ab00 msgr2=0x55efe89856a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 -- 172.21.15.3:0/3986070746 shutdown_connections 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 -- 172.21.15.3:0/3986070746 wait complete. 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 -- start start 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.865+0000 7f9520e17740 1 Processor -- start 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9520e17740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe89a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9520e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe8990b00 0x55efe89a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9520e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89998d0 0x55efe8a1da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9520e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951806f3c0 con 0x55efe8990b00 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9520e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95180706a0 con 0x55efe89998d0 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9520e17740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951806f9e0 con 0x55efe89901e0 2024-03-21T08:33:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9516a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe8990b00 0x55efe89a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.866+0000 7f9520e17740 10 client.0 _ll_register_callbacks cb 0x7f9510035ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9516a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe8990b00 0x55efe89a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39970/0 (socket says 172.21.15.3:39970) 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9516a00640 1 -- 172.21.15.3:0/598469068 learned_addr learned my addr 172.21.15.3:0/598469068 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9517400640 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe89a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9517e00640 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89998d0 0x55efe8a1da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9516a00640 1 -- 172.21.15.3:0/598469068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 msgr2=0x55efe89a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9516a00640 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe89a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9516a00640 1 -- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89998d0 msgr2=0x55efe8a1da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9516a00640 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89998d0 0x55efe8a1da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.867+0000 7f9516a00640 1 -- 172.21.15.3:0/598469068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efe8a1df40 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9516a00640 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe8990b00 0x55efe89a5cf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f951810e7e0 tx=0x7f95040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457c4a1cafdb4f9c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9500c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9500c00640 1 -- 172.21.15.3:0/598469068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504004020 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9500c00640 1 -- 172.21.15.3:0/598469068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f95040041c0 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efe88fc0c0 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9500c00640 1 -- 172.21.15.3:0/598469068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95040043c0 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9520e17740 10 client.16071 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.868+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55efe8a1f120 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9500c00640 1 -- 172.21.15.3:0/598469068 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55efe88fc0c0 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9500c00640 1 -- 172.21.15.3:0/598469068 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9504028610 con 0x55efe8990b00 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9520e17740 10 client.16071 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9520e17740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9520e17740 20 client.16071 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9520e17740 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe8a1f900 0x55efe8a21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9520e17740 4 client.16071 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:33:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.869+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55efe887a5d0 con 0x55efe8a1f900 2024-03-21T08:33:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.870+0000 7f9517400640 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe8a1f900 0x55efe8a21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.870+0000 7f9517400640 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe8a1f900 0x55efe8a21d20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a5be6b805c5d14 server_cookie=47ca78a95c493ca in_seq=0 out_seq=0 2024-03-21T08:33:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.870+0000 7f9500c00640 10 client.16071 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9500c00640 1 -- 172.21.15.3:0/598469068 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55efe887a5d0 con 0x55efe8a1f900 2024-03-21T08:33:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9500c00640 10 client.16071 handle_command_reply: tid=0 2024-03-21T08:33:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 client.16071 shutdown 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe8990b00 msgr2=0x55efe89a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe8990b00 0x55efe89a5cf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f951810e7e0 tx=0x7f95040046c0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 shutdown_connections 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55efe89998d0 0x55efe8a1da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efe8990b00 0x55efe89a5cf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55efe8a1f900 0x55efe8a21d20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 --2- 172.21.15.3:0/598469068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efe89901e0 0x55efe89a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9500c00640 0 client.16071 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.883+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 >> 172.21.15.3:0/598469068 conn(0x55efe898ab00 msgr2=0x55efe898bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.884+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 shutdown_connections 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.884+0000 7f9520e17740 1 -- 172.21.15.3:0/598469068 wait complete. 2024-03-21T08:33:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.884+0000 7f9520e17740 20 client.16071 trim_cache size 0 max 16384 2024-03-21T08:33:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:33:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:33:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:33:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:33:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f94fc0a3190 msgr2=0x7f94fc0a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f94fc0a3190 0x7f94fc0a5650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9504005fd0 tx=0x7f9504005f40 comp rx=0 tx=0).stop 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95181075b0 msgr2=0x7f951819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95181075b0 0x7f951819e670 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9510037040 tx=0x7f9510035ce0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 shutdown_connections 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95181075b0 0x7f951819e670 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f94fc0a3190 0x7f94fc0a5650 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518069850 0x7f951819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 --2- 172.21.15.3:0/2485733097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9518068e90 0x7f9518199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 >> 172.21.15.3:0/2485733097 conn(0x7f9518075f00 msgr2=0x7f95180feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 shutdown_connections 2024-03-21T08:33:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:12.885+0000 7f951ca00640 1 -- 172.21.15.3:0/2485733097 wait complete. 2024-03-21T08:33:14.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:13 smithi084 ceph-mon[35477]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T08:33:14.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:13 smithi171 ceph-mon[35388]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T08:33:14.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:13 smithi003 ceph-mon[31675]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T08:33:16.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:15 smithi084 ceph-mon[35477]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T08:33:16.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:15 smithi171 ceph-mon[35388]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T08:33:16.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:15 smithi003 ceph-mon[31675]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T08:33:17.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:33:17.316 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:33:17.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.528+0000 7fc27e000640 1 -- 172.21.15.3:0/2491377616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2781014a0 msgr2=0x7fc278101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.528+0000 7fc27e000640 1 --2- 172.21.15.3:0/2491377616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2781014a0 0x7fc278101920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc264009900 tx=0x7fc26402bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.529+0000 7fc27e000640 1 -- 172.21.15.3:0/2491377616 shutdown_connections 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.529+0000 7fc27e000640 1 --2- 172.21.15.3:0/2491377616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc278101e60 0x7fc27810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.529+0000 7fc27e000640 1 --2- 172.21.15.3:0/2491377616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2781014a0 0x7fc278101920 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.529+0000 7fc27e000640 1 --2- 172.21.15.3:0/2491377616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc278107560 0x7fc278107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.529+0000 7fc27e000640 1 -- 172.21.15.3:0/2491377616 >> 172.21.15.3:0/2491377616 conn(0x7fc2780fcfb0 msgr2=0x7fc2780ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.529+0000 7fc27e000640 1 -- 172.21.15.3:0/2491377616 shutdown_connections 2024-03-21T08:33:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.529+0000 7fc27e000640 1 -- 172.21.15.3:0/2491377616 wait complete. 2024-03-21T08:33:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.530+0000 7fc27e000640 1 Processor -- start 2024-03-21T08:33:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.530+0000 7fc27e000640 1 -- start start 2024-03-21T08:33:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.530+0000 7fc27e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2781014a0 0x7fc278199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.530+0000 7fc27e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278101e60 0x7fc27819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc27e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc278107560 0x7fc27819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc27e000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278111740 con 0x7fc278101e60 2024-03-21T08:33:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc27e000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781115c0 con 0x7fc2781014a0 2024-03-21T08:33:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc27e000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781118c0 con 0x7fc278107560 2024-03-21T08:33:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc277e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278101e60 0x7fc27819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc277e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278101e60 0x7fc27819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39978/0 (socket says 172.21.15.3:39978) 2024-03-21T08:33:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc277e00640 1 -- 172.21.15.3:0/3583504378 learned_addr learned my addr 172.21.15.3:0/3583504378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc27cc00640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2781014a0 0x7fc278199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.531+0000 7fc27d600640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc278107560 0x7fc27819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.532+0000 7fc277e00640 1 -- 172.21.15.3:0/3583504378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc278107560 msgr2=0x7fc27819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.532+0000 7fc277e00640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc278107560 0x7fc27819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.532+0000 7fc277e00640 1 -- 172.21.15.3:0/3583504378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2781014a0 msgr2=0x7fc278199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.532+0000 7fc277e00640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2781014a0 0x7fc278199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.532+0000 7fc277e00640 1 -- 172.21.15.3:0/3583504378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27819ee40 con 0x7fc278101e60 2024-03-21T08:33:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.532+0000 7fc27cc00640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2781014a0 0x7fc278199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.532+0000 7fc277e00640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278101e60 0x7fc27819a390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc264000c00 tx=0x7fc26402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.533+0000 7fc275600640 1 -- 172.21.15.3:0/3583504378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264043020 con 0x7fc278101e60 2024-03-21T08:33:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.533+0000 7fc275600640 1 -- 172.21.15.3:0/3583504378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2640052b0 con 0x7fc278101e60 2024-03-21T08:33:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.533+0000 7fc275600640 1 -- 172.21.15.3:0/3583504378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264003c70 con 0x7fc278101e60 2024-03-21T08:33:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.533+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27819f0d0 con 0x7fc278101e60 2024-03-21T08:33:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.533+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2781a69c0 con 0x7fc278101e60 2024-03-21T08:33:17.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 Processor -- start 2024-03-21T08:33:17.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 -- start start 2024-03-21T08:33:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc275600640 1 -- 172.21.15.3:0/3583504378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7fc264046020 con 0x7fc278101e60 2024-03-21T08:33:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac1414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac1414b0f0 0x55ac1414b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac14154460 0x55ac14156850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac140af250 con 0x55ac1414bdd0 2024-03-21T08:33:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac14057a30 con 0x55ac14154460 2024-03-21T08:33:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.538+0000 7fc2823f5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac140ab2f0 con 0x55ac1414b0f0 2024-03-21T08:33:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.539+0000 7fc27d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac1414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.539+0000 7fc27cc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac1414b0f0 0x55ac1414b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.539+0000 7fc27d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac1414ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39990/0 (socket says 172.21.15.3:39990) 2024-03-21T08:33:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.539+0000 7fc27d600640 1 -- 172.21.15.3:0/3489137010 learned_addr learned my addr 172.21.15.3:0/3489137010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.539+0000 7fc277e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac14154460 0x55ac14156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.540+0000 7fc27d600640 1 -- 172.21.15.3:0/3489137010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac1414b0f0 msgr2=0x55ac1414b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.540+0000 7fc27d600640 1 --2- 172.21.15.3:0/3489137010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac1414b0f0 0x55ac1414b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.540+0000 7fc27d600640 1 -- 172.21.15.3:0/3489137010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac14154460 msgr2=0x55ac14156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.540+0000 7fc27d600640 1 --2- 172.21.15.3:0/3489137010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac14154460 0x55ac14156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.540+0000 7fc27d600640 1 -- 172.21.15.3:0/3489137010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac14042690 con 0x55ac1414bdd0 2024-03-21T08:33:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.540+0000 7fc27d600640 1 --2- 172.21.15.3:0/3489137010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac1414ab20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc270009bb0 tx=0x7fc27000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6d26b8665f64c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.540+0000 7fc275600640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc2600a3230 0x7fc2600a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.541+0000 7fc275600640 1 -- 172.21.15.3:0/3583504378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fc2640f20c0 con 0x7fc278101e60 2024-03-21T08:33:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.541+0000 7fc269c00640 1 -- 172.21.15.3:0/3489137010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270017b70 con 0x55ac1414bdd0 2024-03-21T08:33:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.541+0000 7fc269c00640 1 -- 172.21.15.3:0/3489137010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc270017d10 con 0x55ac1414bdd0 2024-03-21T08:33:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.541+0000 7fc269c00640 1 -- 172.21.15.3:0/3489137010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700058f0 con 0x55ac1414bdd0 2024-03-21T08:33:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.541+0000 7fc27cc00640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc2600a3230 0x7fc2600a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 -- 172.21.15.3:0/3489137010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 msgr2=0x55ac1414ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3489137010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac1414ab20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc270009bb0 tx=0x7fc27000e6c0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 -- 172.21.15.3:0/3489137010 shutdown_connections 2024-03-21T08:33:17.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3489137010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac14154460 0x55ac14156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3489137010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac1414b0f0 0x55ac1414b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3489137010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac1414ab20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 -- 172.21.15.3:0/3489137010 >> 172.21.15.3:0/3489137010 conn(0x55ac14112200 msgr2=0x55ac14145b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:17.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc27cc00640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc2600a3230 0x7fc2600a56f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55ac1414c320 tx=0x7fc26c00b5a0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 -- 172.21.15.3:0/3489137010 shutdown_connections 2024-03-21T08:33:17.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.542+0000 7fc2823f5740 1 -- 172.21.15.3:0/3489137010 wait complete. 2024-03-21T08:33:17.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.543+0000 7fc2823f5740 1 -- start start 2024-03-21T08:33:17.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.543+0000 7fc2823f5740 1 Processor -- start 2024-03-21T08:33:17.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac14153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac14150f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14151440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc2823f5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac141572d0 con 0x55ac1414b0f0 2024-03-21T08:33:17.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc2823f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1406ae80 con 0x55ac1414bdd0 2024-03-21T08:33:17.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc2823f5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac14157450 con 0x55ac14154460 2024-03-21T08:33:17.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc2823f5740 10 client.0 _ll_register_callbacks cb 0x55ac1407d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:17.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc27cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac14153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc27cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac14153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40000/0 (socket says 172.21.15.3:40000) 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.544+0000 7fc27cc00640 1 -- 172.21.15.3:0/485110829 learned_addr learned my addr 172.21.15.3:0/485110829 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc277e00640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac14150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27d600640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27cc00640 1 -- 172.21.15.3:0/485110829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 msgr2=0x55ac14151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27cc00640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27cc00640 1 -- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 msgr2=0x55ac14150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27cc00640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac14150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27cc00640 1 -- 172.21.15.3:0/485110829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac13e8d7d0 con 0x55ac1414b0f0 2024-03-21T08:33:17.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc277e00640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac14150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:17.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27d600640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:17.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc27cc00640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac14153b70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc26c010ae0 tx=0x7fc26c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7f868b08b6c78f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.545+0000 7fc25e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:17.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.546+0000 7fc25e000640 1 -- 172.21.15.3:0/485110829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00fb80 con 0x55ac1414b0f0 2024-03-21T08:33:17.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.546+0000 7fc25e000640 1 -- 172.21.15.3:0/485110829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fc26c00fd20 con 0x55ac1414b0f0 2024-03-21T08:33:17.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.546+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc270017d10 con 0x55ac1414b0f0 2024-03-21T08:33:17.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.546+0000 7fc25e000640 1 -- 172.21.15.3:0/485110829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc26c027400 con 0x55ac1414b0f0 2024-03-21T08:33:17.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.547+0000 7fc2823f5740 10 client.16083 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:17.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.547+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ac13dc33f0 con 0x55ac1414b0f0 2024-03-21T08:33:17.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.547+0000 7fc25e000640 1 -- 172.21.15.3:0/485110829 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc270017d10 con 0x55ac1414b0f0 2024-03-21T08:33:17.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.547+0000 7fc25e000640 1 -- 172.21.15.3:0/485110829 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc26c031680 con 0x55ac1414b0f0 2024-03-21T08:33:17.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.547+0000 7fc2823f5740 10 client.16083 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:17.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.547+0000 7fc2823f5740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:17.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.548+0000 7fc2823f5740 20 client.16083 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:17.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.548+0000 7fc2823f5740 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac14164430 0x55ac14166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.548+0000 7fc2823f5740 4 client.16083 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:17.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.548+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac140bc790 con 0x55ac14164430 2024-03-21T08:33:17.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.548+0000 7fc277e00640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac14164430 0x55ac14166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.549+0000 7fc25e000640 10 client.16083 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:17.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.549+0000 7fc277e00640 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac14164430 0x55ac14166820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e5e91aa2c11f9 server_cookie=231068bd8857e68b in_seq=0 out_seq=0 2024-03-21T08:33:17.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.551+0000 7fc25e000640 1 -- 172.21.15.3:0/485110829 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ac140bc790 con 0x55ac14164430 2024-03-21T08:33:17.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.551+0000 7fc25e000640 10 client.16083 handle_command_reply: tid=0 2024-03-21T08:33:17.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.551+0000 7fc2823f5740 1 client.16083 shutdown 2024-03-21T08:33:17.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 msgr2=0x55ac14153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac14153b70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc26c010ae0 tx=0x7fc26c0046c0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 shutdown_connections 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac14164430 0x55ac14166820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac14150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 --2- 172.21.15.3:0/485110829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac14153b70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 >> 172.21.15.3:0/485110829 conn(0x55ac14112200 msgr2=0x55ac14147890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc25e000640 0 client.16083 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:17.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 shutdown_connections 2024-03-21T08:33:17.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 1 -- 172.21.15.3:0/485110829 wait complete. 2024-03-21T08:33:17.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.552+0000 7fc2823f5740 20 client.16083 trim_cache size 0 max 16384 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 Processor -- start 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 -- start start 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414b0f0 0x55ac14153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac14153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14155850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781aed00 con 0x55ac1414bdd0 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781ac800 con 0x55ac1414b0f0 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.583+0000 7fc2823f5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781aa8e0 con 0x55ac14154460 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac14153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac14153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40008/0 (socket says 172.21.15.3:40008) 2024-03-21T08:33:17.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 -- 172.21.15.3:0/3582797645 learned_addr learned my addr 172.21.15.3:0/3582797645 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:17.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc27cc00640 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414b0f0 0x55ac14153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc27d600640 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 -- 172.21.15.3:0/3582797645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 msgr2=0x55ac14155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 -- 172.21.15.3:0/3582797645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414b0f0 msgr2=0x55ac14153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414b0f0 0x55ac14153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc277e00640 1 -- 172.21.15.3:0/3582797645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac13d830f0 con 0x55ac1414bdd0 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.584+0000 7fc27cc00640 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414b0f0 0x55ac14153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.585+0000 7fc277e00640 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac14153b70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc264005ab0 tx=0x7fc2640de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdf6c270cef9451 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.585+0000 7fc269200640 1 -- 172.21.15.3:0/3582797645 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2640bdd20 con 0x55ac1414bdd0 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.585+0000 7fc269200640 1 -- 172.21.15.3:0/3582797645 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2640035f0 con 0x55ac1414bdd0 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.585+0000 7fc269200640 1 -- 172.21.15.3:0/3582797645 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2640bd8b0 con 0x55ac1414bdd0 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 -- 172.21.15.3:0/3582797645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 msgr2=0x55ac14153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac14153b70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc264005ab0 tx=0x7fc2640de040 comp rx=0 tx=0).stop 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 -- 172.21.15.3:0/3582797645 shutdown_connections 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac14155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414bdd0 0x55ac14153b70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 --2- 172.21.15.3:0/3582797645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414b0f0 0x55ac14153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 -- 172.21.15.3:0/3582797645 >> 172.21.15.3:0/3582797645 conn(0x55ac14112200 msgr2=0x55ac14145670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 -- 172.21.15.3:0/3582797645 shutdown_connections 2024-03-21T08:33:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.586+0000 7fc2823f5740 1 -- 172.21.15.3:0/3582797645 wait complete. 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.587+0000 7fc2823f5740 1 -- start start 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.587+0000 7fc2823f5740 1 Processor -- start 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac1414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac1414e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc2823f5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac1415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc2823f5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781aaee0 con 0x55ac1414b0f0 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc2823f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781ac1a0 con 0x55ac1414bdd0 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc2823f5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781ab4e0 con 0x55ac14154460 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc2823f5740 10 client.0 _ll_register_callbacks cb 0x7fc26c00bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc27cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac1414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc27cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac1414e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40014/0 (socket says 172.21.15.3:40014) 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc27cc00640 1 -- 172.21.15.3:0/2859066898 learned_addr learned my addr 172.21.15.3:0/2859066898 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc27cc00640 1 -- 172.21.15.3:0/2859066898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 msgr2=0x55ac1415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc27cc00640 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac1415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc27cc00640 1 -- 172.21.15.3:0/2859066898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 msgr2=0x55ac1414e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.588+0000 7fc27cc00640 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac1414e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.589+0000 7fc27cc00640 1 -- 172.21.15.3:0/2859066898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac1415f6e0 con 0x55ac1414b0f0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.589+0000 7fc27cc00640 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac1414e1c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005b90 tx=0x7fc26c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e2441a0844eab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.589+0000 7fc25e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.589+0000 7fc25e000640 1 -- 172.21.15.3:0/2859066898 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c03f060 con 0x55ac1414b0f0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.589+0000 7fc25e000640 1 -- 172.21.15.3:0/2859066898 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fc26c00fb80 con 0x55ac1414b0f0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.589+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac140b6ae0 con 0x55ac1414b0f0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc25e000640 1 -- 172.21.15.3:0/2859066898 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc26c046050 con 0x55ac1414b0f0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 10 client.16089 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ac141c0380 con 0x55ac1414b0f0 2024-03-21T08:33:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc25e000640 1 -- 172.21.15.3:0/2859066898 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55ac140b6ae0 con 0x55ac1414b0f0 2024-03-21T08:33:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc25e000640 1 -- 172.21.15.3:0/2859066898 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc26c004730 con 0x55ac1414b0f0 2024-03-21T08:33:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 10 client.16089 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 20 client.16089 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac141c0ab0 0x55ac141c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 4 client.16089 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.590+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac140343a0 con 0x55ac141c0ab0 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.591+0000 7fc277e00640 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac141c0ab0 0x55ac141c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.592+0000 7fc277e00640 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac141c0ab0 0x55ac141c2f10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e274c9973ba36d server_cookie=425a346d56e8392a in_seq=0 out_seq=0 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.592+0000 7fc25e000640 10 client.16089 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc25e000640 1 -- 172.21.15.3:0/2859066898 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ac140343a0 con 0x55ac141c0ab0 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc25e000640 10 client.16089 handle_command_reply: tid=0 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 client.16089 shutdown 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 msgr2=0x55ac1414e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac1414e1c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005b90 tx=0x7fc26c005bc0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 shutdown_connections 2024-03-21T08:33:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ac141c0ab0 0x55ac141c2f10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac14154460 0x55ac1415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac1414bdd0 0x55ac1414e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 --2- 172.21.15.3:0/2859066898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ac1414b0f0 0x55ac1414e1c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc25e000640 0 client.16089 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 >> 172.21.15.3:0/2859066898 conn(0x55ac14112200 msgr2=0x55ac1414f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.593+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 shutdown_connections 2024-03-21T08:33:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.594+0000 7fc2823f5740 1 -- 172.21.15.3:0/2859066898 wait complete. 2024-03-21T08:33:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.594+0000 7fc2823f5740 20 client.16089 trim_cache size 0 max 16384 2024-03-21T08:33:17.597 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:33:17.598 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-03-21T08:33:17.598 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-03-21T08:33:17.598 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:33:17.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.594+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc2600a3230 msgr2=0x7fc2600a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.594+0000 7fc27e000640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc2600a3230 0x7fc2600a56f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55ac1414c320 tx=0x7fc26c00b5a0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.594+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278101e60 msgr2=0x7fc27819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.594+0000 7fc27e000640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278101e60 0x7fc27819a390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc264000c00 tx=0x7fc26402c800 comp rx=0 tx=0).stop 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 shutdown_connections 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fc2600a3230 0x7fc2600a56f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc278107560 0x7fc27819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc278101e60 0x7fc27819a390 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 --2- 172.21.15.3:0/3583504378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2781014a0 0x7fc278199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 >> 172.21.15.3:0/3583504378 conn(0x7fc2780fcfb0 msgr2=0x7fc2780fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 shutdown_connections 2024-03-21T08:33:17.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.595+0000 7fc27e000640 1 -- 172.21.15.3:0/3583504378 wait complete. 2024-03-21T08:33:17.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T08:33:17.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:33:17.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 -- 172.21.15.3:0/3229905604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 msgr2=0x7f2de810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3229905604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de810d890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009900 tx=0x7f2dd402bcd0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 -- 172.21.15.3:0/3229905604 shutdown_connections 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3229905604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2de810ddd0 0x7f2de81101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3229905604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de810d890 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3229905604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de8073a70 0x7f2de8073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 -- 172.21.15.3:0/3229905604 >> 172.21.15.3:0/3229905604 conn(0x7f2de80fd060 msgr2=0x7f2de80ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 -- 172.21.15.3:0/3229905604 shutdown_connections 2024-03-21T08:33:17.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.794+0000 7f2de7e00640 1 -- 172.21.15.3:0/3229905604 wait complete. 2024-03-21T08:33:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 Processor -- start 2024-03-21T08:33:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 -- start start 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2de8073a70 0x7f2de819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de810ddd0 0x7f2de819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8113a50 con 0x7f2de8074390 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de81138d0 con 0x7f2de810ddd0 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.796+0000 7f2de7e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8113bd0 con 0x7f2de8073a70 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.797+0000 7f2de6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.797+0000 7f2de6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40036/0 (socket says 172.21.15.3:40036) 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.797+0000 7f2de6000640 1 -- 172.21.15.3:0/3275172031 learned_addr learned my addr 172.21.15.3:0/3275172031 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:17.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.797+0000 7f2de6a00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2de8073a70 0x7f2de819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.797+0000 7f2de7400640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de810ddd0 0x7f2de819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2de6000640 1 -- 172.21.15.3:0/3275172031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2de8073a70 msgr2=0x7f2de819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2de6000640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2de8073a70 0x7f2de819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2de6000640 1 -- 172.21.15.3:0/3275172031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de810ddd0 msgr2=0x7f2de819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2de6000640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de810ddd0 0x7f2de819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2de6000640 1 -- 172.21.15.3:0/3275172031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de819efb0 con 0x7f2de8074390 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2de6000640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de819a550 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4000c00 tx=0x7f2dd402d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2deca00640 1 -- 172.21.15.3:0/3275172031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4043020 con 0x7f2de8074390 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2deca00640 1 -- 172.21.15.3:0/3275172031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2dd402dd50 con 0x7f2de8074390 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de819f240 con 0x7f2de8074390 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.798+0000 7f2deca00640 1 -- 172.21.15.3:0/3275172031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4003ab0 con 0x7f2de8074390 2024-03-21T08:33:17.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.799+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de81a6b30 con 0x7f2de8074390 2024-03-21T08:33:17.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.800+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de804f470 con 0x7f2de8074390 2024-03-21T08:33:17.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.803+0000 7f2deca00640 1 -- 172.21.15.3:0/3275172031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f2dd4046020 con 0x7f2de8074390 2024-03-21T08:33:17.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.807+0000 7f2deca00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2dcc0a3220 0x7f2dcc0a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:17.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.807+0000 7f2deca00640 1 -- 172.21.15.3:0/3275172031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2dd40f1e50 con 0x7f2de8074390 2024-03-21T08:33:17.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.807+0000 7f2de6a00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2dcc0a3220 0x7f2dcc0a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:17.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.808+0000 7f2de6a00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2dcc0a3220 0x7f2dcc0a56e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8005d00 tx=0x7f2dd800b410 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:17.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:17.814+0000 7f2deca00640 1 -- 172.21.15.3:0/3275172031 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2dd404d020 con 0x7f2de8074390 2024-03-21T08:33:18.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.066+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2de8108f50 con 0x7f2de8074390 2024-03-21T08:33:18.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.068+0000 7f2deca00640 1 -- 172.21.15.3:0/3275172031 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f2dd40be380 con 0x7f2de8074390 2024-03-21T08:33:18.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:33:18.070 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:31:29.351404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34472,34409]},"id":1}]} 2024-03-21T08:33:18.070 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.070+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2dcc0a3220 msgr2=0x7f2dcc0a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.070+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2dcc0a3220 0x7f2dcc0a56e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8005d00 tx=0x7f2dd800b410 comp rx=0 tx=0).stop 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 msgr2=0x7f2de819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de819a550 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4000c00 tx=0x7f2dd402d890 comp rx=0 tx=0).stop 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 shutdown_connections 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2de810ddd0 0x7f2de819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2de8074390 0x7f2de819a550 secure :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4000c00 tx=0x7f2dd402d890 comp rx=0 tx=0).stop 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2dcc0a3220 0x7f2dcc0a56e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 --2- 172.21.15.3:0/3275172031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2de8073a70 0x7f2de819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 >> 172.21.15.3:0/3275172031 conn(0x7f2de80fd060 msgr2=0x7f2de80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 shutdown_connections 2024-03-21T08:33:18.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.071+0000 7f2de7e00640 1 -- 172.21.15.3:0/3275172031 wait complete. 2024-03-21T08:33:18.098 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:17 smithi003 ceph-mon[31675]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T08:33:18.100 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:33:18.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T08:33:18.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:17 smithi084 ceph-mon[35477]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T08:33:18.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:17 smithi171 ceph-mon[35388]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T08:33:18.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.286+0000 7f45e7e00640 1 -- 172.21.15.3:0/4080524301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e81124b0 msgr2=0x7f45e81148a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.286+0000 7f45e7e00640 1 --2- 172.21.15.3:0/4080524301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e81124b0 0x7f45e81148a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009ca0 tx=0x7f45e002c190 comp rx=0 tx=0).stop 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.286+0000 7f45e7e00640 1 -- 172.21.15.3:0/4080524301 shutdown_connections 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.286+0000 7f45e7e00640 1 --2- 172.21.15.3:0/4080524301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e81124b0 0x7f45e81148a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.286+0000 7f45e7e00640 1 --2- 172.21.15.3:0/4080524301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45e8074360 0x7f45e8111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.286+0000 7f45e7e00640 1 --2- 172.21.15.3:0/4080524301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8073a40 0x7f45e8073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.286+0000 7f45e7e00640 1 -- 172.21.15.3:0/4080524301 >> 172.21.15.3:0/4080524301 conn(0x7f45e80fcfb0 msgr2=0x7f45e80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.287+0000 7f45e7e00640 1 -- 172.21.15.3:0/4080524301 shutdown_connections 2024-03-21T08:33:18.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.287+0000 7f45e7e00640 1 -- 172.21.15.3:0/4080524301 wait complete. 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.287+0000 7f45e7e00640 1 Processor -- start 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e7e00640 1 -- start start 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e8073a40 0x7f45e819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8074360 0x7f45e819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45e81124b0 0x7f45e81a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8117d60 con 0x7f45e8074360 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8117be0 con 0x7f45e8073a40 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e7e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8117ee0 con 0x7f45e81124b0 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8074360 0x7f45e819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8074360 0x7f45e819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40048/0 (socket says 172.21.15.3:40048) 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e6000640 1 -- 172.21.15.3:0/2489677559 learned_addr learned my addr 172.21.15.3:0/2489677559 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.288+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e8073a40 0x7f45e819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.289+0000 7f45e7400640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45e81124b0 0x7f45e81a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.289+0000 7f45e6000640 1 -- 172.21.15.3:0/2489677559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45e81124b0 msgr2=0x7f45e81a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.289+0000 7f45e6000640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45e81124b0 0x7f45e81a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.289+0000 7f45e6000640 1 -- 172.21.15.3:0/2489677559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e8073a40 msgr2=0x7f45e819e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.289+0000 7f45e6000640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e8073a40 0x7f45e819e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.289+0000 7f45e6000640 1 -- 172.21.15.3:0/2489677559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e81a32a0 con 0x7f45e8074360 2024-03-21T08:33:18.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.289+0000 7f45e6000640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8074360 0x7f45e819e840 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45d400e9f0 tx=0x7f45d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.290+0000 7f45df400640 1 -- 172.21.15.3:0/2489677559 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d400cd80 con 0x7f45e8074360 2024-03-21T08:33:18.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.290+0000 7f45df400640 1 -- 172.21.15.3:0/2489677559 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45d40108e0 con 0x7f45e8074360 2024-03-21T08:33:18.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.290+0000 7f45df400640 1 -- 172.21.15.3:0/2489677559 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4018920 con 0x7f45e8074360 2024-03-21T08:33:18.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.290+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e81a3590 con 0x7f45e8074360 2024-03-21T08:33:18.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.290+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e81aae20 con 0x7f45e8074360 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.293+0000 7f45f0637740 1 Processor -- start 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.293+0000 7f45f0637740 1 -- start start 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45f0637740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ecbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45f0637740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecc0c0 0x555e48ee1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45f0637740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ee1e20 0x555e48ee22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45f0637740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e48e2f2c0 con 0x555e48ee1e20 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45f0637740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e48dd7c90 con 0x555e48ecc0c0 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45f0637740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e48e2b360 con 0x555e48ecb710 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecc0c0 0x555e48ee1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45e7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ecbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecc0c0 0x555e48ee1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:41436/0 (socket says 172.21.15.3:41436) 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.294+0000 7f45e6a00640 1 -- 172.21.15.3:0/2999080018 learned_addr learned my addr 172.21.15.3:0/2999080018 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.295+0000 7f45e7400640 1 -- 172.21.15.3:0/2999080018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecc0c0 msgr2=0x555e48ee1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.295+0000 7f45e7400640 1 --2- 172.21.15.3:0/2999080018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecc0c0 0x555e48ee1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.295+0000 7f45e7400640 1 -- 172.21.15.3:0/2999080018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ee1e20 msgr2=0x555e48ee22a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.295+0000 7f45e7400640 1 --2- 172.21.15.3:0/2999080018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ee1e20 0x555e48ee22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.295+0000 7f45e7400640 1 -- 172.21.15.3:0/2999080018 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e48dfc080 con 0x555e48ecb710 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.296+0000 7f45e7400640 1 --2- 172.21.15.3:0/2999080018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ecbaf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45e002c6d0 tx=0x7f45e00093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d439d16e6ee57f9f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.296+0000 7f45d3e00640 1 -- 172.21.15.3:0/2999080018 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0043020 con 0x555e48ecb710 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.296+0000 7f45d3e00640 1 -- 172.21.15.3:0/2999080018 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45e0003e20 con 0x555e48ecb710 2024-03-21T08:33:18.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.296+0000 7f45d3e00640 1 -- 172.21.15.3:0/2999080018 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0039040 con 0x555e48ecb710 2024-03-21T08:33:18.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45df400640 1 -- 172.21.15.3:0/2489677559 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f45d4010a90 con 0x7f45e8074360 2024-03-21T08:33:18.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 -- 172.21.15.3:0/2999080018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 msgr2=0x555e48ecbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 --2- 172.21.15.3:0/2999080018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ecbaf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45e002c6d0 tx=0x7f45e00093f0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 -- 172.21.15.3:0/2999080018 shutdown_connections 2024-03-21T08:33:18.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 --2- 172.21.15.3:0/2999080018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ee1e20 0x555e48ee22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 --2- 172.21.15.3:0/2999080018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecc0c0 0x555e48ee1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 --2- 172.21.15.3:0/2999080018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ecbaf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 -- 172.21.15.3:0/2999080018 >> 172.21.15.3:0/2999080018 conn(0x555e48ec5f80 msgr2=0x555e48ec6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 -- 172.21.15.3:0/2999080018 shutdown_connections 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.297+0000 7f45f0637740 1 -- 172.21.15.3:0/2999080018 wait complete. 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 -- start start 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 Processor -- start 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ed2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ed31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48ece0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e48ecc990 con 0x555e48ecc0c0 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e48deb0d0 con 0x555e48ee1e20 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.298+0000 7f45f0637740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e48eccb10 con 0x555e48ecb710 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.299+0000 7f45f0637740 10 client.0 _ll_register_callbacks cb 0x555e48defe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.299+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ed31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.299+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ed31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40062/0 (socket says 172.21.15.3:40062) 2024-03-21T08:33:18.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.299+0000 7f45e6a00640 1 -- 172.21.15.3:0/4247545516 learned_addr learned my addr 172.21.15.3:0/4247545516 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.299+0000 7f45e6000640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48ece0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.299+0000 7f45e7400640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ed2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45e7400640 1 -- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 msgr2=0x555e48ece0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45e7400640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48ece0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45e7400640 1 -- 172.21.15.3:0/4247545516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 msgr2=0x555e48ed31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45e7400640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ed31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45e7400640 1 -- 172.21.15.3:0/4247545516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e48b433f0 con 0x555e48ecb710 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45e6000640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48ece0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45e7400640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ed2ca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009bf0 tx=0x7f45e0009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b5007f67946cbb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.300+0000 7f45d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.301+0000 7f45d0c00640 1 -- 172.21.15.3:0/4247545516 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e004e020 con 0x555e48ecb710 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.301+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45e0003e20 con 0x555e48ecb710 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.301+0000 7f45d0c00640 1 -- 172.21.15.3:0/4247545516 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f45e0004200 con 0x555e48ecb710 2024-03-21T08:33:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.301+0000 7f45d0c00640 1 -- 172.21.15.3:0/4247545516 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e0059050 con 0x555e48ecb710 2024-03-21T08:33:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.301+0000 7f45f0637740 10 client.25561 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.301+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555e48c0d7d0 con 0x555e48ecb710 2024-03-21T08:33:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45d0c00640 1 -- 172.21.15.3:0/4247545516 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45e0003e20 con 0x555e48ecb710 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45df400640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f45cc0a3290 0x7f45cc0a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45d0c00640 1 -- 172.21.15.3:0/4247545516 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f45e003f470 con 0x555e48ecb710 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45df400640 1 -- 172.21.15.3:0/2489677559 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f45d40d05e0 con 0x7f45e8074360 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45f0637740 10 client.25561 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45f0637740 10 client.25561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45f0637740 20 client.25561 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45f0637740 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48eddd70 0x555e48ee0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45f0637740 4 client.25561 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f45cc0a3290 0x7f45cc0a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.302+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e48e3c800 con 0x555e48eddd70 2024-03-21T08:33:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.303+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f45cc0a3290 0x7f45cc0a5750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x555e48ecf670 tx=0x7f45d8036040 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.303+0000 7f45e6000640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48eddd70 0x555e48ee0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.304+0000 7f45e6000640 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48eddd70 0x555e48ee0180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a260236e223b2b server_cookie=30947239015ce49b in_seq=0 out_seq=0 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.304+0000 7f45d0c00640 10 client.25561 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45d0c00640 1 -- 172.21.15.3:0/4247545516 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e48e3c800 con 0x555e48eddd70 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45d0c00640 10 client.25561 handle_command_reply: tid=0 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 client.25561 shutdown 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 msgr2=0x555e48ed2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ed2ca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009bf0 tx=0x7f45e0009800 comp rx=0 tx=0).stop 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 shutdown_connections 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48eddd70 0x555e48ee0180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48ece0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ed31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 --2- 172.21.15.3:0/4247545516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecb710 0x555e48ed2ca0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 >> 172.21.15.3:0/4247545516 conn(0x555e48ec5f80 msgr2=0x555e48ec7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45d0c00640 0 client.25561 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.306+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 shutdown_connections 2024-03-21T08:33:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.307+0000 7f45f0637740 1 -- 172.21.15.3:0/4247545516 wait complete. 2024-03-21T08:33:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.307+0000 7f45f0637740 20 client.25561 trim_cache size 0 max 16384 2024-03-21T08:33:18.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 Processor -- start 2024-03-21T08:33:18.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 -- start start 2024-03-21T08:33:18.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecb710 0x555e48ec53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ee1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ee1e20 0x555e48ee16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81aed40 con 0x555e48ecc0c0 2024-03-21T08:33:18.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81ac840 con 0x555e48ecb710 2024-03-21T08:33:18.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.320+0000 7f45f0637740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e810d820 con 0x555e48ee1e20 2024-03-21T08:33:18.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecb710 0x555e48ec53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecb710 0x555e48ec53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:41464/0 (socket says 172.21.15.3:41464) 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 -- 172.21.15.3:0/2288280207 learned_addr learned my addr 172.21.15.3:0/2288280207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6000640 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ee1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e7400640 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ee1e20 0x555e48ee16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 -- 172.21.15.3:0/2288280207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ee1e20 msgr2=0x555e48ee16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ee1e20 0x555e48ee16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 -- 172.21.15.3:0/2288280207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 msgr2=0x555e48ee1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ee1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6a00640 1 -- 172.21.15.3:0/2288280207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e48ec18b0 con 0x555e48ecb710 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.321+0000 7f45e6000640 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ee1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:18.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.322+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecb710 0x555e48ec53a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009d30 tx=0x7f45d804d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee0156af7bd496f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.322+0000 7f45d3400640 1 -- 172.21.15.3:0/2288280207 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8061070 con 0x555e48ecb710 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45d3400640 1 -- 172.21.15.3:0/2288280207 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45d8058040 con 0x555e48ecb710 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45d3400640 1 -- 172.21.15.3:0/2288280207 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8051660 con 0x555e48ecb710 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45f0637740 1 -- 172.21.15.3:0/2288280207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecb710 msgr2=0x555e48ec53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45f0637740 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecb710 0x555e48ec53a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009d30 tx=0x7f45d804d000 comp rx=0 tx=0).stop 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45f0637740 1 -- 172.21.15.3:0/2288280207 shutdown_connections 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45f0637740 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ee1e20 0x555e48ee16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45f0637740 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecc0c0 0x555e48ee1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45f0637740 1 --2- 172.21.15.3:0/2288280207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ecb710 0x555e48ec53a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.323+0000 7f45f0637740 1 -- 172.21.15.3:0/2288280207 >> 172.21.15.3:0/2288280207 conn(0x555e48ec5f80 msgr2=0x555e48eda080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.324+0000 7f45f0637740 1 -- 172.21.15.3:0/2288280207 shutdown_connections 2024-03-21T08:33:18.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.324+0000 7f45f0637740 1 -- 172.21.15.3:0/2288280207 wait complete. 2024-03-21T08:33:18.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.324+0000 7f45f0637740 1 -- start start 2024-03-21T08:33:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.324+0000 7f45f0637740 1 Processor -- start 2024-03-21T08:33:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45f0637740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecb710 0x555e48eec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45f0637740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecc0c0 0x555e48eec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45f0637740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48eecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45f0637740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81ab050 con 0x555e48ecb710 2024-03-21T08:33:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45f0637740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81ac1e0 con 0x555e48ee1e20 2024-03-21T08:33:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45f0637740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81ab570 con 0x555e48ecc0c0 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45f0637740 10 client.0 _ll_register_callbacks cb 0x7f45d8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecb710 0x555e48eec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45e6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecb710 0x555e48eec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40076/0 (socket says 172.21.15.3:40076) 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45e6a00640 1 -- 172.21.15.3:0/2668897528 learned_addr learned my addr 172.21.15.3:0/2668897528 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45e7400640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48eecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.325+0000 7f45e6000640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecc0c0 0x555e48eec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45e6a00640 1 -- 172.21.15.3:0/2668897528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecc0c0 msgr2=0x555e48eec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecc0c0 0x555e48eec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45e6a00640 1 -- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 msgr2=0x555e48eecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48eecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45e6a00640 1 -- 172.21.15.3:0/2668897528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e48f3f8e0 con 0x555e48ecb710 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45e7400640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48eecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45e6a00640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecb710 0x555e48eec100 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009df0 tx=0x7f45d8005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6122c89760b4ec87 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.326+0000 7f45d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.327+0000 7f45d0c00640 1 -- 172.21.15.3:0/2668897528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8061070 con 0x555e48ecb710 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.327+0000 7f45d0c00640 1 -- 172.21.15.3:0/2668897528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f45d805c430 con 0x555e48ecb710 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.327+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45d8058040 con 0x555e48ecb710 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.327+0000 7f45d0c00640 1 -- 172.21.15.3:0/2668897528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45d805c630 con 0x555e48ecb710 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.327+0000 7f45f0637740 10 client.16110 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.327+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555e48f40bc0 con 0x555e48ecb710 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.327+0000 7f45d0c00640 1 -- 172.21.15.3:0/2668897528 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45d8058040 con 0x555e48ecb710 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.328+0000 7f45d0c00640 1 -- 172.21.15.3:0/2668897528 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f45d806f030 con 0x555e48ecb710 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.328+0000 7f45f0637740 10 client.16110 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.328+0000 7f45f0637740 10 client.16110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.328+0000 7f45f0637740 20 client.16110 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.328+0000 7f45f0637740 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48f411d0 0x555e48f435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.328+0000 7f45f0637740 4 client.16110 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:33:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.328+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e48e36b50 con 0x555e48f411d0 2024-03-21T08:33:18.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.329+0000 7f45e6000640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48f411d0 0x555e48f435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.329+0000 7f45e6000640 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48f411d0 0x555e48f435c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e377abf57d9a10ed server_cookie=36bed6a97b0384f3 in_seq=0 out_seq=0 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.329+0000 7f45d0c00640 10 client.16110 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45d0c00640 1 -- 172.21.15.3:0/2668897528 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e48e36b50 con 0x555e48f411d0 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45d0c00640 10 client.16110 handle_command_reply: tid=0 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 client.16110 shutdown 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecb710 msgr2=0x555e48eec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecb710 0x555e48eec100 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009df0 tx=0x7f45d8005f80 comp rx=0 tx=0).stop 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 shutdown_connections 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x555e48f411d0 0x555e48f435c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e48ee1e20 0x555e48eecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555e48ecc0c0 0x555e48eec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 --2- 172.21.15.3:0/2668897528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555e48ecb710 0x555e48eec100 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 >> 172.21.15.3:0/2668897528 conn(0x555e48ec5f80 msgr2=0x555e48ee4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45d0c00640 0 client.16110 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 shutdown_connections 2024-03-21T08:33:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 1 -- 172.21.15.3:0/2668897528 wait complete. 2024-03-21T08:33:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.330+0000 7f45f0637740 20 client.16110 trim_cache size 0 max 16384 2024-03-21T08:33:18.335 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:33:18.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f45cc0a3290 msgr2=0x7f45cc0a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f45cc0a3290 0x7f45cc0a5750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x555e48ecf670 tx=0x7f45d8036040 comp rx=0 tx=0).stop 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8074360 msgr2=0x7f45e819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8074360 0x7f45e819e840 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45d400e9f0 tx=0x7f45d400eeb0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 shutdown_connections 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f45cc0a3290 0x7f45cc0a5750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45e81124b0 0x7f45e81a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e8074360 0x7f45e819e840 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 --2- 172.21.15.3:0/2489677559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45e8073a40 0x7f45e819e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.331+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 >> 172.21.15.3:0/2489677559 conn(0x7f45e80fcfb0 msgr2=0x7f45e80fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.332+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 shutdown_connections 2024-03-21T08:33:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.332+0000 7f45e7e00640 1 -- 172.21.15.3:0/2489677559 wait complete. 2024-03-21T08:33:18.357 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:33:18.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T08:33:18.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 -- 172.21.15.3:0/3518887832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 msgr2=0x7fafb0107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3518887832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 0x7fafb0107940 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009900 tx=0x7fafa002bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:18.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 -- 172.21.15.3:0/3518887832 shutdown_connections 2024-03-21T08:33:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3518887832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 0x7fafb010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3518887832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafb01014a0 0x7fafb0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3518887832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 0x7fafb0107940 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 -- 172.21.15.3:0/3518887832 >> 172.21.15.3:0/3518887832 conn(0x7fafb00fcfb0 msgr2=0x7fafb00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 -- 172.21.15.3:0/3518887832 shutdown_connections 2024-03-21T08:33:18.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.536+0000 7fafb5a00640 1 -- 172.21.15.3:0/3518887832 wait complete. 2024-03-21T08:33:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 Processor -- start 2024-03-21T08:33:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 -- start start 2024-03-21T08:33:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafb01014a0 0x7fafb019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 0x7fafb019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 0x7fafb0195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb010f590 con 0x7fafb0107560 2024-03-21T08:33:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb010f410 con 0x7fafb0101e60 2024-03-21T08:33:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.538+0000 7fafb5a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb010f710 con 0x7fafb01014a0 2024-03-21T08:33:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.539+0000 7fafafe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafb01014a0 0x7fafb019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.539+0000 7fafaf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 0x7fafb019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.539+0000 7fafaf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 0x7fafb019bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:41494/0 (socket says 172.21.15.3:41494) 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.539+0000 7fafaf400640 1 -- 172.21.15.3:0/3720434077 learned_addr learned my addr 172.21.15.3:0/3720434077 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.539+0000 7fafb5000640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 0x7fafb0195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafaf400640 1 -- 172.21.15.3:0/3720434077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafb01014a0 msgr2=0x7fafb019b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafaf400640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafb01014a0 0x7fafb019b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafaf400640 1 -- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 msgr2=0x7fafb0195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafaf400640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 0x7fafb0195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafaf400640 1 -- 172.21.15.3:0/3720434077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0195ff0 con 0x7fafb0101e60 2024-03-21T08:33:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafb5000640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 0x7fafb0195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafaf400640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 0x7fafb019bc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d950 tx=0x7faf9c00dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafacc00640 1 -- 172.21.15.3:0/3720434077 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c017070 con 0x7fafb0101e60 2024-03-21T08:33:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb01962e0 con 0x7fafb0101e60 2024-03-21T08:33:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafacc00640 1 -- 172.21.15.3:0/3720434077 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf9c00de80 con 0x7fafb0101e60 2024-03-21T08:33:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.540+0000 7fafacc00640 1 -- 172.21.15.3:0/3720434077 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00f620 con 0x7fafb0101e60 2024-03-21T08:33:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.541+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb01a2580 con 0x7fafb0101e60 2024-03-21T08:33:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.544+0000 7fafb9dec740 1 Processor -- start 2024-03-21T08:33:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.544+0000 7fafb9dec740 1 -- start start 2024-03-21T08:33:18.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.544+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e52568b0 0x55c1e5256c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.544+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e525f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525fde0 0x55c1e5262230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafb9dec740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e51ba2c0 con 0x55c1e5257260 2024-03-21T08:33:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafb9dec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e5162c90 con 0x55c1e52568b0 2024-03-21T08:33:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafb9dec740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e51b6360 con 0x55c1e525fde0 2024-03-21T08:33:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafb5000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e52568b0 0x55c1e5256c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafafe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e525f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafafe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e525f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40106/0 (socket says 172.21.15.3:40106) 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.545+0000 7fafafe00640 1 -- 172.21.15.3:0/3096675568 learned_addr learned my addr 172.21.15.3:0/3096675568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafafe00640 1 -- 172.21.15.3:0/3096675568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525fde0 msgr2=0x55c1e5262230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafafe00640 1 --2- 172.21.15.3:0/3096675568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525fde0 0x55c1e5262230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafafe00640 1 -- 172.21.15.3:0/3096675568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e52568b0 msgr2=0x55c1e5256c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafafe00640 1 --2- 172.21.15.3:0/3096675568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e52568b0 0x55c1e5256c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafafe00640 1 -- 172.21.15.3:0/3096675568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e5187080 con 0x55c1e5257260 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafafe00640 1 --2- 172.21.15.3:0/3096675568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e525f810 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009230 tx=0x7fafa0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d895e9d65201ca28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafa5600640 1 -- 172.21.15.3:0/3096675568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0045020 con 0x55c1e5257260 2024-03-21T08:33:18.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafa5600640 1 -- 172.21.15.3:0/3096675568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fafa002c780 con 0x55c1e5257260 2024-03-21T08:33:18.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.546+0000 7fafa5600640 1 -- 172.21.15.3:0/3096675568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa00046c0 con 0x55c1e5257260 2024-03-21T08:33:18.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.547+0000 7fafb9dec740 1 -- 172.21.15.3:0/3096675568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 msgr2=0x55c1e525f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.547+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3096675568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e525f810 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009230 tx=0x7fafa0009560 comp rx=0 tx=0).stop 2024-03-21T08:33:18.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.547+0000 7fafb9dec740 1 -- 172.21.15.3:0/3096675568 shutdown_connections 2024-03-21T08:33:18.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.547+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3096675568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525fde0 0x55c1e5262230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.547+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3096675568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e525f810 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.547+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3096675568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e52568b0 0x55c1e5256c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.547+0000 7fafb9dec740 1 -- 172.21.15.3:0/3096675568 >> 172.21.15.3:0/3096675568 conn(0x55c1e5250f80 msgr2=0x55c1e5251360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.548+0000 7fafacc00640 1 -- 172.21.15.3:0/3720434077 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7faf9c009c30 con 0x7fafb0101e60 2024-03-21T08:33:18.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.548+0000 7fafb9dec740 1 -- 172.21.15.3:0/3096675568 shutdown_connections 2024-03-21T08:33:18.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.548+0000 7fafb9dec740 1 -- 172.21.15.3:0/3096675568 wait complete. 2024-03-21T08:33:18.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.549+0000 7fafb9dec740 1 -- start start 2024-03-21T08:33:18.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.549+0000 7fafb9dec740 1 Processor -- start 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 0x55c1e525cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e525d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e525d590 0x55c1e52749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafb9dec740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e5262cf0 con 0x55c1e525d590 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafb9dec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e51760d0 con 0x55c1e5257260 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafb9dec740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e5262e70 con 0x55c1e52568b0 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafb9dec740 10 client.0 _ll_register_callbacks cb 0x55c1e517ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.550+0000 7fafacc00640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faf940a3150 0x7faf940a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafacc00640 1 -- 172.21.15.3:0/3720434077 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7faf9c0cf9d0 con 0x7fafb0101e60 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafafe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e525d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafb5000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 0x55c1e525cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafafe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e525d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:41498/0 (socket says 172.21.15.3:41498) 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafafe00640 1 -- 172.21.15.3:0/3728191193 learned_addr learned my addr 172.21.15.3:0/3728191193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafaf400640 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e525d590 0x55c1e52749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafafe00640 1 -- 172.21.15.3:0/3728191193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 msgr2=0x55c1e525cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafb5000640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faf940a3150 0x7faf940a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafafe00640 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 0x55c1e525cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.551+0000 7fafafe00640 1 -- 172.21.15.3:0/3728191193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e525d590 msgr2=0x55c1e52749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.552+0000 7fafafe00640 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e525d590 0x55c1e52749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.552+0000 7fafafe00640 1 -- 172.21.15.3:0/3728191193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1e4ece3f0 con 0x55c1e5257260 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.552+0000 7fafaf400640 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e525d590 0x55c1e52749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.552+0000 7fafafe00640 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e525d050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafa0036040 tx=0x7fafa002cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad502d6a209f60c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.552+0000 7faf9a000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.553+0000 7faf9a000640 1 -- 172.21.15.3:0/3728191193 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0045040 con 0x55c1e5257260 2024-03-21T08:33:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.553+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafa002c780 con 0x55c1e5257260 2024-03-21T08:33:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.553+0000 7fafb5000640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faf940a3150 0x7faf940a5610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005fd0 tx=0x7fafa8013680 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.553+0000 7faf9a000640 1 -- 172.21.15.3:0/3728191193 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fafa00046c0 con 0x55c1e5257260 2024-03-21T08:33:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.553+0000 7faf9a000640 1 -- 172.21.15.3:0/3728191193 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafa005b050 con 0x55c1e5257260 2024-03-21T08:33:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.553+0000 7fafb9dec740 10 client.35858 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.553+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c1e4f987d0 con 0x55c1e5257260 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7faf9a000640 1 -- 172.21.15.3:0/3728191193 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fafa002c780 con 0x55c1e5257260 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7faf9a000640 1 -- 172.21.15.3:0/3728191193 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fafa0003710 con 0x55c1e5257260 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7fafb9dec740 10 client.35858 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7fafb9dec740 10 client.35858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7fafb9dec740 20 client.35858 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52787f0 0x55c1e527abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7fafb9dec740 4 client.35858 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:18.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.554+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1e51c7800 con 0x55c1e52787f0 2024-03-21T08:33:18.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.555+0000 7fafaf400640 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52787f0 0x55c1e527abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.555+0000 7fafaf400640 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52787f0 0x55c1e527abe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2629f1c9fedaecd6 server_cookie=cce6a889c2ee76e9 in_seq=0 out_seq=0 2024-03-21T08:33:18.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.556+0000 7faf9a000640 10 client.35858 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.557+0000 7faf9a000640 1 -- 172.21.15.3:0/3728191193 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c1e51c7800 con 0x55c1e52787f0 2024-03-21T08:33:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.557+0000 7faf9a000640 10 client.35858 handle_command_reply: tid=0 2024-03-21T08:33:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.557+0000 7fafb9dec740 1 client.35858 shutdown 2024-03-21T08:33:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.557+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 msgr2=0x55c1e525d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.557+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e525d050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafa0036040 tx=0x7fafa002cb60 comp rx=0 tx=0).stop 2024-03-21T08:33:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 shutdown_connections 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e525d590 0x55c1e52749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e525d050 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52787f0 0x55c1e527abe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3728191193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 0x55c1e525cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7faf9a000640 0 client.35858 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 >> 172.21.15.3:0/3728191193 conn(0x55c1e5250f80 msgr2=0x55c1e5252cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 shutdown_connections 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 1 -- 172.21.15.3:0/3728191193 wait complete. 2024-03-21T08:33:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.558+0000 7fafb9dec740 20 client.35858 trim_cache size 0 max 16384 2024-03-21T08:33:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.571+0000 7fafb9dec740 1 Processor -- start 2024-03-21T08:33:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.571+0000 7fafb9dec740 1 -- start start 2024-03-21T08:33:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e52568b0 0x55c1e52749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e5259d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525d590 0x55c1e525a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafb9dec740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb01aec90 con 0x55c1e52568b0 2024-03-21T08:33:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafb9dec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb01ac790 con 0x55c1e5257260 2024-03-21T08:33:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafb9dec740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb01aa830 con 0x55c1e525d590 2024-03-21T08:33:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafafe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e52568b0 0x55c1e52749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafafe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e52568b0 0x55c1e52749c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40122/0 (socket says 172.21.15.3:40122) 2024-03-21T08:33:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafafe00640 1 -- 172.21.15.3:0/3225016114 learned_addr learned my addr 172.21.15.3:0/3225016114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.572+0000 7fafaf400640 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e5259d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafafe00640 1 -- 172.21.15.3:0/3225016114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525d590 msgr2=0x55c1e525a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafafe00640 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525d590 0x55c1e525a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafafe00640 1 -- 172.21.15.3:0/3225016114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 msgr2=0x55c1e5259d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafafe00640 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e5259d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafafe00640 1 -- 172.21.15.3:0/3225016114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e5257000 con 0x55c1e52568b0 2024-03-21T08:33:18.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafaf400640 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e5259d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafafe00640 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e52568b0 0x55c1e52749c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fafa0003ae0 tx=0x7fafa0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a679bc726aa7ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafa4c00640 1 -- 172.21.15.3:0/3225016114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa004e040 con 0x55c1e52568b0 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafa4c00640 1 -- 172.21.15.3:0/3225016114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fafa002c780 con 0x55c1e52568b0 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.573+0000 7fafa4c00640 1 -- 172.21.15.3:0/3225016114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0045040 con 0x55c1e52568b0 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.574+0000 7fafb9dec740 1 -- 172.21.15.3:0/3225016114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e52568b0 msgr2=0x55c1e52749c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.574+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e52568b0 0x55c1e52749c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fafa0003ae0 tx=0x7fafa0005fd0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.574+0000 7fafb9dec740 1 -- 172.21.15.3:0/3225016114 shutdown_connections 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.574+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e525d590 0x55c1e525a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.574+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e5257260 0x55c1e5259d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.574+0000 7fafb9dec740 1 --2- 172.21.15.3:0/3225016114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e52568b0 0x55c1e52749c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.574+0000 7fafb9dec740 1 -- 172.21.15.3:0/3225016114 >> 172.21.15.3:0/3225016114 conn(0x55c1e5250f80 msgr2=0x55c1e5250160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.575+0000 7fafb9dec740 1 -- 172.21.15.3:0/3225016114 shutdown_connections 2024-03-21T08:33:18.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.575+0000 7fafb9dec740 1 -- 172.21.15.3:0/3225016114 wait complete. 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.575+0000 7fafb9dec740 1 -- start start 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.575+0000 7fafb9dec740 1 Processor -- start 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 0x55c1e527da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e527ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafb9dec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e525d590 0x55c1e527b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafb9dec740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb01aae30 con 0x55c1e5257260 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafb9dec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb01ac130 con 0x55c1e525d590 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafb9dec740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb01ab470 con 0x55c1e52568b0 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafb9dec740 10 client.0 _ll_register_callbacks cb 0x7fafa8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafaf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e527ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafaf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e527ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40126/0 (socket says 172.21.15.3:40126) 2024-03-21T08:33:18.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafaf400640 1 -- 172.21.15.3:0/1270205375 learned_addr learned my addr 172.21.15.3:0/1270205375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafaf400640 1 -- 172.21.15.3:0/1270205375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 msgr2=0x55c1e527da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafaf400640 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 0x55c1e527da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafaf400640 1 -- 172.21.15.3:0/1270205375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e525d590 msgr2=0x55c1e527b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.576+0000 7fafaf400640 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e525d590 0x55c1e527b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7fafaf400640 1 -- 172.21.15.3:0/1270205375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1e527bb30 con 0x55c1e5257260 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7fafaf400640 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e527ada0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d1010 tx=0x7faf9c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc8fca9a96686ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7faf9a000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7faf9a000640 1 -- 172.21.15.3:0/1270205375 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c020040 con 0x55c1e5257260 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7faf9a000640 1 -- 172.21.15.3:0/1270205375 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7faf9c097e40 con 0x55c1e5257260 2024-03-21T08:33:18.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafa002c780 con 0x55c1e5257260 2024-03-21T08:33:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7faf9a000640 1 -- 172.21.15.3:0/1270205375 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf9c09f920 con 0x55c1e5257260 2024-03-21T08:33:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7fafb9dec740 10 client.16128 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.577+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c1e52d3db0 con 0x55c1e5257260 2024-03-21T08:33:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7faf9a000640 1 -- 172.21.15.3:0/1270205375 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fafa002c780 con 0x55c1e5257260 2024-03-21T08:33:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7faf9a000640 1 -- 172.21.15.3:0/1270205375 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faf9c09ac20 con 0x55c1e5257260 2024-03-21T08:33:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7fafb9dec740 10 client.16128 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:18.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7fafb9dec740 10 client.16128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7fafb9dec740 20 client.16128 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7fafb9dec740 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52d4440 0x55c1e52d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7fafb9dec740 4 client.16128 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.578+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c1e51c1b50 con 0x55c1e52d4440 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.579+0000 7fafafe00640 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52d4440 0x55c1e52d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.579+0000 7fafafe00640 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52d4440 0x55c1e52d68a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e690523a7fe6042 server_cookie=717523a93d1dc81f in_seq=0 out_seq=0 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.579+0000 7faf9a000640 10 client.16128 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7faf9a000640 1 -- 172.21.15.3:0/1270205375 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c1e51c1b50 con 0x55c1e52d4440 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7faf9a000640 10 client.16128 handle_command_reply: tid=0 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 client.16128 shutdown 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 msgr2=0x55c1e527ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e527ada0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d1010 tx=0x7faf9c005d30 comp rx=0 tx=0).stop 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 shutdown_connections 2024-03-21T08:33:18.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c1e525d590 0x55c1e527b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1e5257260 0x55c1e527ada0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55c1e52d4440 0x55c1e52d68a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 --2- 172.21.15.3:0/1270205375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1e52568b0 0x55c1e527da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7faf9a000640 0 client.16128 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 >> 172.21.15.3:0/1270205375 conn(0x55c1e5250f80 msgr2=0x55c1e5252300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 shutdown_connections 2024-03-21T08:33:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.580+0000 7fafb9dec740 1 -- 172.21.15.3:0/1270205375 wait complete. 2024-03-21T08:33:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.581+0000 7fafb9dec740 20 client.16128 trim_cache size 0 max 16384 2024-03-21T08:33:18.585 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.581+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faf940a3150 msgr2=0x7faf940a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.581+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faf940a3150 0x7faf940a5610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005fd0 tx=0x7fafa8013680 comp rx=0 tx=0).stop 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.581+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 msgr2=0x7fafb019bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.581+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 0x7fafb019bc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d950 tx=0x7faf9c00dd50 comp rx=0 tx=0).stop 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 shutdown_connections 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafb0107560 0x7fafb0195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb0101e60 0x7fafb019bc00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7faf940a3150 0x7faf940a5610 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 --2- 172.21.15.3:0/3720434077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafb01014a0 0x7fafb019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 >> 172.21.15.3:0/3720434077 conn(0x7fafb00fcfb0 msgr2=0x7fafb00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 shutdown_connections 2024-03-21T08:33:18.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.582+0000 7fafb5a00640 1 -- 172.21.15.3:0/3720434077 wait complete. 2024-03-21T08:33:18.600 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:33:18.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T08:33:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 -- 172.21.15.3:0/1665787174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084107560 msgr2=0x7f8084107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 --2- 172.21.15.3:0/1665787174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084107560 0x7f8084107940 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8074009900 tx=0x7f807402bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 -- 172.21.15.3:0/1665787174 shutdown_connections 2024-03-21T08:33:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 --2- 172.21.15.3:0/1665787174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8084101e60 0x7f808410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 --2- 172.21.15.3:0/1665787174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80841014a0 0x7f8084101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 --2- 172.21.15.3:0/1665787174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084107560 0x7f8084107940 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 -- 172.21.15.3:0/1665787174 >> 172.21.15.3:0/1665787174 conn(0x7f80840fcfb0 msgr2=0x7f80840ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 -- 172.21.15.3:0/1665787174 shutdown_connections 2024-03-21T08:33:18.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.790+0000 7f8089a00640 1 -- 172.21.15.3:0/1665787174 wait complete. 2024-03-21T08:33:18.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.791+0000 7f8089a00640 1 Processor -- start 2024-03-21T08:33:18.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.792+0000 7f8089a00640 1 -- start start 2024-03-21T08:33:18.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.792+0000 7f8089a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80841014a0 0x7f808419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.792+0000 7f8089a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084101e60 0x7f808419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.792+0000 7f8089a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8084107560 0x7f8084197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.792+0000 7f8089a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80841116d0 con 0x7f8084101e60 2024-03-21T08:33:18.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.792+0000 7f8089a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084111550 con 0x7f8084107560 2024-03-21T08:33:18.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.792+0000 7f8089a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084111850 con 0x7f80841014a0 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8083400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084101e60 0x7f808419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8083e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80841014a0 0x7f808419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8089000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8084107560 0x7f8084197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8083400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084101e60 0x7f808419ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40156/0 (socket says 172.21.15.3:40156) 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8083e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80841014a0 0x7f808419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.3:41368/0 (socket says 172.21.15.3:41368) 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8083400640 1 -- 172.21.15.3:0/3736942313 learned_addr learned my addr 172.21.15.3:0/3736942313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8089000640 1 -- 172.21.15.3:0/3736942313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80841014a0 msgr2=0x7f808419d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8089000640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80841014a0 0x7f808419d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.793+0000 7f8089000640 1 -- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084101e60 msgr2=0x7f808419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.794+0000 7f8089000640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084101e60 0x7f808419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.794+0000 7f8089000640 1 -- 172.21.15.3:0/3736942313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80841981b0 con 0x7f8084107560 2024-03-21T08:33:18.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.794+0000 7f8083400640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084101e60 0x7f808419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.794+0000 7f8089000640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8084107560 0x7f8084197a30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f807c009fd0 tx=0x7f807c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.794+0000 7f8080c00640 1 -- 172.21.15.3:0/3736942313 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00ce20 con 0x7f8084107560 2024-03-21T08:33:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.795+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80841984a0 con 0x7f8084107560 2024-03-21T08:33:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.795+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80840738d0 con 0x7f8084107560 2024-03-21T08:33:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.795+0000 7f8080c00640 1 -- 172.21.15.3:0/3736942313 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f807c005020 con 0x7f8084107560 2024-03-21T08:33:18.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.795+0000 7f8080c00640 1 -- 172.21.15.3:0/3736942313 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c010690 con 0x7f8084107560 2024-03-21T08:33:18.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.798+0000 7f808de34740 1 Processor -- start 2024-03-21T08:33:18.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.801+0000 7f8080c00640 1 -- 172.21.15.3:0/3736942313 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f807c004b20 con 0x7f8084107560 2024-03-21T08:33:18.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.801+0000 7f808de34740 1 -- start start 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.801+0000 7f808de34740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a290bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.801+0000 7f808de34740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a2914810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f808de34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2917230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f808de34740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a286f2c0 con 0x55c0a290b8b0 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f808de34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a2817c90 con 0x55c0a2914de0 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f808de34740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a286b360 con 0x55c0a290c260 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f8083400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a290bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f8083400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a290bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40166/0 (socket says 172.21.15.3:40166) 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f8083400640 1 -- 172.21.15.3:0/868882019 learned_addr learned my addr 172.21.15.3:0/868882019 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f8083e00640 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a2914810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.802+0000 7f8083400640 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2917230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8083400640 1 -- 172.21.15.3:0/868882019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 msgr2=0x55c0a2914810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8083400640 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a2914810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8083400640 1 -- 172.21.15.3:0/868882019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 msgr2=0x55c0a2917230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8083400640 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2917230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8083400640 1 -- 172.21.15.3:0/868882019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a283c080 con 0x55c0a290b8b0 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8080c00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f80680a31b0 0x7f80680a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8080c00640 1 -- 172.21.15.3:0/3736942313 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f807c0d0470 con 0x7f8084107560 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8083400640 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a290bc90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f807000e970 tx=0x7f807000f1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522e43926df4dabe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8083e00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f80680a31b0 0x7f80680a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.803+0000 7f8079600640 1 -- 172.21.15.3:0/868882019 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070005020 con 0x55c0a290b8b0 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.804+0000 7f8079600640 1 -- 172.21.15.3:0/868882019 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8070005670 con 0x55c0a290b8b0 2024-03-21T08:33:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.804+0000 7f8079600640 1 -- 172.21.15.3:0/868882019 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070009740 con 0x55c0a290b8b0 2024-03-21T08:33:18.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.804+0000 7f808de34740 1 -- 172.21.15.3:0/868882019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 msgr2=0x55c0a290bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.804+0000 7f808de34740 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a290bc90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f807000e970 tx=0x7f807000f1d0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.804+0000 7f8083e00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f80680a31b0 0x7f80680a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8074009230 tx=0x7f8074005fd0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.805+0000 7f808de34740 1 -- 172.21.15.3:0/868882019 shutdown_connections 2024-03-21T08:33:18.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.805+0000 7f808de34740 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2917230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.805+0000 7f808de34740 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a2914810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.805+0000 7f808de34740 1 --2- 172.21.15.3:0/868882019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a290bc90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.805+0000 7f808de34740 1 -- 172.21.15.3:0/868882019 >> 172.21.15.3:0/868882019 conn(0x55c0a2905f80 msgr2=0x55c0a2906360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.805+0000 7f808de34740 1 -- 172.21.15.3:0/868882019 shutdown_connections 2024-03-21T08:33:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.806+0000 7f808de34740 1 -- 172.21.15.3:0/868882019 wait complete. 2024-03-21T08:33:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.806+0000 7f808de34740 1 -- start start 2024-03-21T08:33:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.806+0000 7f808de34740 1 Processor -- start 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f808de34740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a2914610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f808de34740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a29119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f808de34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2911ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f808de34740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a2917d50 con 0x55c0a290b8b0 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f808de34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a282b0d0 con 0x55c0a2914de0 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f808de34740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a2917ed0 con 0x55c0a290c260 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f808de34740 10 client.0 _ll_register_callbacks cb 0x55c0a282fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f8083e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a2914610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f8083e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a2914610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40182/0 (socket says 172.21.15.3:40182) 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f8083e00640 1 -- 172.21.15.3:0/3493373354 learned_addr learned my addr 172.21.15.3:0/3493373354 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f8083400640 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a29119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.807+0000 7f8089000640 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2911ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.808+0000 7f8083e00640 1 -- 172.21.15.3:0/3493373354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 msgr2=0x55c0a29119a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.808+0000 7f8083e00640 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a29119a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.808+0000 7f8083e00640 1 -- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 msgr2=0x55c0a2911ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.808+0000 7f8083e00640 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2911ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.808+0000 7f8083e00640 1 -- 172.21.15.3:0/3493373354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0a25833f0 con 0x55c0a290b8b0 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.808+0000 7f8083e00640 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a2914610 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8074035000 tx=0x7f807402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984bff4de99f1e4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.808+0000 7f806e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.809+0000 7f806e000640 1 -- 172.21.15.3:0/3493373354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740039c0 con 0x55c0a290b8b0 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.809+0000 7f806e000640 1 -- 172.21.15.3:0/3493373354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f8074003b60 con 0x55c0a290b8b0 2024-03-21T08:33:18.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.809+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8070005670 con 0x55c0a290b8b0 2024-03-21T08:33:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.809+0000 7f806e000640 1 -- 172.21.15.3:0/3493373354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8074003d60 con 0x55c0a290b8b0 2024-03-21T08:33:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.809+0000 7f808de34740 10 client.16140 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.809+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c0a264d7d0 con 0x55c0a290b8b0 2024-03-21T08:33:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f806e000640 1 -- 172.21.15.3:0/3493373354 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8070005670 con 0x55c0a290b8b0 2024-03-21T08:33:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f806e000640 1 -- 172.21.15.3:0/3493373354 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8074040570 con 0x55c0a290b8b0 2024-03-21T08:33:18.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f808de34740 10 client.16140 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f808de34740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f808de34740 20 client.16140 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f808de34740 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2924bd0 0x55c0a2926ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f808de34740 4 client.16140 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.810+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0a287c800 con 0x55c0a2924bd0 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.811+0000 7f8083400640 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2924bd0 0x55c0a2926ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.812+0000 7f8083400640 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2924bd0 0x55c0a2926ff0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2ac66925e82847 server_cookie=687bebac1d6cd723 in_seq=0 out_seq=0 2024-03-21T08:33:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.812+0000 7f806e000640 10 client.16140 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f806e000640 1 -- 172.21.15.3:0/3493373354 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c0a287c800 con 0x55c0a2924bd0 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f806e000640 10 client.16140 handle_command_reply: tid=0 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 client.16140 shutdown 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 msgr2=0x55c0a2914610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a2914610 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8074035000 tx=0x7f807402bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 shutdown_connections 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a2914de0 0x55c0a2911ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2924bd0 0x55c0a2926ff0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a290c260 0x55c0a29119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 --2- 172.21.15.3:0/3493373354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a2914610 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 >> 172.21.15.3:0/3493373354 conn(0x55c0a2905f80 msgr2=0x55c0a2907cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.813+0000 7f806e000640 0 client.16140 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.814+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 shutdown_connections 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.814+0000 7f808de34740 1 -- 172.21.15.3:0/3493373354 wait complete. 2024-03-21T08:33:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.814+0000 7f808de34740 20 client.16140 trim_cache size 0 max 16384 2024-03-21T08:33:18.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 Processor -- start 2024-03-21T08:33:18.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 -- start start 2024-03-21T08:33:18.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2905f80 0x55c0a2910270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a29140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a2914610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8064012560 con 0x55c0a290b8b0 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8064010060 con 0x55c0a290c260 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.827+0000 7f808de34740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806400e100 con 0x55c0a2905f80 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a29140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a29140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40192/0 (socket says 172.21.15.3:40192) 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 -- 172.21.15.3:0/778750659 learned_addr learned my addr 172.21.15.3:0/778750659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8089000640 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a2914610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083e00640 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2905f80 0x55c0a2910270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 -- 172.21.15.3:0/778750659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2905f80 msgr2=0x55c0a2910270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2905f80 0x55c0a2910270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 -- 172.21.15.3:0/778750659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 msgr2=0x55c0a2914610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a2914610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.828+0000 7f8083400640 1 -- 172.21.15.3:0/778750659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a2910950 con 0x55c0a290b8b0 2024-03-21T08:33:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.829+0000 7f8083400640 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a29140d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8070004e70 tx=0x7f807000e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382508e92e44cce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.829+0000 7f8078c00640 1 -- 172.21.15.3:0/778750659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070034070 con 0x55c0a290b8b0 2024-03-21T08:33:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.829+0000 7f8078c00640 1 -- 172.21.15.3:0/778750659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f80700050c0 con 0x55c0a290b8b0 2024-03-21T08:33:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.829+0000 7f8078c00640 1 -- 172.21.15.3:0/778750659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807002eaa0 con 0x55c0a290b8b0 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 -- 172.21.15.3:0/778750659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 msgr2=0x55c0a29140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a29140d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8070004e70 tx=0x7f807000e710 comp rx=0 tx=0).stop 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 -- 172.21.15.3:0/778750659 shutdown_connections 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a2914610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a290b8b0 0x55c0a29140d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 --2- 172.21.15.3:0/778750659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2905f80 0x55c0a2910270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 -- 172.21.15.3:0/778750659 >> 172.21.15.3:0/778750659 conn(0x55c0a2912040 msgr2=0x55c0a2905160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 -- 172.21.15.3:0/778750659 shutdown_connections 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 -- 172.21.15.3:0/778750659 wait complete. 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.830+0000 7f808de34740 1 -- start start 2024-03-21T08:33:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 1 Processor -- start 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a2905f80 0x55c0a291ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a291cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2914de0 0x55c0a2918e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806400e700 con 0x55c0a2905f80 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806400fa00 con 0x55c0a290c260 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806400ed40 con 0x55c0a2914de0 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.831+0000 7f808de34740 10 client.0 _ll_register_callbacks cb 0x7f8074009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a2905f80 0x55c0a291ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a2905f80 0x55c0a291ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40194/0 (socket says 172.21.15.3:40194) 2024-03-21T08:33:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 -- 172.21.15.3:0/1820387814 learned_addr learned my addr 172.21.15.3:0/1820387814 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083400640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a291cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8089000640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2914de0 0x55c0a2918e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 -- 172.21.15.3:0/1820387814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2914de0 msgr2=0x55c0a2918e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2914de0 0x55c0a2918e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 -- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 msgr2=0x55c0a291cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a291cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083e00640 1 -- 172.21.15.3:0/1820387814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0a2919670 con 0x55c0a2905f80 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.832+0000 7f8083400640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a291cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f8083e00640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a2905f80 0x55c0a291ca90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f807402c3a0 tx=0x7f8074040e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ac43bf967852c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f806e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f806e000640 1 -- 172.21.15.3:0/1820387814 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074066070 con 0x55c0a2905f80 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f806e000640 1 -- 172.21.15.3:0/1820387814 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f80740046c0 con 0x55c0a2905f80 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80700050c0 con 0x55c0a2905f80 2024-03-21T08:33:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f806e000640 1 -- 172.21.15.3:0/1820387814 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80740048c0 con 0x55c0a2905f80 2024-03-21T08:33:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f808de34740 10 client.16146 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.833+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c0a2981090 con 0x55c0a2905f80 2024-03-21T08:33:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f806e000640 1 -- 172.21.15.3:0/1820387814 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f80700050c0 con 0x55c0a2905f80 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f806e000640 1 -- 172.21.15.3:0/1820387814 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f807404e460 con 0x55c0a2905f80 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f808de34740 10 client.16146 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f808de34740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f808de34740 20 client.16146 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f808de34740 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2981790 0x55c0a2983bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f808de34740 4 client.16146 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.834+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0a2876b50 con 0x55c0a2981790 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.835+0000 7f8083400640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2981790 0x55c0a2983bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f806e000640 10 client.16146 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f8083400640 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2981790 0x55c0a2983bb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36ff43528567b91 server_cookie=cc1463e9ed090177 in_seq=0 out_seq=0 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f806e000640 1 -- 172.21.15.3:0/1820387814 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0a2876b50 con 0x55c0a2981790 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f806e000640 10 client.16146 handle_command_reply: tid=0 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 client.16146 shutdown 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a2905f80 msgr2=0x55c0a291ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a2905f80 0x55c0a291ca90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f807402c3a0 tx=0x7f8074040e30 comp rx=0 tx=0).stop 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 shutdown_connections 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c0a2981790 0x55c0a2983bb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0a2914de0 0x55c0a2918e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a290c260 0x55c0a291cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 --2- 172.21.15.3:0/1820387814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0a2905f80 0x55c0a291ca90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 >> 172.21.15.3:0/1820387814 conn(0x55c0a2912040 msgr2=0x55c0a2905160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.836+0000 7f806e000640 0 client.16146 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.837+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 shutdown_connections 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.837+0000 7f808de34740 1 -- 172.21.15.3:0/1820387814 wait complete. 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.837+0000 7f808de34740 20 client.16146 trim_cache size 0 max 16384 2024-03-21T08:33:18.840 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:33:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.837+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f80680a31b0 msgr2=0x7f80680a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.837+0000 7f8089a00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f80680a31b0 0x7f80680a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8074009230 tx=0x7f8074005fd0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.837+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8084107560 msgr2=0x7f8084197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.837+0000 7f8089a00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8084107560 0x7f8084197a30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f807c009fd0 tx=0x7f807c00ef10 comp rx=0 tx=0).stop 2024-03-21T08:33:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 shutdown_connections 2024-03-21T08:33:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8084107560 0x7f8084197a30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8084101e60 0x7f808419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f80680a31b0 0x7f80680a5670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 --2- 172.21.15.3:0/3736942313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80841014a0 0x7f808419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 >> 172.21.15.3:0/3736942313 conn(0x7f80840fcfb0 msgr2=0x7f80840fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 shutdown_connections 2024-03-21T08:33:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:18.838+0000 7f8089a00640 1 -- 172.21.15.3:0/3736942313 wait complete. 2024-03-21T08:33:18.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:18 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3275172031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:33:19.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:18 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3275172031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:33:19.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:18 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3275172031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:33:19.857 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T08:33:19.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:33:19.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T08:33:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 -- 172.21.15.3:0/1899906404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268107640 msgr2=0x7fd268107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 --2- 172.21.15.3:0/1899906404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268107640 0x7fd268107a20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009930 tx=0x7fd25c02bc20 comp rx=0 tx=0).stop 2024-03-21T08:33:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 -- 172.21.15.3:0/1899906404 shutdown_connections 2024-03-21T08:33:20.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 --2- 172.21.15.3:0/1899906404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268101ff0 0x7fd26810e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 --2- 172.21.15.3:0/1899906404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd268101630 0x7fd268101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 --2- 172.21.15.3:0/1899906404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268107640 0x7fd268107a20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 -- 172.21.15.3:0/1899906404 >> 172.21.15.3:0/1899906404 conn(0x7fd2680fd140 msgr2=0x7fd2680ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 -- 172.21.15.3:0/1899906404 shutdown_connections 2024-03-21T08:33:20.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.067+0000 7fd270200640 1 -- 172.21.15.3:0/1899906404 wait complete. 2024-03-21T08:33:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.068+0000 7fd270200640 1 Processor -- start 2024-03-21T08:33:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.068+0000 7fd270200640 1 -- start start 2024-03-21T08:33:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.068+0000 7fd270200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268101630 0x7fd2680739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.068+0000 7fd270200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd268101ff0 0x7fd268073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.068+0000 7fd270200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268107640 0x7fd268074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd270200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681118c0 con 0x7fd268101630 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd270200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd268111740 con 0x7fd268107640 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd270200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd268111a40 con 0x7fd268101ff0 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268101630 0x7fd2680739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268101630 0x7fd2680739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48014/0 (socket says 172.21.15.3:48014) 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 -- 172.21.15.3:0/479250545 learned_addr learned my addr 172.21.15.3:0/479250545 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26e200640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268107640 0x7fd268074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 -- 172.21.15.3:0/479250545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd268101ff0 msgr2=0x7fd268073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26ce00640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd268101ff0 0x7fd268073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd268101ff0 0x7fd268073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 -- 172.21.15.3:0/479250545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268107640 msgr2=0x7fd268074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268107640 0x7fd268074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26d800640 1 -- 172.21.15.3:0/479250545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2681a6380 con 0x7fd268101630 2024-03-21T08:33:20.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.069+0000 7fd26ce00640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd268101ff0 0x7fd268073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.070+0000 7fd26d800640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268101630 0x7fd2680739a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009230 tx=0x7fd25c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.070+0000 7fd266000640 1 -- 172.21.15.3:0/479250545 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c043020 con 0x7fd268101630 2024-03-21T08:33:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.070+0000 7fd266000640 1 -- 172.21.15.3:0/479250545 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd25c02cb60 con 0x7fd268101630 2024-03-21T08:33:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.070+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2681a6610 con 0x7fd268101630 2024-03-21T08:33:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.070+0000 7fd266000640 1 -- 172.21.15.3:0/479250545 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c003b50 con 0x7fd268101630 2024-03-21T08:33:20.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.070+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2681a6b60 con 0x7fd268101630 2024-03-21T08:33:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.073+0000 7fd266000640 1 -- 172.21.15.3:0/479250545 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7fd25c046020 con 0x7fd268101630 2024-03-21T08:33:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.073+0000 7fd272e10740 1 Processor -- start 2024-03-21T08:33:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.073+0000 7fd272e10740 1 -- start start 2024-03-21T08:33:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd272e10740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c6080 0x5619201c6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd272e10740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd272e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 0x5619201d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd272e10740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56192012aeb0 con 0x5619201c6080 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd272e10740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619200d3a60 con 0x5619201c69a0 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd272e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561920126f50 con 0x5619201cf580 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c6080 0x5619201c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c6080 0x5619201c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48020/0 (socket says 172.21.15.3:48020) 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 -- 172.21.15.3:0/632770799 learned_addr learned my addr 172.21.15.3:0/632770799 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26d800640 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 -- 172.21.15.3:0/632770799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 msgr2=0x5619201d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 0x5619201d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 -- 172.21.15.3:0/632770799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 msgr2=0x5619201cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.074+0000 7fd26e200640 1 -- 172.21.15.3:0/632770799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56191ff077e0 con 0x5619201c6080 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.075+0000 7fd26e200640 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c6080 0x5619201c6460 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd260009e60 tx=0x7fd260010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125e0d43eff9f626 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.075+0000 7fd266000640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd2500a31b0 0x7fd2500a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.075+0000 7fd266000640 1 -- 172.21.15.3:0/479250545 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fd25c0f1ff0 con 0x7fd268101630 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.075+0000 7fd25aa00640 1 -- 172.21.15.3:0/632770799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260016070 con 0x5619201c6080 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.075+0000 7fd25aa00640 1 -- 172.21.15.3:0/632770799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd260004d20 con 0x5619201c6080 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.075+0000 7fd25aa00640 1 -- 172.21.15.3:0/632770799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2600126b0 con 0x5619201c6080 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.075+0000 7fd26d800640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd2500a31b0 0x7fd2500a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 -- 172.21.15.3:0/632770799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c6080 msgr2=0x5619201c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd26d800640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd2500a31b0 0x7fd2500a5670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0ef2d0 tx=0x7fd25c0de040 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c6080 0x5619201c6460 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd260009e60 tx=0x7fd260010740 comp rx=0 tx=0).stop 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 -- 172.21.15.3:0/632770799 shutdown_connections 2024-03-21T08:33:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 0x5619201d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 --2- 172.21.15.3:0/632770799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c6080 0x5619201c6460 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 -- 172.21.15.3:0/632770799 >> 172.21.15.3:0/632770799 conn(0x5619201c07b0 msgr2=0x5619201c0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 -- 172.21.15.3:0/632770799 shutdown_connections 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.076+0000 7fd272e10740 1 -- 172.21.15.3:0/632770799 wait complete. 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 -- start start 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 Processor -- start 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c6080 0x5619201cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 0x5619201cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619201d2370 con 0x5619201c69a0 2024-03-21T08:33:20.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619200e6eb0 con 0x5619201c6080 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619201d24f0 con 0x5619201cf580 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd272e10740 10 client.0 _ll_register_callbacks cb 0x5619200db150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd26ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd26ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48024/0 (socket says 172.21.15.3:48024) 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.077+0000 7fd26ce00640 1 -- 172.21.15.3:0/4172116129 learned_addr learned my addr 172.21.15.3:0/4172116129 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd26e200640 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c6080 0x5619201cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd26ce00640 1 -- 172.21.15.3:0/4172116129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 msgr2=0x5619201cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd26ce00640 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 0x5619201cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd26ce00640 1 -- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c6080 msgr2=0x5619201cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd26ce00640 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c6080 0x5619201cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd26ce00640 1 -- 172.21.15.3:0/4172116129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56191ff67f70 con 0x5619201c69a0 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd26ce00640 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201cc200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d440 tx=0x7fd25400d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce29c110f5ed350 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd24f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd24f400640 1 -- 172.21.15.3:0/4172116129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400f8f0 con 0x5619201c69a0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd24f400640 1 -- 172.21.15.3:0/4172116129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fd25400fa90 con 0x5619201c69a0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.078+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd260004d20 con 0x5619201c69a0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd24f400640 1 -- 172.21.15.3:0/4172116129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd254022050 con 0x5619201c69a0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 10 client.16158 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56191fdff170 con 0x5619201c69a0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd24f400640 1 -- 172.21.15.3:0/4172116129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd260004d20 con 0x5619201c69a0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd24f400640 1 -- 172.21.15.3:0/4172116129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd2540163d0 con 0x5619201c69a0 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 10 client.16158 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 20 client.16158 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619201df720 0x5619201e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 4 client.16158 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:20.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.079+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619201383f0 con 0x5619201df720 2024-03-21T08:33:20.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.080+0000 7fd26e200640 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619201df720 0x5619201e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.081+0000 7fd26e200640 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619201df720 0x5619201e1b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95244da6b03d4bcd server_cookie=5a7a6d44cd4f485a in_seq=0 out_seq=0 2024-03-21T08:33:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.081+0000 7fd24f400640 10 client.16158 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd24f400640 1 -- 172.21.15.3:0/4172116129 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5619201383f0 con 0x5619201df720 2024-03-21T08:33:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd24f400640 10 client.16158 handle_command_reply: tid=0 2024-03-21T08:33:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 client.16158 shutdown 2024-03-21T08:33:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 msgr2=0x5619201cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201cc200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d440 tx=0x7fd25400d900 comp rx=0 tx=0).stop 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 shutdown_connections 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619201df720 0x5619201e1b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201cf580 0x5619201cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201cc200 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 --2- 172.21.15.3:0/4172116129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c6080 0x5619201cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 >> 172.21.15.3:0/4172116129 conn(0x5619201c07b0 msgr2=0x5619201c2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd24f400640 0 client.16158 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 shutdown_connections 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.082+0000 7fd272e10740 1 -- 172.21.15.3:0/4172116129 wait complete. 2024-03-21T08:33:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.083+0000 7fd272e10740 20 client.16158 trim_cache size 0 max 16384 2024-03-21T08:33:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 Processor -- start 2024-03-21T08:33:20.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 -- start start 2024-03-21T08:33:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 0x5619201cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201cf580 0x5619201cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681b3270 con 0x5619201cf580 2024-03-21T08:33:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681b0d70 con 0x5619201c69a0 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.100+0000 7fd272e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681aee10 con 0x5619201c6080 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201cf580 0x5619201cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201cf580 0x5619201cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48026/0 (socket says 172.21.15.3:48026) 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 -- 172.21.15.3:0/1810967024 learned_addr learned my addr 172.21.15.3:0/1810967024 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 0x5619201cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 -- 172.21.15.3:0/1810967024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 msgr2=0x5619201cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 0x5619201cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 -- 172.21.15.3:0/1810967024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 msgr2=0x5619201ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.101+0000 7fd26e200640 1 -- 172.21.15.3:0/1810967024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619201ca770 con 0x5619201cf580 2024-03-21T08:33:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.102+0000 7fd26e200640 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201cf580 0x5619201cee70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x5619201de300 tx=0x7fd260000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c726cbc96b826184 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.102+0000 7fd25a000640 1 -- 172.21.15.3:0/1810967024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260016040 con 0x5619201cf580 2024-03-21T08:33:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.102+0000 7fd25a000640 1 -- 172.21.15.3:0/1810967024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd26002a070 con 0x5619201cf580 2024-03-21T08:33:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.102+0000 7fd25a000640 1 -- 172.21.15.3:0/1810967024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260012a20 con 0x5619201cf580 2024-03-21T08:33:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 -- 172.21.15.3:0/1810967024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201cf580 msgr2=0x5619201cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201cf580 0x5619201cee70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x5619201de300 tx=0x7fd260000f40 comp rx=0 tx=0).stop 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 -- 172.21.15.3:0/1810967024 shutdown_connections 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201cf580 0x5619201cee70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201c69a0 0x5619201ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 --2- 172.21.15.3:0/1810967024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 0x5619201cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 -- 172.21.15.3:0/1810967024 >> 172.21.15.3:0/1810967024 conn(0x5619201c07b0 msgr2=0x5619201befa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 -- 172.21.15.3:0/1810967024 shutdown_connections 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 -- 172.21.15.3:0/1810967024 wait complete. 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 -- start start 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.103+0000 7fd272e10740 1 Processor -- start 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd272e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 0x5619201d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd272e10740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd272e10740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201cf580 0x5619201d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd272e10740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681af410 con 0x5619201c69a0 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd272e10740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681b0710 con 0x5619201cf580 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd272e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681afa50 con 0x5619201c6080 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd272e10740 10 client.0 _ll_register_callbacks cb 0x7fd254005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd26ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd26ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201d4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48028/0 (socket says 172.21.15.3:48028) 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd26ce00640 1 -- 172.21.15.3:0/2406187874 learned_addr learned my addr 172.21.15.3:0/2406187874 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd26e200640 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201cf580 0x5619201d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd26ce00640 1 -- 172.21.15.3:0/2406187874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 msgr2=0x5619201d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd26ce00640 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 0x5619201d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.104+0000 7fd26ce00640 1 -- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201cf580 msgr2=0x5619201d51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd26ce00640 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201cf580 0x5619201d51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd26ce00640 1 -- 172.21.15.3:0/2406187874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619201ca260 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd26ce00640 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201d4cb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5619201cd7d0 tx=0x7fd25400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe1abdf12312a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd24f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd24f400640 1 -- 172.21.15.3:0/2406187874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254016be0 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd24f400640 1 -- 172.21.15.3:0/2406187874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fd254016d80 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd26002a070 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd24f400640 1 -- 172.21.15.3:0/2406187874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd25400f6f0 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd272e10740 10 client.16164 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.105+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56192023c0b0 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd24f400640 1 -- 172.21.15.3:0/2406187874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd26002a070 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd24f400640 1 -- 172.21.15.3:0/2406187874 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd254031820 con 0x5619201c69a0 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd272e10740 10 client.16164 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd272e10740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd272e10740 20 client.16164 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd272e10740 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56192023c7b0 0x56192023ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd272e10740 4 client.16164 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T08:33:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.106+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561920132740 con 0x56192023c7b0 2024-03-21T08:33:20.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.107+0000 7fd26e200640 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56192023c7b0 0x56192023ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.107+0000 7fd26e200640 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56192023c7b0 0x56192023ebd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e9df4516de46e8 server_cookie=6d146dad7c0934c9 in_seq=0 out_seq=0 2024-03-21T08:33:20.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.107+0000 7fd24f400640 10 client.16164 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.109+0000 7fd24f400640 1 -- 172.21.15.3:0/2406187874 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561920132740 con 0x56192023c7b0 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.109+0000 7fd24f400640 10 client.16164 handle_command_reply: tid=0 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.109+0000 7fd272e10740 1 client.16164 shutdown 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.109+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 msgr2=0x5619201d4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.109+0000 7fd272e10740 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201d4cb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5619201cd7d0 tx=0x7fd25400de70 comp rx=0 tx=0).stop 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 shutdown_connections 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619201cf580 0x5619201d51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619201c69a0 0x5619201d4cb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56192023c7b0 0x56192023ebd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 --2- 172.21.15.3:0/2406187874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619201c6080 0x5619201d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 >> 172.21.15.3:0/2406187874 conn(0x5619201c07b0 msgr2=0x5619201bf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd24f400640 0 client.16164 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 shutdown_connections 2024-03-21T08:33:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 1 -- 172.21.15.3:0/2406187874 wait complete. 2024-03-21T08:33:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd272e10740 20 client.16164 trim_cache size 0 max 16384 2024-03-21T08:33:20.114 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:33:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-03-21T08:33:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "2918819f-ba80-4e7c-b665-accd01219ff3", 2024-03-21T08:33:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-03-21T08:33:20.115 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:33:20.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd2500a31b0 msgr2=0x7fd2500a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd270200640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd2500a31b0 0x7fd2500a5670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0ef2d0 tx=0x7fd25c0de040 comp rx=0 tx=0).stop 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268101630 msgr2=0x7fd2680739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.110+0000 7fd270200640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268101630 0x7fd2680739a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009230 tx=0x7fd25c0045e0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 shutdown_connections 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268107640 0x7fd268074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd2500a31b0 0x7fd2500a5670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd268101ff0 0x7fd268073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 --2- 172.21.15.3:0/479250545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd268101630 0x7fd2680739a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 >> 172.21.15.3:0/479250545 conn(0x7fd2680fd140 msgr2=0x7fd2680fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 shutdown_connections 2024-03-21T08:33:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.111+0000 7fd270200640 1 -- 172.21.15.3:0/479250545 wait complete. 2024-03-21T08:33:20.124 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:19 smithi003 ceph-mon[31675]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T08:33:20.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:19 smithi084 ceph-mon[35477]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T08:33:20.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:33:20.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:33:20.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:19 smithi171 ceph-mon[35388]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T08:33:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 -- 172.21.15.3:0/2558914573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c3010dff0 msgr2=0x7f5c301103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 --2- 172.21.15.3:0/2558914573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c3010dff0 0x7f5c301103e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009ca0 tx=0x7f5c2802c190 comp rx=0 tx=0).stop 2024-03-21T08:33:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 -- 172.21.15.3:0/2558914573 shutdown_connections 2024-03-21T08:33:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 --2- 172.21.15.3:0/2558914573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c3010dff0 0x7f5c301103e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 --2- 172.21.15.3:0/2558914573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c30100d60 0x7f5c3010dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 --2- 172.21.15.3:0/2558914573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c30100440 0x7f5c30100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 -- 172.21.15.3:0/2558914573 >> 172.21.15.3:0/2558914573 conn(0x7f5c30077750 msgr2=0x7f5c30077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 -- 172.21.15.3:0/2558914573 shutdown_connections 2024-03-21T08:33:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.326+0000 7f5c36600640 1 -- 172.21.15.3:0/2558914573 wait complete. 2024-03-21T08:33:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 Processor -- start 2024-03-21T08:33:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 -- start start 2024-03-21T08:33:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c30100440 0x7f5c3019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c30100d60 0x7f5c3019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c3010dff0 0x7f5c3019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30113b00 con 0x7f5c30100440 2024-03-21T08:33:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30113980 con 0x7f5c3010dff0 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.327+0000 7f5c36600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30113c80 con 0x7f5c30100d60 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c30100440 0x7f5c3019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c30100440 0x7f5c3019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48044/0 (socket says 172.21.15.3:48044) 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 -- 172.21.15.3:0/2912821691 learned_addr learned my addr 172.21.15.3:0/2912821691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c3010dff0 0x7f5c3019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c30100d60 0x7f5c3019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 -- 172.21.15.3:0/2912821691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c30100d60 msgr2=0x7f5c3019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c30100d60 0x7f5c3019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 -- 172.21.15.3:0/2912821691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c3010dff0 msgr2=0x7f5c3019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c3010dff0 0x7f5c3019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2f400640 1 -- 172.21.15.3:0/2912821691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3019f030 con 0x7f5c30100440 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.328+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c30100d60 0x7f5c3019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.329+0000 7f5c2f400640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c30100440 0x7f5c3019a040 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000d950 tx=0x7f5c2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.329+0000 7f5c27e00640 1 -- 172.21.15.3:0/2912821691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20017070 con 0x7f5c30100440 2024-03-21T08:33:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.329+0000 7f5c27e00640 1 -- 172.21.15.3:0/2912821691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c20009c30 con 0x7f5c30100440 2024-03-21T08:33:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.329+0000 7f5c27e00640 1 -- 172.21.15.3:0/2912821691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000f620 con 0x7f5c30100440 2024-03-21T08:33:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.329+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3019f320 con 0x7f5c30100440 2024-03-21T08:33:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.329+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c301a6bb0 con 0x7f5c30100440 2024-03-21T08:33:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.332+0000 7f5c39259740 1 Processor -- start 2024-03-21T08:33:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.332+0000 7f5c27e00640 1 -- 172.21.15.3:0/2912821691 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) v1 ==== 138086+0+0 (secure 0 0 0) 0x7f5c200109a0 con 0x7f5c30100440 2024-03-21T08:33:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.332+0000 7f5c39259740 1 -- start start 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c39259740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e4275a1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c39259740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c39259740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e4275ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c39259740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e427506250 con 0x55e4275a2c00 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c39259740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4274aea30 con 0x55e4275a1f20 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c39259740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4275022f0 con 0x55e4275ab290 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e4275a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e4275a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48060/0 (socket says 172.21.15.3:48060) 2024-03-21T08:33:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 -- 172.21.15.3:0/359355161 learned_addr learned my addr 172.21.15.3:0/359355161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2f400640 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e4275ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 -- 172.21.15.3:0/359355161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 msgr2=0x55e4275ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e4275ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 -- 172.21.15.3:0/359355161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 msgr2=0x55e4275a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.333+0000 7f5c2fe00640 1 -- 172.21.15.3:0/359355161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e427499690 con 0x55e4275a2c00 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.334+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e4275a1950 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005cc0 tx=0x7f5c280098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04f1a6bb1660bbc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.334+0000 7f5c27e00640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5c140a3230 0x7f5c140a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.334+0000 7f5c24c00640 1 -- 172.21.15.3:0/359355161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28043070 con 0x55e4275a2c00 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.334+0000 7f5c24c00640 1 -- 172.21.15.3:0/359355161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c28003e20 con 0x55e4275a2c00 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.334+0000 7f5c24c00640 1 -- 172.21.15.3:0/359355161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28039040 con 0x55e4275a2c00 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.334+0000 7f5c27e00640 1 -- 172.21.15.3:0/2912821691 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5c200cfa90 con 0x7f5c30100440 2024-03-21T08:33:20.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.334+0000 7f5c2f400640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5c140a3230 0x7f5c140a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 -- 172.21.15.3:0/359355161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 msgr2=0x55e4275a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e4275a1950 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005cc0 tx=0x7f5c280098e0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c2f400640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5c140a3230 0x7f5c140a56f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c200ccca0 tx=0x7f5c2000bf60 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 -- 172.21.15.3:0/359355161 shutdown_connections 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e4275ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 --2- 172.21.15.3:0/359355161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e4275a1950 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 -- 172.21.15.3:0/359355161 >> 172.21.15.3:0/359355161 conn(0x55e427569200 msgr2=0x55e42759c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 -- 172.21.15.3:0/359355161 shutdown_connections 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.335+0000 7f5c39259740 1 -- 172.21.15.3:0/359355161 wait complete. 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 -- start start 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 Processor -- start 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275a2c00 0x55e4275a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275ab290 0x55e4275a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4275ae0a0 con 0x55e4275ab290 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4274c1e80 con 0x55e4275a1f20 2024-03-21T08:33:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.336+0000 7f5c39259740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4275ae220 con 0x55e4275a2c00 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c39259740 10 client.0 _ll_register_callbacks cb 0x55e4274d4130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275ab290 0x55e4275a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275ab290 0x55e4275a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48068/0 (socket says 172.21.15.3:48068) 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 -- 172.21.15.3:0/3599657429 learned_addr learned my addr 172.21.15.3:0/3599657429 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275a2c00 0x55e4275a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 -- 172.21.15.3:0/3599657429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275a2c00 msgr2=0x55e4275a7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275a2c00 0x55e4275a7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 -- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 msgr2=0x55e4275aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 -- 172.21.15.3:0/3599657429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4272e47d0 con 0x55e4275ab290 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.337+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275ab290 0x55e4275a8190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802c7a0 tx=0x7f5c2802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b88cd54d6c0840 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c19600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:20.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c19600640 1 -- 172.21.15.3:0/3599657429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28039030 con 0x55e4275ab290 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c19600640 1 -- 172.21.15.3:0/3599657429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5c280325f0 con 0x55e4275ab290 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c28003e20 con 0x55e4275ab290 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c19600640 1 -- 172.21.15.3:0/3599657429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c28043040 con 0x55e4275ab290 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c39259740 10 client.16176 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e42721a3f0 con 0x55e4275ab290 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.338+0000 7f5c19600640 1 -- 172.21.15.3:0/3599657429 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c28003e20 con 0x55e4275ab290 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c19600640 1 -- 172.21.15.3:0/3599657429 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5c28048650 con 0x55e4275ab290 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c39259740 10 client.16176 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c39259740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c39259740 20 client.16176 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c39259740 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4275baf20 0x55e4275bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c39259740 4 client.16176 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e427513790 con 0x55e4275baf20 2024-03-21T08:33:20.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.339+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4275baf20 0x55e4275bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.340+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4275baf20 0x55e4275bd310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2199c08363f554a1 server_cookie=6b3f12f2c6ef47d9 in_seq=0 out_seq=0 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.340+0000 7f5c19600640 10 client.16176 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c19600640 1 -- 172.21.15.3:0/3599657429 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e427513790 con 0x55e4275baf20 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c19600640 10 client.16176 handle_command_reply: tid=0 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 client.16176 shutdown 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275ab290 msgr2=0x55e4275a8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275ab290 0x55e4275a8190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802c7a0 tx=0x7f5c2802e040 comp rx=0 tx=0).stop 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 shutdown_connections 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275ab290 0x55e4275a8190 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4275baf20 0x55e4275bd310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275a2c00 0x55e4275a7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 --2- 172.21.15.3:0/3599657429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 >> 172.21.15.3:0/3599657429 conn(0x55e427569200 msgr2=0x55e42759e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c19600640 0 client.16176 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 shutdown_connections 2024-03-21T08:33:20.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.342+0000 7f5c39259740 1 -- 172.21.15.3:0/3599657429 wait complete. 2024-03-21T08:33:20.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.343+0000 7f5c39259740 20 client.16176 trim_cache size 0 max 16384 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 Processor -- start 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 -- start start 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e42759e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e4275aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301b3370 con 0x55e4275a2c00 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301b0e70 con 0x55e4275a1f20 2024-03-21T08:33:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c39259740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301aef10 con 0x55e4275ab290 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e42759e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e42759e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48070/0 (socket says 172.21.15.3:48070) 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.360+0000 7f5c2ea00640 1 -- 172.21.15.3:0/2881074353 learned_addr learned my addr 172.21.15.3:0/2881074353 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c2ea00640 1 -- 172.21.15.3:0/2881074353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 msgr2=0x55e4275aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e4275aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c2ea00640 1 -- 172.21.15.3:0/2881074353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 msgr2=0x55e4275b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c2ea00640 1 -- 172.21.15.3:0/2881074353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4271da0f0 con 0x55e4275a2c00 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e42759e460 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c004700 tx=0x7f5c1c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d238f6c7eac315f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c1be00640 1 -- 172.21.15.3:0/2881074353 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c012650 con 0x55e4275a2c00 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c1be00640 1 -- 172.21.15.3:0/2881074353 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c1c012ca0 con 0x55e4275a2c00 2024-03-21T08:33:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.361+0000 7f5c1be00640 1 -- 172.21.15.3:0/2881074353 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c010cb0 con 0x55e4275a2c00 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 -- 172.21.15.3:0/2881074353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 msgr2=0x55e42759e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e42759e460 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c004700 tx=0x7f5c1c01c9e0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 -- 172.21.15.3:0/2881074353 shutdown_connections 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e4275aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a2c00 0x55e42759e460 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 --2- 172.21.15.3:0/2881074353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a1f20 0x55e4275b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 -- 172.21.15.3:0/2881074353 >> 172.21.15.3:0/2881074353 conn(0x55e427569200 msgr2=0x55e42759dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 -- 172.21.15.3:0/2881074353 shutdown_connections 2024-03-21T08:33:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.362+0000 7f5c39259740 1 -- 172.21.15.3:0/2881074353 wait complete. 2024-03-21T08:33:20.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 -- start start 2024-03-21T08:33:20.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 Processor -- start 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a1f20 0x55e4275a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a2c00 0x55e4275a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e427615440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301af510 con 0x55e4275a1f20 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301b0810 con 0x55e4275a2c00 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301afb50 con 0x55e4275ab290 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c39259740 10 client.0 _ll_register_callbacks cb 0x7f5c1c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a1f20 0x55e4275a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a1f20 0x55e4275a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48086/0 (socket says 172.21.15.3:48086) 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.363+0000 7f5c2fe00640 1 -- 172.21.15.3:0/2599225463 learned_addr learned my addr 172.21.15.3:0/2599225463 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a2c00 0x55e4275a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c2fe00640 1 -- 172.21.15.3:0/2599225463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 msgr2=0x55e427615440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e427615440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c2fe00640 1 -- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a2c00 msgr2=0x55e4275a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a2c00 0x55e4275a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c2fe00640 1 -- 172.21.15.3:0/2599225463 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e427615980 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c2fe00640 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a1f20 0x55e4275a4a50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802c6d0 tx=0x7f5c28009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7040e070a59c2d68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c19600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c19600640 1 -- 172.21.15.3:0/2599225463 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28039070 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c19600640 1 -- 172.21.15.3:0/2599225463 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5c28003970 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.364+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c1c012ca0 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c19600640 1 -- 172.21.15.3:0/2599225463 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c28032340 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 10 client.16182 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e427616c40 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c19600640 1 -- 172.21.15.3:0/2599225463 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c1c012ca0 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c19600640 1 -- 172.21.15.3:0/2599225463 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5c28032a70 con 0x55e4275a1f20 2024-03-21T08:33:20.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 10 client.16182 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:20.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:20.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 20 client.16182 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:20.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4276171d0 0x55e4276195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:20.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 4 client.16182 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:33:20.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.365+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e42750dae0 con 0x55e4276171d0 2024-03-21T08:33:20.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.366+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4276171d0 0x55e4276195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:20.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.366+0000 7f5c2ea00640 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4276171d0 0x55e4276195f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237e5230e6933b39 server_cookie=fa9ced5b3ed64227 in_seq=0 out_seq=0 2024-03-21T08:33:20.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.366+0000 7f5c19600640 10 client.16182 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c19600640 1 -- 172.21.15.3:0/2599225463 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e42750dae0 con 0x55e4276171d0 2024-03-21T08:33:20.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c19600640 10 client.16182 handle_command_reply: tid=0 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 client.16182 shutdown 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a1f20 msgr2=0x55e4275a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a1f20 0x55e4275a4a50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802c6d0 tx=0x7f5c28009540 comp rx=0 tx=0).stop 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 shutdown_connections 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55e4276171d0 0x55e4276195f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e4275ab290 0x55e427615440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4275a2c00 0x55e4275a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c19600640 0 client.16182 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 --2- 172.21.15.3:0/2599225463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4275a1f20 0x55e4275a4a50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 >> 172.21.15.3:0/2599225463 conn(0x55e427569200 msgr2=0x55e42759ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 shutdown_connections 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 1 -- 172.21.15.3:0/2599225463 wait complete. 2024-03-21T08:33:20.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.367+0000 7f5c39259740 20 client.16182 trim_cache size 0 max 16384 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "2918819f-ba80-4e7c-b665-accd01219ff3": { 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "2918819f-ba80-4e7c-b665-accd01219ff3", 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:33:20.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5c140a3230 msgr2=0x7f5c140a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5c140a3230 0x7f5c140a56f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c200ccca0 tx=0x7f5c2000bf60 comp rx=0 tx=0).stop 2024-03-21T08:33:20.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c30100440 msgr2=0x7f5c3019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:20.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c30100440 0x7f5c3019a040 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000d950 tx=0x7f5c2000de10 comp rx=0 tx=0).stop 2024-03-21T08:33:20.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 shutdown_connections 2024-03-21T08:33:20.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c3010dff0 0x7f5c3019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5c140a3230 0x7f5c140a56f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c30100d60 0x7f5c3019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 --2- 172.21.15.3:0/2912821691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c30100440 0x7f5c3019a040 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:20.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 >> 172.21.15.3:0/2912821691 conn(0x7f5c30077750 msgr2=0x7f5c3010f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:20.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 shutdown_connections 2024-03-21T08:33:20.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:20.368+0000 7f5c36600640 1 -- 172.21.15.3:0/2912821691 wait complete. 2024-03-21T08:33:20.384 INFO:tasks.cephfs.filesystem:scrub status for tag:2918819f-ba80-4e7c-b665-accd01219ff3 - {'path': '/', 'tag': '2918819f-ba80-4e7c-b665-accd01219ff3', 'options': 'recursive,force'} 2024-03-21T08:33:22.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:21 smithi084 ceph-mon[35477]: scrub queued for path: / 2024-03-21T08:33:22.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:21 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:22.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:21 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:33:22.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:21 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:22.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:21 smithi084 ceph-mon[35477]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T08:33:22.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:21 smithi084 ceph-mon[35477]: scrub summary: idle 2024-03-21T08:33:22.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:21 smithi171 ceph-mon[35388]: scrub queued for path: / 2024-03-21T08:33:22.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:21 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:22.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:21 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:33:22.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:21 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:22.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:21 smithi171 ceph-mon[35388]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T08:33:22.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:21 smithi171 ceph-mon[35388]: scrub summary: idle 2024-03-21T08:33:22.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:21 smithi003 ceph-mon[31675]: scrub queued for path: / 2024-03-21T08:33:22.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:21 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:22.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:21 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:33:22.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:21 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:22.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:21 smithi003 ceph-mon[31675]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T08:33:22.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:21 smithi003 ceph-mon[31675]: scrub summary: idle 2024-03-21T08:33:22.848 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 59% (7303/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 98% (12078/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-21T08:33:23.659 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to '2edfee68ad7243bed422a049b6f5f58619300f11'. 2024-03-21T08:33:23.659 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:33:23.659 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T08:33:23.659 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:33:23.660 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at 2edfee68ad7 Merge PR #55768 into wip-vshankar-testing-20240320.053002 2024-03-21T08:33:23.669 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T08:33:23.732 INFO:tasks.workunit.client.0.smithi003.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T08:33:23.740 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T08:33:23.740 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T08:33:24.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:23 smithi084 ceph-mon[35477]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-03-21T08:33:24.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:23 smithi084 ceph-mon[35477]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-21T08:33:24.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:23 smithi171 ceph-mon[35388]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-03-21T08:33:24.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:23 smithi171 ceph-mon[35388]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-21T08:33:24.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:23 smithi003 ceph-mon[31675]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-03-21T08:33:24.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:23 smithi003 ceph-mon[31675]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-21T08:33:24.691 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T08:33:24.807 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T08:33:24.901 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T08:33:24.908 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T08:33:24.908 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T08:33:24.996 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T08:33:25.007 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:33:25.007 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T08:33:25.062 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-03-21T08:33:25.063 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-03-21T08:33:25.064 DEBUG:teuthology.orchestra.run.smithi003:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-03-21T08:33:25.161 INFO:tasks.workunit.client.0.smithi003.stderr:+ set -e 2024-03-21T08:33:25.161 INFO:tasks.workunit.client.0.smithi003.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-03-21T08:33:25.179 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into 'xfstests-dev'... 2024-03-21T08:33:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 -- 172.21.15.3:0/2143946754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 msgr2=0x7f030c10e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 --2- 172.21.15.3:0/2143946754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 0x7f030c10e330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0304009c10 tx=0x7f030402bef0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 -- 172.21.15.3:0/2143946754 shutdown_connections 2024-03-21T08:33:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 --2- 172.21.15.3:0/2143946754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 0x7f030c10e330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 --2- 172.21.15.3:0/2143946754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c068e90 0x7f030c069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 --2- 172.21.15.3:0/2143946754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f030c1075b0 0x7f030c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 -- 172.21.15.3:0/2143946754 >> 172.21.15.3:0/2143946754 conn(0x7f030c075f00 msgr2=0x7f030c076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 -- 172.21.15.3:0/2143946754 shutdown_connections 2024-03-21T08:33:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.041+0000 7f030b200640 1 -- 172.21.15.3:0/2143946754 wait complete. 2024-03-21T08:33:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 Processor -- start 2024-03-21T08:33:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 -- start start 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f030c068e90 0x7f030c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 0x7f030c100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c1075b0 0x7f030c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1116e0 con 0x7f030c1075b0 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c111560 con 0x7f030c069850 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.042+0000 7f030b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c111860 con 0x7f030c068e90 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c1075b0 0x7f030c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c1075b0 0x7f030c101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48096/0 (socket says 172.21.15.3:48096) 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 -- 172.21.15.3:0/1054598746 learned_addr learned my addr 172.21.15.3:0/1054598746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f0309400640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 0x7f030c100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f0309e00640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f030c068e90 0x7f030c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 -- 172.21.15.3:0/1054598746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f030c068e90 msgr2=0x7f030c1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f030c068e90 0x7f030c1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 -- 172.21.15.3:0/1054598746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 msgr2=0x7f030c100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 0x7f030c100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.043+0000 7f030a800640 1 -- 172.21.15.3:0/1054598746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c1a60c0 con 0x7f030c1075b0 2024-03-21T08:33:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.044+0000 7f030a800640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c1075b0 0x7f030c101170 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0304034fd0 tx=0x7f0304033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.044+0000 7f0302a00640 1 -- 172.21.15.3:0/1054598746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304043020 con 0x7f030c1075b0 2024-03-21T08:33:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.044+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c1a6350 con 0x7f030c1075b0 2024-03-21T08:33:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.045+0000 7f0302a00640 1 -- 172.21.15.3:0/1054598746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f030402ca60 con 0x7f030c1075b0 2024-03-21T08:33:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.045+0000 7f0302a00640 1 -- 172.21.15.3:0/1054598746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304034430 con 0x7f030c1075b0 2024-03-21T08:33:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.045+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c1a68f0 con 0x7f030c1075b0 2024-03-21T08:33:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.048+0000 7f0313b83740 1 Processor -- start 2024-03-21T08:33:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.048+0000 7f0302a00640 1 -- 172.21.15.3:0/1054598746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f0304046020 con 0x7f030c1075b0 2024-03-21T08:33:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0313b83740 1 -- start start 2024-03-21T08:33:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0313b83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abace1e0 0x55d7abace5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0313b83740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abaceb00 0x55d7abad7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0313b83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0313b83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aba903b0 con 0x55d7abad78d0 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0313b83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aba32830 con 0x55d7abace1e0 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0313b83740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ab9db5d0 con 0x55d7abaceb00 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.049+0000 7f0309400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48106/0 (socket says 172.21.15.3:48106) 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 -- 172.21.15.3:0/4230946285 learned_addr learned my addr 172.21.15.3:0/4230946285 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abace1e0 0x55d7abace5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309e00640 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abaceb00 0x55d7abad7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 -- 172.21.15.3:0/4230946285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abaceb00 msgr2=0x55d7abad7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abaceb00 0x55d7abad7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 -- 172.21.15.3:0/4230946285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abace1e0 msgr2=0x55d7abace5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abace1e0 0x55d7abace5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 -- 172.21.15.3:0/4230946285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7ab9f32c0 con 0x55d7abad78d0 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.050+0000 7f0309400640 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad9cc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f02f80058c0 tx=0x7f02f8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61773caf3086c3ce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.051+0000 7f0302a00640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f02f00a3070 0x7f02f00a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.051+0000 7f02f7400640 1 -- 172.21.15.3:0/4230946285 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8004050 con 0x55d7abad78d0 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.051+0000 7f0302a00640 1 -- 172.21.15.3:0/1054598746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f03040f2240 con 0x7f030c1075b0 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.051+0000 7f02f7400640 1 -- 172.21.15.3:0/4230946285 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02f800fc70 con 0x55d7abad78d0 2024-03-21T08:33:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.051+0000 7f02f7400640 1 -- 172.21.15.3:0/4230946285 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8003900 con 0x55d7abad78d0 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.051+0000 7f0309e00640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f02f00a3070 0x7f02f00a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 -- 172.21.15.3:0/4230946285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 msgr2=0x55d7abad9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad9cc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f02f80058c0 tx=0x7f02f8005850 comp rx=0 tx=0).stop 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 -- 172.21.15.3:0/4230946285 shutdown_connections 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad9cc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abaceb00 0x55d7abad7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 --2- 172.21.15.3:0/4230946285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abace1e0 0x55d7abace5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0309e00640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f02f00a3070 0x7f02f00a5530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55d7abacdc90 tx=0x7f02fc00b4d0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 -- 172.21.15.3:0/4230946285 >> 172.21.15.3:0/4230946285 conn(0x55d7abac8b00 msgr2=0x55d7abac8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 -- 172.21.15.3:0/4230946285 shutdown_connections 2024-03-21T08:33:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 -- 172.21.15.3:0/4230946285 wait complete. 2024-03-21T08:33:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.052+0000 7f0313b83740 1 -- start start 2024-03-21T08:33:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 1 Processor -- start 2024-03-21T08:33:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abad4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ab9eea20 con 0x55d7abad78d0 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aba2e8d0 con 0x55d7abaceb00 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7abada500 con 0x55d7abace1e0 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.053+0000 7f0313b83740 10 client.0 _ll_register_callbacks cb 0x55d7aba2b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f0309e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48118/0 (socket says 172.21.15.3:48118) 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 -- 172.21.15.3:0/1289537976 learned_addr learned my addr 172.21.15.3:0/1289537976 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f0309400640 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abad4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 -- 172.21.15.3:0/1289537976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 msgr2=0x55d7abad49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 -- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 msgr2=0x55d7abad4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abad4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.054+0000 7f030a800640 1 -- 172.21.15.3:0/1289537976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7ab747210 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f030a800640 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad5470 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0304009310 tx=0x7f03040bde60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6016548a33b8b8b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f02ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f02ebe00640 1 -- 172.21.15.3:0/1289537976 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030403ce30 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f02ebe00640 1 -- 172.21.15.3:0/1289537976 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f0304003ca0 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02f800fc70 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f02ebe00640 1 -- 172.21.15.3:0/1289537976 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03040bea20 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f0313b83740 10 client.16194 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.055+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d7ab813620 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f02ebe00640 1 -- 172.21.15.3:0/1289537976 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02f800fc70 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f02ebe00640 1 -- 172.21.15.3:0/1289537976 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f03040c68a0 con 0x55d7abad78d0 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f0313b83740 10 client.16194 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f0313b83740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f0313b83740 20 client.16194 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f0313b83740 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abae79b0 0x55d7abae9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f0313b83740 4 client.16194 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.056+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7aba3fd70 con 0x55d7abae79b0 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.057+0000 7f0309e00640 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abae79b0 0x55d7abae9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.057+0000 7f0309e00640 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abae79b0 0x55d7abae9e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c164dd998490f1bc server_cookie=96578a1840fada57 in_seq=0 out_seq=0 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.057+0000 7f02ebe00640 10 client.16194 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f02ebe00640 1 -- 172.21.15.3:0/1289537976 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d7aba3fd70 con 0x55d7abae79b0 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f02ebe00640 10 client.16194 handle_command_reply: tid=0 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 client.16194 shutdown 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 msgr2=0x55d7abad5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad5470 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0304009310 tx=0x7f03040bde60 comp rx=0 tx=0).stop 2024-03-21T08:33:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 shutdown_connections 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abad5470 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abad4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abae79b0 0x55d7abae9e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.059+0000 7f0313b83740 1 --2- 172.21.15.3:0/1289537976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.060+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 >> 172.21.15.3:0/1289537976 conn(0x55d7abac8b00 msgr2=0x55d7abaca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.060+0000 7f02ebe00640 0 client.16194 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.060+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 shutdown_connections 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.060+0000 7f0313b83740 1 -- 172.21.15.3:0/1289537976 wait complete. 2024-03-21T08:33:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.060+0000 7f0313b83740 20 client.16194 trim_cache size 0 max 16384 2024-03-21T08:33:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 Processor -- start 2024-03-21T08:33:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 -- start start 2024-03-21T08:33:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abae2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abae3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1b30a0 con 0x55d7abad78d0 2024-03-21T08:33:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1b0ba0 con 0x55d7abaceb00 2024-03-21T08:33:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.078+0000 7f0313b83740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1aec40 con 0x55d7abace1e0 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abae3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abae3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48120/0 (socket says 172.21.15.3:48120) 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 -- 172.21.15.3:0/2801562281 learned_addr learned my addr 172.21.15.3:0/2801562281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f0309e00640 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f0309400640 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abae2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 -- 172.21.15.3:0/2801562281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 msgr2=0x55d7abad7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 -- 172.21.15.3:0/2801562281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 msgr2=0x55d7abae2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abae2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.079+0000 7f030a800640 1 -- 172.21.15.3:0/2801562281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7abad2fa0 con 0x55d7abad78d0 2024-03-21T08:33:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.080+0000 7f030a800640 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abae3150 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f03040ef730 tx=0x7f0304005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50a442215d8e550 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.080+0000 7f02f6a00640 1 -- 172.21.15.3:0/2801562281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040bdc80 con 0x55d7abad78d0 2024-03-21T08:33:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.080+0000 7f02f6a00640 1 -- 172.21.15.3:0/2801562281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f030403ce30 con 0x55d7abad78d0 2024-03-21T08:33:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.080+0000 7f02f6a00640 1 -- 172.21.15.3:0/2801562281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040c6a00 con 0x55d7abad78d0 2024-03-21T08:33:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 -- 172.21.15.3:0/2801562281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 msgr2=0x55d7abae3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abae3150 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f03040ef730 tx=0x7f0304005fd0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 -- 172.21.15.3:0/2801562281 shutdown_connections 2024-03-21T08:33:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abad78d0 0x55d7abae3150 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abaceb00 0x55d7abae2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 --2- 172.21.15.3:0/2801562281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 -- 172.21.15.3:0/2801562281 >> 172.21.15.3:0/2801562281 conn(0x55d7abac8b00 msgr2=0x55d7abaca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 -- 172.21.15.3:0/2801562281 shutdown_connections 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 -- 172.21.15.3:0/2801562281 wait complete. 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.081+0000 7f0313b83740 1 -- start start 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 1 Processor -- start 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abaceb00 0x55d7abad1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abad78d0 0x55d7abb4aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1af240 con 0x55d7abaceb00 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1b0540 con 0x55d7abad78d0 2024-03-21T08:33:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1af880 con 0x55d7abace1e0 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0313b83740 10 client.0 _ll_register_callbacks cb 0x7f0304005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.082+0000 7f0309400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abaceb00 0x55d7abad1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abaceb00 0x55d7abad1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48132/0 (socket says 172.21.15.3:48132) 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 -- 172.21.15.3:0/1396674798 learned_addr learned my addr 172.21.15.3:0/1396674798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f030a800640 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abad78d0 0x55d7abb4aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309e00640 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 -- 172.21.15.3:0/1396674798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 msgr2=0x55d7abad18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 -- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abad78d0 msgr2=0x55d7abb4aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abad78d0 0x55d7abb4aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 -- 172.21.15.3:0/1396674798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7abb4b0c0 con 0x55d7abaceb00 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f0309400640 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abaceb00 0x55d7abad1e30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f02f80058c0 tx=0x7f02f8009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6ad5ad6e3a3b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.083+0000 7f02ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f02ebe00640 1 -- 172.21.15.3:0/1396674798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8030060 con 0x55d7abaceb00 2024-03-21T08:33:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f02ebe00640 1 -- 172.21.15.3:0/1396674798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f02f800f990 con 0x55d7abaceb00 2024-03-21T08:33:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f030403ce30 con 0x55d7abaceb00 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f02ebe00640 1 -- 172.21.15.3:0/1396674798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f8037050 con 0x55d7abaceb00 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f0313b83740 10 client.16200 fetch_fsmap learned FSMap version 23 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d7abb4c3d0 con 0x55d7abaceb00 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f02ebe00640 1 -- 172.21.15.3:0/1396674798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f030403ce30 con 0x55d7abaceb00 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.084+0000 7f02ebe00640 1 -- 172.21.15.3:0/1396674798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f02f8004bf0 con 0x55d7abaceb00 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.085+0000 7f0313b83740 10 client.16200 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.085+0000 7f0313b83740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.085+0000 7f0313b83740 20 client.16200 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.085+0000 7f0313b83740 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abb4c860 0x55d7abb4ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.085+0000 7f0313b83740 4 client.16200 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.085+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7aba3a0c0 con 0x55d7abb4c860 2024-03-21T08:33:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.085+0000 7f0309e00640 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abb4c860 0x55d7abb4ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.086+0000 7f0309e00640 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abb4c860 0x55d7abb4ec80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9ab871dd1c03e5 server_cookie=ac9c45785cbb7c94 in_seq=0 out_seq=0 2024-03-21T08:33:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.086+0000 7f02ebe00640 10 client.16200 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:26.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:25 smithi084 ceph-mon[35477]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-03-21T08:33:26.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:25 smithi171 ceph-mon[35388]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-03-21T08:33:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.136+0000 7f02ebe00640 1 -- 172.21.15.3:0/1396674798 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7aba3a0c0 con 0x55d7abb4c860 2024-03-21T08:33:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.136+0000 7f02ebe00640 10 client.16200 handle_command_reply: tid=0 2024-03-21T08:33:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.136+0000 7f0313b83740 1 client.16200 shutdown 2024-03-21T08:33:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.136+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abaceb00 msgr2=0x55d7abad1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.136+0000 7f0313b83740 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abaceb00 0x55d7abad1e30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f02f80058c0 tx=0x7f02f8009600 comp rx=0 tx=0).stop 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 shutdown_connections 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7abad78d0 0x55d7abb4aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7abaceb00 0x55d7abad1e30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d7abb4c860 0x55d7abb4ec80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 --2- 172.21.15.3:0/1396674798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d7abace1e0 0x55d7abad18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 >> 172.21.15.3:0/1396674798 conn(0x55d7abac8b00 msgr2=0x55d7abac72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f02ebe00640 0 client.16200 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 shutdown_connections 2024-03-21T08:33:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 1 -- 172.21.15.3:0/1396674798 wait complete. 2024-03-21T08:33:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.137+0000 7f0313b83740 20 client.16200 trim_cache size 0 max 16384 2024-03-21T08:33:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:33:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:33:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:33:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.138+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f02f00a3070 msgr2=0x7f02f00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.138+0000 7f030b200640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f02f00a3070 0x7f02f00a5530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55d7abacdc90 tx=0x7f02fc00b4d0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.138+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c1075b0 msgr2=0x7f030c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.138+0000 7f030b200640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c1075b0 0x7f030c101170 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0304034fd0 tx=0x7f0304033c90 comp rx=0 tx=0).stop 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 shutdown_connections 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c1075b0 0x7f030c101170 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f030c069850 0x7f030c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f02f00a3070 0x7f02f00a5530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 --2- 172.21.15.3:0/1054598746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f030c068e90 0x7f030c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 >> 172.21.15.3:0/1054598746 conn(0x7f030c075f00 msgr2=0x7f030c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 shutdown_connections 2024-03-21T08:33:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:26.139+0000 7f030b200640 1 -- 172.21.15.3:0/1054598746 wait complete. 2024-03-21T08:33:26.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:25 smithi003 ceph-mon[31675]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-03-21T08:33:28.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:27 smithi084 ceph-mon[35477]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-03-21T08:33:28.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:27 smithi171 ceph-mon[35388]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-03-21T08:33:28.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:27 smithi003 ceph-mon[31675]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-03-21T08:33:30.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:29 smithi084 ceph-mon[35477]: pgmap v52: 129 pgs: 129 active+clean; 1.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-03-21T08:33:30.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:29 smithi171 ceph-mon[35388]: pgmap v52: 129 pgs: 129 active+clean; 1.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-03-21T08:33:30.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:29 smithi003 ceph-mon[31675]: pgmap v52: 129 pgs: 129 active+clean; 1.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-03-21T08:33:32.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:31 smithi084 ceph-mon[35477]: pgmap v53: 129 pgs: 129 active+clean; 1.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T08:33:32.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:31 smithi171 ceph-mon[35388]: pgmap v53: 129 pgs: 129 active+clean; 1.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T08:33:32.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:31 smithi003 ceph-mon[31675]: pgmap v53: 129 pgs: 129 active+clean; 1.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T08:33:33.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:32 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:33.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:32 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:33.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:32 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:34.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:33 smithi084 ceph-mon[35477]: pgmap v54: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 28 op/s 2024-03-21T08:33:34.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:33 smithi171 ceph-mon[35388]: pgmap v54: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 28 op/s 2024-03-21T08:33:34.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:33 smithi003 ceph-mon[31675]: pgmap v54: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 28 op/s 2024-03-21T08:33:36.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:35 smithi084 ceph-mon[35477]: pgmap v55: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-21T08:33:36.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:35 smithi171 ceph-mon[35388]: pgmap v55: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-21T08:33:36.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:35 smithi003 ceph-mon[31675]: pgmap v55: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-21T08:33:38.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:37 smithi084 ceph-mon[35477]: pgmap v56: 129 pgs: 129 active+clean; 6.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 481 KiB/s wr, 27 op/s 2024-03-21T08:33:38.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:37 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:37 smithi171 ceph-mon[35388]: pgmap v56: 129 pgs: 129 active+clean; 6.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 481 KiB/s wr, 27 op/s 2024-03-21T08:33:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:37 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:38.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:37 smithi003 ceph-mon[31675]: pgmap v56: 129 pgs: 129 active+clean; 6.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 481 KiB/s wr, 27 op/s 2024-03-21T08:33:38.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:37 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:39.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:38 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:39.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:38 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:39.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:38 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:33:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 -- 172.21.15.3:0/3095294285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0112660 msgr2=0x7f51d0114a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 --2- 172.21.15.3:0/3095294285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0112660 0x7f51d0114a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009860 tx=0x7f51c802bc10 comp rx=0 tx=0).stop 2024-03-21T08:33:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 -- 172.21.15.3:0/3095294285 shutdown_connections 2024-03-21T08:33:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 --2- 172.21.15.3:0/3095294285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0112660 0x7f51d0114a50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 --2- 172.21.15.3:0/3095294285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51d0074360 0x7f51d0112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 --2- 172.21.15.3:0/3095294285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d0073a40 0x7f51d0073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 -- 172.21.15.3:0/3095294285 >> 172.21.15.3:0/3095294285 conn(0x7f51d00fd120 msgr2=0x7f51d00ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 -- 172.21.15.3:0/3095294285 shutdown_connections 2024-03-21T08:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.306+0000 7f51d6e00640 1 -- 172.21.15.3:0/3095294285 wait complete. 2024-03-21T08:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.307+0000 7f51d6e00640 1 Processor -- start 2024-03-21T08:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.308+0000 7f51d6e00640 1 -- start start 2024-03-21T08:33:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.308+0000 7f51d6e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d0073a40 0x7f51d019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.308+0000 7f51d6e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0074360 0x7f51d019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.308+0000 7f51d6e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51d0112660 0x7f51d01a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.308+0000 7f51d6e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0117f10 con 0x7f51d0112660 2024-03-21T08:33:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.308+0000 7f51d6e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0117d90 con 0x7f51d0074360 2024-03-21T08:33:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.308+0000 7f51d6e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0118090 con 0x7f51d0073a40 2024-03-21T08:33:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0074360 0x7f51d019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51d0112660 0x7f51d01a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0074360 0x7f51d019e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:39300/0 (socket says 172.21.15.3:39300) 2024-03-21T08:33:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 -- 172.21.15.3:0/2087865372 learned_addr learned my addr 172.21.15.3:0/2087865372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51cfe00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d0073a40 0x7f51d019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 -- 172.21.15.3:0/2087865372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d0073a40 msgr2=0x7f51d019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d0073a40 0x7f51d019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 -- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51d0112660 msgr2=0x7f51d01a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51d0112660 0x7f51d01a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51c7400640 1 -- 172.21.15.3:0/2087865372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d01a3480 con 0x7f51d0074360 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.309+0000 7f51d4e00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51d0112660 0x7f51d01a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.310+0000 7f51c7400640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0074360 0x7f51d019e9d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51b800d950 tx=0x7f51b800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.310+0000 7f51cd600640 1 -- 172.21.15.3:0/2087865372 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8017070 con 0x7f51d0074360 2024-03-21T08:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.310+0000 7f51cd600640 1 -- 172.21.15.3:0/2087865372 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51b8009c30 con 0x7f51d0074360 2024-03-21T08:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.310+0000 7f51cd600640 1 -- 172.21.15.3:0/2087865372 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800f620 con 0x7f51d0074360 2024-03-21T08:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.310+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d01a3770 con 0x7f51d0074360 2024-03-21T08:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.311+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d01ab000 con 0x7f51d0074360 2024-03-21T08:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 Processor -- start 2024-03-21T08:33:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 -- start start 2024-03-21T08:33:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51cd600640 1 -- 172.21.15.3:0/2087865372 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f51b80109a0 con 0x7f51d0074360 2024-03-21T08:33:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa206d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2076180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa2078ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa202f3b0 con 0x55eaa206d8f0 2024-03-21T08:33:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa1fd1830 con 0x55eaa206cfd0 2024-03-21T08:33:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.315+0000 7f51d9b75740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa1f7a5d0 con 0x55eaa20766c0 2024-03-21T08:33:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2076180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa206d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa206d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:39316/0 (socket says 172.21.15.3:39316) 2024-03-21T08:33:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 -- 172.21.15.3:0/2169428631 learned_addr learned my addr 172.21.15.3:0/2169428631 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 -- 172.21.15.3:0/2169428631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 msgr2=0x55eaa2078ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa2078ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 -- 172.21.15.3:0/2169428631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 msgr2=0x55eaa2076180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2076180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51d4e00640 1 -- 172.21.15.3:0/2169428631 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa1f922c0 con 0x55eaa206cfd0 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.316+0000 7f51cfe00640 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2076180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.317+0000 7f51d4e00640 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa206d3b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f51d006ad70 tx=0x7f51c80056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae82011cf844185 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.317+0000 7f51c5600640 1 -- 172.21.15.3:0/2169428631 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8039040 con 0x55eaa206cfd0 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.317+0000 7f51cd600640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51b40a31d0 0x7f51b40a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.317+0000 7f51c5600640 1 -- 172.21.15.3:0/2169428631 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51c8004030 con 0x55eaa206cfd0 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.317+0000 7f51c5600640 1 -- 172.21.15.3:0/2169428631 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c802ca40 con 0x55eaa206cfd0 2024-03-21T08:33:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.317+0000 7f51cd600640 1 -- 172.21.15.3:0/2087865372 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f51b80cf700 con 0x7f51d0074360 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.318+0000 7f51cfe00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51b40a31d0 0x7f51b40a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 -- 172.21.15.3:0/2169428631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 msgr2=0x55eaa206d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa206d3b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f51d006ad70 tx=0x7f51c80056d0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 -- 172.21.15.3:0/2169428631 shutdown_connections 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa2078ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2076180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2169428631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa206d3b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 -- 172.21.15.3:0/2169428631 >> 172.21.15.3:0/2169428631 conn(0x55eaa20678f0 msgr2=0x55eaa2067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 -- 172.21.15.3:0/2169428631 shutdown_connections 2024-03-21T08:33:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51cfe00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51b40a31d0 0x7f51b40a5690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55eaa206ca80 tx=0x7f51c00097c0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.319+0000 7f51d9b75740 1 -- 172.21.15.3:0/2169428631 wait complete. 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.320+0000 7f51d9b75740 1 -- start start 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.320+0000 7f51d9b75740 1 Processor -- start 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206cfd0 0x55eaa2073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa20766c0 0x55eaa20741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d9b75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa1f8da20 con 0x55eaa206d8f0 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d9b75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa1fcd8d0 con 0x55eaa20766c0 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d9b75740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2079290 con 0x55eaa206cfd0 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d9b75740 10 client.0 _ll_register_callbacks cb 0x55eaa1fca1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa20766c0 0x55eaa20741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa20766c0 0x55eaa20741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:39330/0 (socket says 172.21.15.3:39330) 2024-03-21T08:33:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51d4e00640 1 -- 172.21.15.3:0/2945928842 learned_addr learned my addr 172.21.15.3:0/2945928842 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51c7400640 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.321+0000 7f51cfe00640 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206cfd0 0x55eaa2073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.322+0000 7f51d4e00640 1 -- 172.21.15.3:0/2945928842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206cfd0 msgr2=0x55eaa2073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.322+0000 7f51d4e00640 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206cfd0 0x55eaa2073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.322+0000 7f51d4e00640 1 -- 172.21.15.3:0/2945928842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 msgr2=0x55eaa2073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.322+0000 7f51d4e00640 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.322+0000 7f51d4e00640 1 -- 172.21.15.3:0/2945928842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaa1ce6210 con 0x55eaa20766c0 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.322+0000 7f51d4e00640 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa20766c0 0x55eaa20741d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51c8037040 tx=0x7f51c8005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4436ef44d45283de server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.322+0000 7f51be000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.323+0000 7f51be000640 1 -- 172.21.15.3:0/2945928842 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8039040 con 0x55eaa20766c0 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.323+0000 7f51be000640 1 -- 172.21.15.3:0/2945928842 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f51c802c7a0 con 0x55eaa20766c0 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.323+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51c8004030 con 0x55eaa20766c0 2024-03-21T08:33:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.323+0000 7f51be000640 1 -- 172.21.15.3:0/2945928842 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c805c050 con 0x55eaa20766c0 2024-03-21T08:33:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.323+0000 7f51d9b75740 10 client.35945 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.323+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55eaa1db2620 con 0x55eaa20766c0 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51be000640 1 -- 172.21.15.3:0/2945928842 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51c8004030 con 0x55eaa20766c0 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51be000640 1 -- 172.21.15.3:0/2945928842 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f51c8053480 con 0x55eaa20766c0 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51d9b75740 10 client.35945 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51d9b75740 10 client.35945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51d9b75740 20 client.35945 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa2086590 0x55eaa2088980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51d9b75740 4 client.35945 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.324+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaa1fded70 con 0x55eaa2086590 2024-03-21T08:33:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.325+0000 7f51cfe00640 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa2086590 0x55eaa2088980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.325+0000 7f51cfe00640 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa2086590 0x55eaa2088980 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9aaca2590d26afa server_cookie=ad92f1235de19dff in_seq=0 out_seq=0 2024-03-21T08:33:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.326+0000 7f51be000640 10 client.35945 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51be000640 1 -- 172.21.15.3:0/2945928842 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eaa1fded70 con 0x55eaa2086590 2024-03-21T08:33:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51be000640 10 client.35945 handle_command_reply: tid=0 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 client.35945 shutdown 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa20766c0 msgr2=0x55eaa20741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa20766c0 0x55eaa20741d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51c8037040 tx=0x7f51c8005b20 comp rx=0 tx=0).stop 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 shutdown_connections 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa20766c0 0x55eaa20741d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa2086590 0x55eaa2088980 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 --2- 172.21.15.3:0/2945928842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206cfd0 0x55eaa2073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 >> 172.21.15.3:0/2945928842 conn(0x55eaa20678f0 msgr2=0x55eaa2069640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51be000640 0 client.35945 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 shutdown_connections 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 1 -- 172.21.15.3:0/2945928842 wait complete. 2024-03-21T08:33:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.327+0000 7f51d9b75740 20 client.35945 trim_cache size 0 max 16384 2024-03-21T08:33:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.347+0000 7f51d9b75740 1 Processor -- start 2024-03-21T08:33:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.347+0000 7f51d9b75740 1 -- start start 2024-03-21T08:33:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa2075e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa20709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51d9b75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01b7770 con 0x55eaa206d8f0 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51d9b75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01b5240 con 0x55eaa206cfd0 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51d9b75740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01b32e0 con 0x55eaa20766c0 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58076/0 (socket says 172.21.15.3:58076) 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51c7400640 1 -- 172.21.15.3:0/1194505939 learned_addr learned my addr 172.21.15.3:0/1194505939 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51cfe00640 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa2075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.348+0000 7f51d4e00640 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa20709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c7400640 1 -- 172.21.15.3:0/1194505939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 msgr2=0x55eaa20709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c7400640 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa20709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c7400640 1 -- 172.21.15.3:0/1194505939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 msgr2=0x55eaa2075e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c7400640 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa2075e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c7400640 1 -- 172.21.15.3:0/1194505939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa2078e20 con 0x55eaa206d8f0 2024-03-21T08:33:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51d4e00640 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa20709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51cfe00640 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa2075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c7400640 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073ff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f51b809b600 tx=0x7f51b809b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d5def574f063f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c4c00640 1 -- 172.21.15.3:0/1194505939 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8020040 con 0x55eaa206d8f0 2024-03-21T08:33:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c4c00640 1 -- 172.21.15.3:0/1194505939 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51b809b670 con 0x55eaa206d8f0 2024-03-21T08:33:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.349+0000 7f51c4c00640 1 -- 172.21.15.3:0/1194505939 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80ccc50 con 0x55eaa206d8f0 2024-03-21T08:33:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 -- 172.21.15.3:0/1194505939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 msgr2=0x55eaa2073ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073ff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f51b809b600 tx=0x7f51b809b090 comp rx=0 tx=0).stop 2024-03-21T08:33:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 -- 172.21.15.3:0/1194505939 shutdown_connections 2024-03-21T08:33:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa20766c0 0x55eaa20709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa206d8f0 0x55eaa2073ff0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1194505939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa2075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 -- 172.21.15.3:0/1194505939 >> 172.21.15.3:0/1194505939 conn(0x55eaa20678f0 msgr2=0x55eaa2081d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 -- 172.21.15.3:0/1194505939 shutdown_connections 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.350+0000 7f51d9b75740 1 -- 172.21.15.3:0/1194505939 wait complete. 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.351+0000 7f51d9b75740 1 -- start start 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.351+0000 7f51d9b75740 1 Processor -- start 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa208b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206d8f0 0x55eaa20899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d9b75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa20766c0 0x55eaa2089ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d9b75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01b38e0 con 0x55eaa20766c0 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d9b75740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01b4be0 con 0x55eaa206cfd0 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d9b75740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01b3f20 con 0x55eaa206d8f0 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d9b75740 10 client.0 _ll_register_callbacks cb 0x7f51c0009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa20766c0 0x55eaa2089ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa208b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa20766c0 0x55eaa2089ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58080/0 (socket says 172.21.15.3:58080) 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa208b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:39350/0 (socket says 172.21.15.3:39350) 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d4e00640 1 -- 172.21.15.3:0/1219547115 learned_addr learned my addr 172.21.15.3:0/1219547115 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51c7400640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206d8f0 0x55eaa20899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d4e00640 1 -- 172.21.15.3:0/1219547115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206d8f0 msgr2=0x55eaa20899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.352+0000 7f51d4e00640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206d8f0 0x55eaa20899a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51d4e00640 1 -- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 msgr2=0x55eaa208b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51d4e00640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa208b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51d4e00640 1 -- 172.21.15.3:0/1219547115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaa208a550 con 0x55eaa20766c0 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51c7400640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206d8f0 0x55eaa20899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51cfe00640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa208b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51d4e00640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa20766c0 0x55eaa2089ee0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009230 tx=0x7f51c8004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7343a3936e43819d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51be000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51be000640 1 -- 172.21.15.3:0/1219547115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8039050 con 0x55eaa20766c0 2024-03-21T08:33:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51be000640 1 -- 172.21.15.3:0/1219547115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f51c8053480 con 0x55eaa20766c0 2024-03-21T08:33:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.353+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51b809b670 con 0x55eaa20766c0 2024-03-21T08:33:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51be000640 1 -- 172.21.15.3:0/1219547115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c8053680 con 0x55eaa20766c0 2024-03-21T08:33:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 10 client.16218 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55eaa20e2900 con 0x55eaa20766c0 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51be000640 1 -- 172.21.15.3:0/1219547115 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51b809b670 con 0x55eaa20766c0 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51be000640 1 -- 172.21.15.3:0/1219547115 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f51c8004980 con 0x55eaa20766c0 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 10 client.16218 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 20 client.16218 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa20e2eb0 0x55eaa20e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 4 client.16218 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.354+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eaa1fd90c0 con 0x55eaa20e2eb0 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.355+0000 7f51cfe00640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa20e2eb0 0x55eaa20e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.356+0000 7f51cfe00640 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa20e2eb0 0x55eaa20e52d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f96a155b64c4a1 server_cookie=51f8bbcffa0314ce in_seq=0 out_seq=0 2024-03-21T08:33:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:39.356+0000 7f51be000640 10 client.16218 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:40.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:39 smithi084 ceph-mon[35477]: pgmap v57: 129 pgs: 129 active+clean; 14 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 24 op/s 2024-03-21T08:33:40.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:39 smithi171 ceph-mon[35388]: pgmap v57: 129 pgs: 129 active+clean; 14 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 24 op/s 2024-03-21T08:33:40.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:39 smithi003 ceph-mon[31675]: pgmap v57: 129 pgs: 129 active+clean; 14 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 24 op/s 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51be000640 1 -- 172.21.15.3:0/1219547115 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eaa1fd90c0 con 0x55eaa20e2eb0 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51be000640 10 client.16218 handle_command_reply: tid=0 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 client.16218 shutdown 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa20766c0 msgr2=0x55eaa2089ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa20766c0 0x55eaa2089ee0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009230 tx=0x7f51c8004560 comp rx=0 tx=0).stop 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 shutdown_connections 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eaa20766c0 0x55eaa2089ee0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55eaa20e2eb0 0x55eaa20e52d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaa206d8f0 0x55eaa20899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 --2- 172.21.15.3:0/1219547115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eaa206cfd0 0x55eaa208b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.844+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 >> 172.21.15.3:0/1219547115 conn(0x55eaa20678f0 msgr2=0x55eaa20660c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.845+0000 7f51be000640 0 client.16218 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.845+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 shutdown_connections 2024-03-21T08:33:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.845+0000 7f51d9b75740 1 -- 172.21.15.3:0/1219547115 wait complete. 2024-03-21T08:33:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.845+0000 7f51d9b75740 20 client.16218 trim_cache size 0 max 16384 2024-03-21T08:33:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:33:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:33:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:33:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:33:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51b40a31d0 msgr2=0x7f51b40a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51b40a31d0 0x7f51b40a5690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55eaa206ca80 tx=0x7f51c00097c0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0074360 msgr2=0x7f51d019e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0074360 0x7f51d019e9d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51b800d950 tx=0x7f51b800de10 comp rx=0 tx=0).stop 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 shutdown_connections 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51d0112660 0x7f51d01a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d0074360 0x7f51d019e9d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51b40a31d0 0x7f51b40a5690 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 --2- 172.21.15.3:0/2087865372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d0073a40 0x7f51d019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 >> 172.21.15.3:0/2087865372 conn(0x7f51d00fd120 msgr2=0x7f51d00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.846+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 shutdown_connections 2024-03-21T08:33:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:40.847+0000 7f51d6e00640 1 -- 172.21.15.3:0/2087865372 wait complete. 2024-03-21T08:33:42.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:41 smithi084 ceph-mon[35477]: pgmap v58: 129 pgs: 129 active+clean; 14 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-21T08:33:42.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:41 smithi171 ceph-mon[35388]: pgmap v58: 129 pgs: 129 active+clean; 14 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-21T08:33:42.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:41 smithi003 ceph-mon[31675]: pgmap v58: 129 pgs: 129 active+clean; 14 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-21T08:33:44.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:43 smithi084 ceph-mon[35477]: pgmap v59: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-21T08:33:44.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:43 smithi171 ceph-mon[35388]: pgmap v59: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-21T08:33:44.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:43 smithi003 ceph-mon[31675]: pgmap v59: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-21T08:33:46.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:45 smithi084 ceph-mon[35477]: pgmap v60: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-21T08:33:46.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:45 smithi171 ceph-mon[35388]: pgmap v60: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-21T08:33:46.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:45 smithi003 ceph-mon[31675]: pgmap v60: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-21T08:33:48.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:47 smithi084 ceph-mon[35477]: pgmap v61: 129 pgs: 129 active+clean; 19 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-21T08:33:48.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:47 smithi171 ceph-mon[35388]: pgmap v61: 129 pgs: 129 active+clean; 19 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-21T08:33:48.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:47 smithi003 ceph-mon[31675]: pgmap v61: 129 pgs: 129 active+clean; 19 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-21T08:33:50.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:49 smithi171 ceph-mon[35388]: pgmap v62: 129 pgs: 129 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-03-21T08:33:50.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:49 smithi003 ceph-mon[31675]: pgmap v62: 129 pgs: 129 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-03-21T08:33:50.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:49 smithi084 ceph-mon[35477]: pgmap v62: 129 pgs: 129 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-03-21T08:33:50.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:33:50.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:33:50.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 -- 172.21.15.3:0/3660957060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 msgr2=0x7f2970103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 --2- 172.21.15.3:0/3660957060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f2970103cc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f295c009890 tx=0x7f295c02bb70 comp rx=0 tx=0).stop 2024-03-21T08:33:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 -- 172.21.15.3:0/3660957060 shutdown_connections 2024-03-21T08:33:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 --2- 172.21.15.3:0/3660957060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2970104200 0x7f2970110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 --2- 172.21.15.3:0/3660957060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f2970103cc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 --2- 172.21.15.3:0/3660957060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2970109850 0x7f2970109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 -- 172.21.15.3:0/3660957060 >> 172.21.15.3:0/3660957060 conn(0x7f29700ff2f0 msgr2=0x7f2970101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 -- 172.21.15.3:0/3660957060 shutdown_connections 2024-03-21T08:33:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.635+0000 7f2977200640 1 -- 172.21.15.3:0/3660957060 wait complete. 2024-03-21T08:33:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.636+0000 7f2977200640 1 Processor -- start 2024-03-21T08:33:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f2977200640 1 -- start start 2024-03-21T08:33:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f2977200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f297019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f2977200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2970104200 0x7f297019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f2977200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2970109850 0x7f297019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f2977200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2970113b10 con 0x7f2970103840 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f2977200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2970113990 con 0x7f2970104200 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f2977200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2970113c90 con 0x7f2970109850 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f296fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f297019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f296fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f297019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47426/0 (socket says 172.21.15.3:47426) 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f296fe00640 1 -- 172.21.15.3:0/3284365266 learned_addr learned my addr 172.21.15.3:0/3284365266 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.637+0000 7f296f400640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2970104200 0x7f297019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f2975200640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2970109850 0x7f297019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f296fe00640 1 -- 172.21.15.3:0/3284365266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2970109850 msgr2=0x7f297019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f296fe00640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2970109850 0x7f297019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f296fe00640 1 -- 172.21.15.3:0/3284365266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2970104200 msgr2=0x7f297019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f296fe00640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2970104200 0x7f297019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f296fe00640 1 -- 172.21.15.3:0/3284365266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297019f090 con 0x7f2970103840 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f296f400640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2970104200 0x7f297019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.638+0000 7f296fe00640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f297019a0a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f296000e9c0 tx=0x7f296000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.639+0000 7f296cc00640 1 -- 172.21.15.3:0/3284365266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296000cde0 con 0x7f2970103840 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.639+0000 7f296cc00640 1 -- 172.21.15.3:0/3284365266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29600035f0 con 0x7f2970103840 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.639+0000 7f296cc00640 1 -- 172.21.15.3:0/3284365266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960010690 con 0x7f2970103840 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.639+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297019f380 con 0x7f2970103840 2024-03-21T08:33:50.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.639+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29701a6c10 con 0x7f2970103840 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.643+0000 7f2979e44740 1 Processor -- start 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.643+0000 7f2979e44740 1 -- start start 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.643+0000 7f296cc00640 1 -- 172.21.15.3:0/3284365266 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f29600037a0 con 0x7f2970103840 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2979e44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596222c00 0x562596221950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2979e44740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x5625962223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2979e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56259622b290 0x56259622d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2979e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562596186250 con 0x56259622b290 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2979e44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259612ea30 con 0x562596222c00 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2979e44740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625961822f0 con 0x562596221f20 2024-03-21T08:33:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2975200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596222c00 0x562596221950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f296f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56259622b290 0x56259622d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2975200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596222c00 0x562596221950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55806/0 (socket says 172.21.15.3:55806) 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f2975200640 1 -- 172.21.15.3:0/2341888856 learned_addr learned my addr 172.21.15.3:0/2341888856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.644+0000 7f296fe00640 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x5625962223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f2975200640 1 -- 172.21.15.3:0/2341888856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 msgr2=0x5625962223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f2975200640 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x5625962223a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f2975200640 1 -- 172.21.15.3:0/2341888856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56259622b290 msgr2=0x56259622d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f2975200640 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56259622b290 0x56259622d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f2975200640 1 -- 172.21.15.3:0/2341888856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562596119690 con 0x562596222c00 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f296f400640 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56259622b290 0x56259622d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f2975200640 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596222c00 0x562596221950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fc0 tx=0x7f2968010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831b1f24c6c90013 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.645+0000 7f296cc00640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f29540a3090 0x7f29540a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.646+0000 7f2965600640 1 -- 172.21.15.3:0/2341888856 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968003b70 con 0x562596222c00 2024-03-21T08:33:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.646+0000 7f2965600640 1 -- 172.21.15.3:0/2341888856 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2968003d10 con 0x562596222c00 2024-03-21T08:33:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.646+0000 7f296fe00640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f29540a3090 0x7f29540a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.646+0000 7f296cc00640 1 -- 172.21.15.3:0/3284365266 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f29600cff60 con 0x7f2970103840 2024-03-21T08:33:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.646+0000 7f2965600640 1 -- 172.21.15.3:0/2341888856 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968011a40 con 0x562596222c00 2024-03-21T08:33:50.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f296fe00640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f29540a3090 0x7f29540a5550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x562596223150 tx=0x7f2960000bc0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 -- 172.21.15.3:0/2341888856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596222c00 msgr2=0x562596221950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596222c00 0x562596221950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fc0 tx=0x7f2968010860 comp rx=0 tx=0).stop 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 -- 172.21.15.3:0/2341888856 shutdown_connections 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56259622b290 0x56259622d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x5625962223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 --2- 172.21.15.3:0/2341888856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596222c00 0x562596221950 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 -- 172.21.15.3:0/2341888856 >> 172.21.15.3:0/2341888856 conn(0x5625961e9200 msgr2=0x56259621c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 -- 172.21.15.3:0/2341888856 shutdown_connections 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.647+0000 7f2979e44740 1 -- 172.21.15.3:0/2341888856 wait complete. 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.648+0000 7f2979e44740 1 -- start start 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.648+0000 7f2979e44740 1 Processor -- start 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.648+0000 7f2979e44740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x56259622a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2979e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x562596227ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2979e44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56259622b290 0x562596228220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2979e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259622e100 con 0x562596222c00 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2979e44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x562596141e80 con 0x56259622b290 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2979e44740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259622e280 con 0x562596221f20 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2979e44740 10 client.0 _ll_register_callbacks cb 0x562596154130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f296f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x562596227ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2975200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56259622b290 0x562596228220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f296f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x562596227ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47440/0 (socket says 172.21.15.3:47440) 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f296f400640 1 -- 172.21.15.3:0/2826135323 learned_addr learned my addr 172.21.15.3:0/2826135323 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.649+0000 7f2975200640 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x56259622a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f296f400640 1 -- 172.21.15.3:0/2826135323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 msgr2=0x56259622a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f296f400640 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x56259622a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f296f400640 1 -- 172.21.15.3:0/2826135323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56259622b290 msgr2=0x562596228220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f296f400640 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56259622b290 0x562596228220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f296f400640 1 -- 172.21.15.3:0/2826135323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562595f647d0 con 0x562596222c00 2024-03-21T08:33:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f2975200640 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x56259622a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f296f400640 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x562596227ce0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c0b0 tx=0x7f295c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8357ff95ff3bf5a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.650+0000 7f295a000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.651+0000 7f295a000640 1 -- 172.21.15.3:0/2826135323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c039040 con 0x562596222c00 2024-03-21T08:33:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.651+0000 7f295a000640 1 -- 172.21.15.3:0/2826135323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f295c034d90 con 0x562596222c00 2024-03-21T08:33:50.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.651+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2968003d10 con 0x562596222c00 2024-03-21T08:33:50.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.651+0000 7f295a000640 1 -- 172.21.15.3:0/2826135323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f295c02c510 con 0x562596222c00 2024-03-21T08:33:50.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.651+0000 7f2979e44740 10 client.16230 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:50.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.651+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562595e9a3f0 con 0x562596222c00 2024-03-21T08:33:50.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.651+0000 7f295a000640 1 -- 172.21.15.3:0/2826135323 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2968003d10 con 0x562596222c00 2024-03-21T08:33:50.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.652+0000 7f295a000640 1 -- 172.21.15.3:0/2826135323 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f295c050020 con 0x562596222c00 2024-03-21T08:33:50.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.652+0000 7f2979e44740 10 client.16230 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.652+0000 7f2979e44740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.652+0000 7f2979e44740 20 client.16230 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.652+0000 7f2979e44740 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56259623b210 0x56259623d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.652+0000 7f2979e44740 4 client.16230 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.652+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562596193790 con 0x56259623b210 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.653+0000 7f2975200640 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56259623b210 0x56259623d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.654+0000 7f2975200640 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56259623b210 0x56259623d600 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5d5b6a02c5737a server_cookie=db167a26d504b57e in_seq=0 out_seq=0 2024-03-21T08:33:50.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.654+0000 7f295a000640 10 client.16230 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:50.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.655+0000 7f295a000640 1 -- 172.21.15.3:0/2826135323 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562596193790 con 0x56259623b210 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.655+0000 7f295a000640 10 client.16230 handle_command_reply: tid=0 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.655+0000 7f2979e44740 1 client.16230 shutdown 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 msgr2=0x562596227ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x562596227ce0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c0b0 tx=0x7f295c0045f0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 shutdown_connections 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56259622b290 0x562596228220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x562596227ce0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56259623b210 0x56259623d600 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 --2- 172.21.15.3:0/2826135323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562596221f20 0x56259622a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 >> 172.21.15.3:0/2826135323 conn(0x5625961e9200 msgr2=0x56259621e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f295a000640 0 client.16230 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 shutdown_connections 2024-03-21T08:33:50.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.656+0000 7f2979e44740 1 -- 172.21.15.3:0/2826135323 wait complete. 2024-03-21T08:33:50.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.657+0000 7f2979e44740 20 client.16230 trim_cache size 0 max 16384 2024-03-21T08:33:50.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.681+0000 7f2979e44740 1 Processor -- start 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.681+0000 7f2979e44740 1 -- start start 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f2979e44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596221f20 0x56259622a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f2979e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x56259622a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f2979e44740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x56259622c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f2979e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701b3350 con 0x562596222c00 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f2979e44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701b0e20 con 0x562596221f20 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f2979e44740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701aeec0 con 0x56259622b290 2024-03-21T08:33:50.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f296f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x56259622a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f296f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x56259622a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47448/0 (socket says 172.21.15.3:47448) 2024-03-21T08:33:50.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f296f400640 1 -- 172.21.15.3:0/3701627855 learned_addr learned my addr 172.21.15.3:0/3701627855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:50.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.682+0000 7f2975200640 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596221f20 0x56259622a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f2975200640 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x56259622c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f296f400640 1 -- 172.21.15.3:0/3701627855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 msgr2=0x56259622c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f296f400640 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x56259622c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f296f400640 1 -- 172.21.15.3:0/3701627855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596221f20 msgr2=0x56259622a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f296f400640 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596221f20 0x56259622a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f296f400640 1 -- 172.21.15.3:0/3701627855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56259623d970 con 0x562596222c00 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f2975200640 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x56259622c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.683+0000 7f296f400640 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x56259622a950 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f295c02bbb0 tx=0x7f295c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c3af71bb598738 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.684+0000 7f2964c00640 1 -- 172.21.15.3:0/3701627855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c039040 con 0x562596222c00 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.684+0000 7f2964c00640 1 -- 172.21.15.3:0/3701627855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f295c055020 con 0x562596222c00 2024-03-21T08:33:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.684+0000 7f2964c00640 1 -- 172.21.15.3:0/3701627855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c003ea0 con 0x562596222c00 2024-03-21T08:33:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 -- 172.21.15.3:0/3701627855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 msgr2=0x56259622a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x56259622a950 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f295c02bbb0 tx=0x7f295c005dc0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 -- 172.21.15.3:0/3701627855 shutdown_connections 2024-03-21T08:33:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x56259622c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596222c00 0x56259622a950 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 --2- 172.21.15.3:0/3701627855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596221f20 0x56259622a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 -- 172.21.15.3:0/3701627855 >> 172.21.15.3:0/3701627855 conn(0x5625961e9200 msgr2=0x56259621c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.685+0000 7f2979e44740 1 -- 172.21.15.3:0/3701627855 shutdown_connections 2024-03-21T08:33:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.686+0000 7f2979e44740 1 -- 172.21.15.3:0/3701627855 wait complete. 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.686+0000 7f2979e44740 1 -- start start 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.686+0000 7f2979e44740 1 Processor -- start 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2979e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596221f20 0x562596230510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2979e44740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x562596230a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2979e44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596225ac0 0x562596230f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2979e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701af4c0 con 0x562596221f20 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2979e44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701b07c0 con 0x562596225ac0 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2979e44740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701afb00 con 0x56259622b290 2024-03-21T08:33:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2979e44740 10 client.0 _ll_register_callbacks cb 0x7f29680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2975200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596221f20 0x562596230510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2975200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596221f20 0x562596230510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47462/0 (socket says 172.21.15.3:47462) 2024-03-21T08:33:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f2975200640 1 -- 172.21.15.3:0/1528420373 learned_addr learned my addr 172.21.15.3:0/1528420373 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.687+0000 7f296f400640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x562596230a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f2975200640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596225ac0 0x562596230f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f2975200640 1 -- 172.21.15.3:0/1528420373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 msgr2=0x562596230a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f2975200640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x562596230a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f2975200640 1 -- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596225ac0 msgr2=0x562596230f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f2975200640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596225ac0 0x562596230f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f2975200640 1 -- 172.21.15.3:0/1528420373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562595e5a0f0 con 0x562596221f20 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f296f400640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x562596230a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f2975200640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596221f20 0x562596230510 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2968011570 tx=0x7f2968013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9d92e5d1bae7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.688+0000 7f295a000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.689+0000 7f295a000640 1 -- 172.21.15.3:0/1528420373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968004bc0 con 0x562596221f20 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.689+0000 7f295a000640 1 -- 172.21.15.3:0/1528420373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2968023040 con 0x562596221f20 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.689+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f295c055020 con 0x562596221f20 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.689+0000 7f295a000640 1 -- 172.21.15.3:0/1528420373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f296803d050 con 0x562596221f20 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.689+0000 7f2979e44740 10 client.16236 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.689+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562596297d30 con 0x562596221f20 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f295a000640 1 -- 172.21.15.3:0/1528420373 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f295c055020 con 0x562596221f20 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f295a000640 1 -- 172.21.15.3:0/1528420373 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f2968029040 con 0x562596221f20 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f2979e44740 10 client.16236 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f2979e44740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f2979e44740 20 client.16236 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f2979e44740 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562596298280 0x56259629a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f2979e44740 4 client.16236 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:33:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.690+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56259618dae0 con 0x562596298280 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.691+0000 7f296f400640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562596298280 0x56259629a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.691+0000 7f296f400640 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562596298280 0x56259629a6a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b00739109470573 server_cookie=4b225ce84d7ae6ba in_seq=0 out_seq=0 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.691+0000 7f295a000640 10 client.16236 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f295a000640 1 -- 172.21.15.3:0/1528420373 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56259618dae0 con 0x562596298280 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f295a000640 10 client.16236 handle_command_reply: tid=0 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 client.16236 shutdown 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596221f20 msgr2=0x562596230510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596221f20 0x562596230510 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2968011570 tx=0x7f2968013040 comp rx=0 tx=0).stop 2024-03-21T08:33:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 shutdown_connections 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562596225ac0 0x562596230f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562596298280 0x56259629a6a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56259622b290 0x562596230a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 --2- 172.21.15.3:0/1528420373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562596221f20 0x562596230510 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 >> 172.21.15.3:0/1528420373 conn(0x5625961e9200 msgr2=0x56259621b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f295a000640 0 client.16236 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 shutdown_connections 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.692+0000 7f2979e44740 1 -- 172.21.15.3:0/1528420373 wait complete. 2024-03-21T08:33:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.693+0000 7f2979e44740 20 client.16236 trim_cache size 0 max 16384 2024-03-21T08:33:50.698 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:33:50.698 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-03-21T08:33:50.698 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-03-21T08:33:50.698 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.693+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f29540a3090 msgr2=0x7f29540a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.693+0000 7f2977200640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f29540a3090 0x7f29540a5550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x562596223150 tx=0x7f2960000bc0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 msgr2=0x7f297019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f297019a0a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f296000e9c0 tx=0x7f296000ee80 comp rx=0 tx=0).stop 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 shutdown_connections 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f29540a3090 0x7f29540a5550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2970109850 0x7f297019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2970104200 0x7f297019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 --2- 172.21.15.3:0/3284365266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2970103840 0x7f297019a0a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 >> 172.21.15.3:0/3284365266 conn(0x7f29700ff2f0 msgr2=0x7f29700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 shutdown_connections 2024-03-21T08:33:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.694+0000 7f2977200640 1 -- 172.21.15.3:0/3284365266 wait complete. 2024-03-21T08:33:50.716 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T08:33:50.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:33:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 -- 172.21.15.3:0/2509812916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 msgr2=0x7fd140103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 --2- 172.21.15.3:0/2509812916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140103bc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009900 tx=0x7fd12c02bc40 comp rx=0 tx=0).stop 2024-03-21T08:33:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 -- 172.21.15.3:0/2509812916 shutdown_connections 2024-03-21T08:33:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 --2- 172.21.15.3:0/2509812916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd140104100 0x7fd140110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 --2- 172.21.15.3:0/2509812916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140103bc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 --2- 172.21.15.3:0/2509812916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd140109750 0x7fd140109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 -- 172.21.15.3:0/2509812916 >> 172.21.15.3:0/2509812916 conn(0x7fd1400ff210 msgr2=0x7fd140101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.945+0000 7fd146c00640 1 -- 172.21.15.3:0/2509812916 shutdown_connections 2024-03-21T08:33:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.946+0000 7fd146c00640 1 -- 172.21.15.3:0/2509812916 wait complete. 2024-03-21T08:33:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 Processor -- start 2024-03-21T08:33:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 -- start start 2024-03-21T08:33:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd140104100 0x7fd14019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd140109750 0x7fd14019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1401139d0 con 0x7fd140109750 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140113850 con 0x7fd140103740 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.947+0000 7fd146c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140113b50 con 0x7fd140104100 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.948+0000 7fd145800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.948+0000 7fd145800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55854/0 (socket says 172.21.15.3:55854) 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.948+0000 7fd145800640 1 -- 172.21.15.3:0/1046646257 learned_addr learned my addr 172.21.15.3:0/1046646257 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:50.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.948+0000 7fd146200640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd140109750 0x7fd14019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.948+0000 7fd144e00640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd140104100 0x7fd14019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146200640 1 -- 172.21.15.3:0/1046646257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd140104100 msgr2=0x7fd14019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146200640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd140104100 0x7fd14019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146200640 1 -- 172.21.15.3:0/1046646257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 msgr2=0x7fd140199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146200640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146200640 1 -- 172.21.15.3:0/1046646257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14019ef40 con 0x7fd140109750 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd145800640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146200640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd140109750 0x7fd14019e810 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd13800ec50 tx=0x7fd13800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd13e000640 1 -- 172.21.15.3:0/1046646257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138003b70 con 0x7fd140109750 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14019f230 con 0x7fd140109750 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd13e000640 1 -- 172.21.15.3:0/1046646257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd138003d10 con 0x7fd140109750 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd13e000640 1 -- 172.21.15.3:0/1046646257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138010760 con 0x7fd140109750 2024-03-21T08:33:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.949+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1401a6ac0 con 0x7fd140109750 2024-03-21T08:33:50.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.951+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14004f470 con 0x7fd140109750 2024-03-21T08:33:50.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.954+0000 7fd13e000640 1 -- 172.21.15.3:0/1046646257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fd138004020 con 0x7fd140109750 2024-03-21T08:33:50.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.956+0000 7fd13e000640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd1280a3100 0x7fd1280a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:50.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.956+0000 7fd13e000640 1 -- 172.21.15.3:0/1046646257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fd1380d01a0 con 0x7fd140109750 2024-03-21T08:33:50.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.956+0000 7fd145800640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd1280a3100 0x7fd1280a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:50.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.957+0000 7fd145800640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd1280a3100 0x7fd1280a55c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd134011440 tx=0x7fd134005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:50.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:50.962+0000 7fd13e000640 1 -- 172.21.15.3:0/1046646257 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd13809c810 con 0x7fd140109750 2024-03-21T08:33:51.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.311+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd14010bd00 con 0x7fd140109750 2024-03-21T08:33:51.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.313+0000 7fd13e000640 1 -- 172.21.15.3:0/1046646257 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11393 (secure 0 0 0) 0x7fd13809c110 con 0x7fd140109750 2024-03-21T08:33:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:33:51.314 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:33:37.042609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34409,34472]},"id":1}]} 2024-03-21T08:33:51.315 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd1280a3100 msgr2=0x7fd1280a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd1280a3100 0x7fd1280a55c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd134011440 tx=0x7fd134005f90 comp rx=0 tx=0).stop 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd140109750 msgr2=0x7fd14019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd140109750 0x7fd14019e810 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd13800ec50 tx=0x7fd13800c6a0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 shutdown_connections 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd140109750 0x7fd14019e810 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fd1280a3100 0x7fd1280a55c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd140104100 0x7fd14019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 --2- 172.21.15.3:0/1046646257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd140103740 0x7fd140199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 >> 172.21.15.3:0/1046646257 conn(0x7fd1400ff210 msgr2=0x7fd1400ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 shutdown_connections 2024-03-21T08:33:51.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.316+0000 7fd146c00640 1 -- 172.21.15.3:0/1046646257 wait complete. 2024-03-21T08:33:51.339 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:33:51.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T08:33:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 -- 172.21.15.3:0/1887100498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904109690 msgr2=0x7f1904109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 --2- 172.21.15.3:0/1887100498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904109690 0x7f1904109a70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009930 tx=0x7f18f802bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 -- 172.21.15.3:0/1887100498 shutdown_connections 2024-03-21T08:33:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 --2- 172.21.15.3:0/1887100498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1904104040 0x7f1904110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 --2- 172.21.15.3:0/1887100498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1904103680 0x7f1904103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 --2- 172.21.15.3:0/1887100498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904109690 0x7f1904109a70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 -- 172.21.15.3:0/1887100498 >> 172.21.15.3:0/1887100498 conn(0x7f19040ff110 msgr2=0x7f1904101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 -- 172.21.15.3:0/1887100498 shutdown_connections 2024-03-21T08:33:51.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.572+0000 7f190ae00640 1 -- 172.21.15.3:0/1887100498 wait complete. 2024-03-21T08:33:51.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.574+0000 7f190ae00640 1 Processor -- start 2024-03-21T08:33:51.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.574+0000 7f190ae00640 1 -- start start 2024-03-21T08:33:51.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f190ae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1904103680 0x7f1904199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f190ae00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904104040 0x7f190419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f190ae00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1904109690 0x7f190419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f190ae00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1904113960 con 0x7f1904104040 2024-03-21T08:33:51.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f190ae00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041137e0 con 0x7f1904109690 2024-03-21T08:33:51.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f190ae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1904113ae0 con 0x7f1904103680 2024-03-21T08:33:51.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f1909000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904104040 0x7f190419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f190a400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1904109690 0x7f190419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f1909000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904104040 0x7f190419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47496/0 (socket says 172.21.15.3:47496) 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f1909000640 1 -- 172.21.15.3:0/1157239622 learned_addr learned my addr 172.21.15.3:0/1157239622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.575+0000 7f1909a00640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1904103680 0x7f1904199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.576+0000 7f190a400640 1 -- 172.21.15.3:0/1157239622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1904103680 msgr2=0x7f1904199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.576+0000 7f190a400640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1904103680 0x7f1904199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.576+0000 7f190a400640 1 -- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904104040 msgr2=0x7f190419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.576+0000 7f190a400640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904104040 0x7f190419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.576+0000 7f190a400640 1 -- 172.21.15.3:0/1157239622 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190419eed0 con 0x7f1904109690 2024-03-21T08:33:51.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.576+0000 7f190a400640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1904109690 0x7f190419e7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009fd0 tx=0x7f18fc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.577+0000 7f1902000640 1 -- 172.21.15.3:0/1157239622 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00ce20 con 0x7f1904109690 2024-03-21T08:33:51.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.577+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190419f1c0 con 0x7f1904109690 2024-03-21T08:33:51.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.577+0000 7f1902000640 1 -- 172.21.15.3:0/1157239622 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f18fc005020 con 0x7f1904109690 2024-03-21T08:33:51.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.577+0000 7f1902000640 1 -- 172.21.15.3:0/1157239622 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc010690 con 0x7f1904109690 2024-03-21T08:33:51.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.577+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19041a6a50 con 0x7f1904109690 2024-03-21T08:33:51.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.581+0000 7f190f030740 1 Processor -- start 2024-03-21T08:33:51.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.581+0000 7f1902000640 1 -- 172.21.15.3:0/1157239622 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f18fc0051d0 con 0x7f1904109690 2024-03-21T08:33:51.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.581+0000 7f190f030740 1 -- start start 2024-03-21T08:33:51.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f190f030740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f669658b0 0x560f66965c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f190f030740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f66966260 0x560f6696e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f190f030740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 0x560f66971230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f190f030740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f668c92c0 con 0x560f669658b0 2024-03-21T08:33:51.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f190f030740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f66871c90 con 0x560f66966260 2024-03-21T08:33:51.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f190f030740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f668c5360 con 0x560f6696ede0 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f1909a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f66966260 0x560f6696e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f1909a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f66966260 0x560f6696e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55882/0 (socket says 172.21.15.3:55882) 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f1909a00640 1 -- 172.21.15.3:0/1057019012 learned_addr learned my addr 172.21.15.3:0/1057019012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.582+0000 7f1909000640 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f669658b0 0x560f66965c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1909a00640 1 -- 172.21.15.3:0/1057019012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 msgr2=0x560f66971230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1909a00640 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 0x560f66971230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1909a00640 1 -- 172.21.15.3:0/1057019012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f669658b0 msgr2=0x560f66965c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1909a00640 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f669658b0 0x560f66965c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1909a00640 1 -- 172.21.15.3:0/1057019012 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f66896080 con 0x560f66966260 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1909000640 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f669658b0 0x560f66965c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:51.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1909a00640 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f66966260 0x560f6696e810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18f8005ab0 tx=0x7f18f802c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599f901e8ee0a6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f1902000640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f18ec0a30d0 0x7f18ec0a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.583+0000 7f18f6a00640 1 -- 172.21.15.3:0/1057019012 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8045020 con 0x560f66966260 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.584+0000 7f1902000640 1 -- 172.21.15.3:0/1157239622 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f18fc0d0290 con 0x7f1904109690 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.584+0000 7f18f6a00640 1 -- 172.21.15.3:0/1057019012 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f18f802c7e0 con 0x560f66966260 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.584+0000 7f18f6a00640 1 -- 172.21.15.3:0/1057019012 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80054a0 con 0x560f66966260 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.584+0000 7f1909000640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f18ec0a30d0 0x7f18ec0a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 -- 172.21.15.3:0/1057019012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f66966260 msgr2=0x560f6696e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f66966260 0x560f6696e810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18f8005ab0 tx=0x7f18f802c6e0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f1909000640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f18ec0a30d0 0x7f18ec0a5590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18f00097f0 tx=0x7f18f0005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 -- 172.21.15.3:0/1057019012 shutdown_connections 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 0x560f66971230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f66966260 0x560f6696e810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 --2- 172.21.15.3:0/1057019012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f669658b0 0x560f66965c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 -- 172.21.15.3:0/1057019012 >> 172.21.15.3:0/1057019012 conn(0x560f6695ff80 msgr2=0x560f66960360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 -- 172.21.15.3:0/1057019012 shutdown_connections 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.585+0000 7f190f030740 1 -- 172.21.15.3:0/1057019012 wait complete. 2024-03-21T08:33:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.586+0000 7f190f030740 1 -- start start 2024-03-21T08:33:51.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.586+0000 7f190f030740 1 Processor -- start 2024-03-21T08:33:51.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.586+0000 7f190f030740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f669658b0 0x560f6696e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.586+0000 7f190f030740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6696baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.586+0000 7f190f030740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f6696ede0 0x560f6696c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190f030740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f66971cf0 con 0x560f66966260 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190f030740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f668850d0 con 0x560f6696ede0 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190f030740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f66971e70 con 0x560f669658b0 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190f030740 10 client.0 _ll_register_callbacks cb 0x560f66889e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f1909000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6696baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190a400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f6696ede0 0x560f6696c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f1909000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6696baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47506/0 (socket says 172.21.15.3:47506) 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f1909000640 1 -- 172.21.15.3:0/1277546102 learned_addr learned my addr 172.21.15.3:0/1277546102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f1909a00640 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f669658b0 0x560f6696e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190a400640 1 -- 172.21.15.3:0/1277546102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f669658b0 msgr2=0x560f6696e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190a400640 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f669658b0 0x560f6696e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.587+0000 7f190a400640 1 -- 172.21.15.3:0/1277546102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 msgr2=0x560f6696baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.588+0000 7f190a400640 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6696baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.588+0000 7f190a400640 1 -- 172.21.15.3:0/1277546102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f665dd3f0 con 0x560f6696ede0 2024-03-21T08:33:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.588+0000 7f1909000640 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6696baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:51.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.588+0000 7f190a400640 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f6696ede0 0x560f6696c100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0cd560 tx=0x7f18fc09c2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cded04c5d1cf6d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.588+0000 7f18eb400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:51.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.589+0000 7f18eb400640 1 -- 172.21.15.3:0/1277546102 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0b5060 con 0x560f6696ede0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.589+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18f802c7e0 con 0x560f6696ede0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.589+0000 7f18eb400640 1 -- 172.21.15.3:0/1277546102 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f18fc098870 con 0x560f6696ede0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.589+0000 7f18eb400640 1 -- 172.21.15.3:0/1277546102 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18fc0a4c10 con 0x560f6696ede0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.589+0000 7f190f030740 10 client.35987 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.589+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560f666a77d0 con 0x560f6696ede0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.589+0000 7f18eb400640 1 -- 172.21.15.3:0/1277546102 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f18f802c7e0 con 0x560f6696ede0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f18eb400640 1 -- 172.21.15.3:0/1277546102 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f18fc0bc160 con 0x560f6696ede0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f190f030740 10 client.35987 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f190f030740 10 client.35987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f190f030740 20 client.35987 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f190f030740 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669874f0 0x560f66989900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f190f030740 4 client.35987 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f668d6800 con 0x560f669874f0 2024-03-21T08:33:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.590+0000 7f1909a00640 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669874f0 0x560f66989900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.591+0000 7f18eb400640 10 client.35987 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:51.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.591+0000 7f1909a00640 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669874f0 0x560f66989900 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=680ac8c897de566c server_cookie=ed3db443a7186b7f in_seq=0 out_seq=0 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.592+0000 7f18eb400640 1 -- 172.21.15.3:0/1277546102 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560f668d6800 con 0x560f669874f0 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.592+0000 7f18eb400640 10 client.35987 handle_command_reply: tid=0 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.592+0000 7f190f030740 1 client.35987 shutdown 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.592+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f6696ede0 msgr2=0x560f6696c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.592+0000 7f190f030740 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f6696ede0 0x560f6696c100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0cd560 tx=0x7f18fc09c2c0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 shutdown_connections 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669874f0 0x560f66989900 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f6696ede0 0x560f6696c100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6696baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f18eb400640 0 client.35987 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:51.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 --2- 172.21.15.3:0/1277546102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f669658b0 0x560f6696e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 >> 172.21.15.3:0/1277546102 conn(0x560f6695ff80 msgr2=0x560f66961cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 shutdown_connections 2024-03-21T08:33:51.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 1 -- 172.21.15.3:0/1277546102 wait complete. 2024-03-21T08:33:51.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.593+0000 7f190f030740 20 client.35987 trim_cache size 0 max 16384 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 Processor -- start 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 -- start start 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6696e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f66966260 0x560f6696e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f6696ede0 0x560f66968d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041aecd0 con 0x560f6696ede0 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041ac7a0 con 0x560f669658b0 2024-03-21T08:33:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.613+0000 7f190f030740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041aa840 con 0x560f66966260 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6696e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f190a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f6696ede0 0x560f66968d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6696e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55904/0 (socket says 172.21.15.3:55904) 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 -- 172.21.15.3:0/1046011055 learned_addr learned my addr 172.21.15.3:0/1046011055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909000640 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f66966260 0x560f6696e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 -- 172.21.15.3:0/1046011055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f66966260 msgr2=0x560f6696e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f66966260 0x560f6696e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 -- 172.21.15.3:0/1046011055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f6696ede0 msgr2=0x560f66968d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f6696ede0 0x560f66968d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f1909a00640 1 -- 172.21.15.3:0/1046011055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f66965ff0 con 0x560f669658b0 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.614+0000 7f190a400640 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f6696ede0 0x560f66968d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.615+0000 7f1909a00640 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6696e220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009850 tx=0x7f18f8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e102ca65f8642ac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.615+0000 7f18f6000640 1 -- 172.21.15.3:0/1046011055 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f804e070 con 0x560f669658b0 2024-03-21T08:33:51.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.615+0000 7f18f6000640 1 -- 172.21.15.3:0/1046011055 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f18f802cc90 con 0x560f669658b0 2024-03-21T08:33:51.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.615+0000 7f18f6000640 1 -- 172.21.15.3:0/1046011055 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8045040 con 0x560f669658b0 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.616+0000 7f190f030740 1 -- 172.21.15.3:0/1046011055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 msgr2=0x560f6696e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.616+0000 7f190f030740 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6696e220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009850 tx=0x7f18f8005c00 comp rx=0 tx=0).stop 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.616+0000 7f190f030740 1 -- 172.21.15.3:0/1046011055 shutdown_connections 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.616+0000 7f190f030740 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f6696ede0 0x560f66968d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.616+0000 7f190f030740 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f66966260 0x560f6696e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.616+0000 7f190f030740 1 --2- 172.21.15.3:0/1046011055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6696e220 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.616+0000 7f190f030740 1 -- 172.21.15.3:0/1046011055 >> 172.21.15.3:0/1046011055 conn(0x560f6695ff80 msgr2=0x560f6695f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.617+0000 7f190f030740 1 -- 172.21.15.3:0/1046011055 shutdown_connections 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.617+0000 7f190f030740 1 -- 172.21.15.3:0/1046011055 wait complete. 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.617+0000 7f190f030740 1 -- start start 2024-03-21T08:33:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.617+0000 7f190f030740 1 Processor -- start 2024-03-21T08:33:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190f030740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6697f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190f030740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6697f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190f030740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 0x560f6697d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190f030740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041aae40 con 0x560f66966260 2024-03-21T08:33:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190f030740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041ac140 con 0x560f669658b0 2024-03-21T08:33:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190f030740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041ab480 con 0x560f6696ede0 2024-03-21T08:33:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190f030740 10 client.0 _ll_register_callbacks cb 0x7f18fc005810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f1909a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6697f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f1909000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6697f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f1909a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6697f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55918/0 (socket says 172.21.15.3:55918) 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f1909a00640 1 -- 172.21.15.3:0/3752225651 learned_addr learned my addr 172.21.15.3:0/3752225651 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.618+0000 7f190a400640 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 0x560f6697d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f1909a00640 1 -- 172.21.15.3:0/3752225651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 msgr2=0x560f6697d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f1909a00640 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 0x560f6697d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f1909a00640 1 -- 172.21.15.3:0/3752225651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 msgr2=0x560f6697f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f1909a00640 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6697f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f1909a00640 1 -- 172.21.15.3:0/3752225651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f6697e030 con 0x560f669658b0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f1909a00640 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6697f350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009930 tx=0x7f18f802cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f65a19b2513593ee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f18eb400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f18eb400640 1 -- 172.21.15.3:0/3752225651 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f804e030 con 0x560f669658b0 2024-03-21T08:33:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f18eb400640 1 -- 172.21.15.3:0/3752225651 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f18f80055f0 con 0x560f669658b0 2024-03-21T08:33:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.619+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18f802cc90 con 0x560f669658b0 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.620+0000 7f18eb400640 1 -- 172.21.15.3:0/3752225651 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18f805e050 con 0x560f669658b0 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.620+0000 7f190f030740 10 client.35993 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.620+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560f669e3ac0 con 0x560f669658b0 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.620+0000 7f18eb400640 1 -- 172.21.15.3:0/3752225651 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f18f802cc90 con 0x560f669658b0 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.621+0000 7f18eb400640 1 -- 172.21.15.3:0/3752225651 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f18f80046c0 con 0x560f669658b0 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.621+0000 7f190f030740 10 client.35993 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.621+0000 7f190f030740 10 client.35993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.621+0000 7f190f030740 20 client.35993 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.621+0000 7f190f030740 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669e3cf0 0x560f669e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.621+0000 7f190f030740 4 client.35993 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:33:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.621+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f668d0b50 con 0x560f669e3cf0 2024-03-21T08:33:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.622+0000 7f1909000640 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669e3cf0 0x560f669e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.623+0000 7f1909000640 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669e3cf0 0x560f669e60e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84404b4da120c01a server_cookie=6ca6cd3243399e25 in_seq=0 out_seq=0 2024-03-21T08:33:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.623+0000 7f18eb400640 10 client.35993 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.623+0000 7f18eb400640 1 -- 172.21.15.3:0/3752225651 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f668d0b50 con 0x560f669e3cf0 2024-03-21T08:33:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.623+0000 7f18eb400640 10 client.35993 handle_command_reply: tid=0 2024-03-21T08:33:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.624+0000 7f190f030740 1 client.35993 shutdown 2024-03-21T08:33:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.624+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 msgr2=0x560f6697f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.624+0000 7f190f030740 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6697f350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009930 tx=0x7f18f802cb20 comp rx=0 tx=0).stop 2024-03-21T08:33:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 shutdown_connections 2024-03-21T08:33:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x560f669e3cf0 0x560f669e60e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f6696ede0 0x560f6697d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560f66966260 0x560f6697f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 --2- 172.21.15.3:0/3752225651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f669658b0 0x560f6697f350 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 >> 172.21.15.3:0/3752225651 conn(0x560f6695ff80 msgr2=0x560f66966640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f18eb400640 0 client.35993 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:33:51.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 shutdown_connections 2024-03-21T08:33:51.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.625+0000 7f190f030740 1 -- 172.21.15.3:0/3752225651 wait complete. 2024-03-21T08:33:51.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.626+0000 7f190f030740 20 client.35993 trim_cache size 0 max 16384 2024-03-21T08:33:51.627 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:33:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.626+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f18ec0a30d0 msgr2=0x7f18ec0a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.626+0000 7f190ae00640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f18ec0a30d0 0x7f18ec0a5590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18f00097f0 tx=0x7f18f0005f90 comp rx=0 tx=0).stop 2024-03-21T08:33:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.626+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1904109690 msgr2=0x7f190419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.626+0000 7f190ae00640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1904109690 0x7f190419e7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009fd0 tx=0x7f18fc00ef10 comp rx=0 tx=0).stop 2024-03-21T08:33:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 shutdown_connections 2024-03-21T08:33:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1904109690 0x7f190419e7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1904104040 0x7f190419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f18ec0a30d0 0x7f18ec0a5590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 --2- 172.21.15.3:0/1157239622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1904103680 0x7f1904199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 >> 172.21.15.3:0/1157239622 conn(0x7f19040ff110 msgr2=0x7f19040ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 shutdown_connections 2024-03-21T08:33:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.627+0000 7f190ae00640 1 -- 172.21.15.3:0/1157239622 wait complete. 2024-03-21T08:33:51.650 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:33:51.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T08:33:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 -- 172.21.15.3:0/2994274812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c10dd60 msgr2=0x7f2c5c110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 --2- 172.21.15.3:0/2994274812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c10dd60 0x7f2c5c110150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009be0 tx=0x7f2c5402bef0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 -- 172.21.15.3:0/2994274812 shutdown_connections 2024-03-21T08:33:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 --2- 172.21.15.3:0/2994274812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c10dd60 0x7f2c5c110150 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 --2- 172.21.15.3:0/2994274812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c5c100b60 0x7f2c5c10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 --2- 172.21.15.3:0/2994274812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 0x7f2c5c100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 -- 172.21.15.3:0/2994274812 >> 172.21.15.3:0/2994274812 conn(0x7f2c5c077130 msgr2=0x7f2c5c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 -- 172.21.15.3:0/2994274812 shutdown_connections 2024-03-21T08:33:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.887+0000 7f2c5b200640 1 -- 172.21.15.3:0/2994274812 wait complete. 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.888+0000 7f2c5b200640 1 Processor -- start 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c5b200640 1 -- start start 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c5b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 0x7f2c5c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c5b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c100b60 0x7f2c5c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c5b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c5c10dd60 0x7f2c5c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c5b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c113950 con 0x7f2c5c1001b0 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c5b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1137d0 con 0x7f2c5c100b60 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c5b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c113ad0 con 0x7f2c5c10dd60 2024-03-21T08:33:51.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 0x7f2c5c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 0x7f2c5c199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47534/0 (socket says 172.21.15.3:47534) 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c59e00640 1 -- 172.21.15.3:0/1562622311 learned_addr learned my addr 172.21.15.3:0/1562622311 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.889+0000 7f2c53e00640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c100b60 0x7f2c5c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.890+0000 7f2c5a800640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c5c10dd60 0x7f2c5c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.890+0000 7f2c59e00640 1 -- 172.21.15.3:0/1562622311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c5c10dd60 msgr2=0x7f2c5c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.890+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c5c10dd60 0x7f2c5c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.890+0000 7f2c59e00640 1 -- 172.21.15.3:0/1562622311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c100b60 msgr2=0x7f2c5c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.890+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c100b60 0x7f2c5c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.890+0000 7f2c59e00640 1 -- 172.21.15.3:0/1562622311 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c19eea0 con 0x7f2c5c1001b0 2024-03-21T08:33:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.890+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 0x7f2c5c199eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00e9c0 tx=0x7f2c4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.891+0000 7f2c52a00640 1 -- 172.21.15.3:0/1562622311 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00cde0 con 0x7f2c5c1001b0 2024-03-21T08:33:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.891+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c19f190 con 0x7f2c5c1001b0 2024-03-21T08:33:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.891+0000 7f2c52a00640 1 -- 172.21.15.3:0/1562622311 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c0035f0 con 0x7f2c5c1001b0 2024-03-21T08:33:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.891+0000 7f2c52a00640 1 -- 172.21.15.3:0/1562622311 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c010690 con 0x7f2c5c1001b0 2024-03-21T08:33:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.891+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c1a6a20 con 0x7f2c5c1001b0 2024-03-21T08:33:51.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.895+0000 7f2c52a00640 1 -- 172.21.15.3:0/1562622311 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f2c4c0037a0 con 0x7f2c5c1001b0 2024-03-21T08:33:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.896+0000 7f2c63a55740 1 Processor -- start 2024-03-21T08:33:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.896+0000 7f2c63a55740 1 -- start start 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b28b0 0x55acec0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b3260 0x55acec0bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c63a55740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acec0162c0 con 0x55acec0b3260 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c63a55740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebfbec90 con 0x55acec0bbde0 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c63a55740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acec012360 con 0x55acec0b28b0 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c53e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c53e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0be230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55956/0 (socket says 172.21.15.3:55956) 2024-03-21T08:33:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c53e00640 1 -- 172.21.15.3:0/1479720441 learned_addr learned my addr 172.21.15.3:0/1479720441 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.897+0000 7f2c5a800640 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b28b0 0x55acec0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c52a00640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2c400a3110 0x7f2c400a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c53e00640 1 -- 172.21.15.3:0/1479720441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b28b0 msgr2=0x55acec0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c53e00640 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b28b0 0x55acec0b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c53e00640 1 -- 172.21.15.3:0/1479720441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b3260 msgr2=0x55acec0bb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c52a00640 1 -- 172.21.15.3:0/1562622311 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2c4c0cffd0 con 0x7f2c5c1001b0 2024-03-21T08:33:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b3260 0x55acec0bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c53e00640 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b3260 0x55acec0bb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c53e00640 1 -- 172.21.15.3:0/1479720441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acebfe3080 con 0x55acec0bbde0 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b3260 0x55acec0bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.898+0000 7f2c5a800640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2c400a3110 0x7f2c400a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.899+0000 7f2c53e00640 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0be230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b690 tx=0x7f2c4400bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b2604d96da477e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.899+0000 7f2c4b400640 1 -- 172.21.15.3:0/1479720441 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44003bd0 con 0x55acec0bbde0 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.899+0000 7f2c4b400640 1 -- 172.21.15.3:0/1479720441 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c44015070 con 0x55acec0bbde0 2024-03-21T08:33:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.899+0000 7f2c4b400640 1 -- 172.21.15.3:0/1479720441 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44004d40 con 0x55acec0bbde0 2024-03-21T08:33:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 -- 172.21.15.3:0/1479720441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 msgr2=0x55acec0be230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0be230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b690 tx=0x7f2c4400bb50 comp rx=0 tx=0).stop 2024-03-21T08:33:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c5a800640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2c400a3110 0x7f2c400a55d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c5402c430 tx=0x7f2c54009500 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 -- 172.21.15.3:0/1479720441 shutdown_connections 2024-03-21T08:33:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0be230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b3260 0x55acec0bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1479720441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b28b0 0x55acec0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 -- 172.21.15.3:0/1479720441 >> 172.21.15.3:0/1479720441 conn(0x55acec0acf80 msgr2=0x55acec0ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 -- 172.21.15.3:0/1479720441 shutdown_connections 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.900+0000 7f2c63a55740 1 -- 172.21.15.3:0/1479720441 wait complete. 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.901+0000 7f2c63a55740 1 -- start start 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.901+0000 7f2c63a55740 1 Processor -- start 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0bb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c63a55740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acec0bed50 con 0x55acec0b28b0 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c63a55740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebfd20d0 con 0x55acec0bbde0 2024-03-21T08:33:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c63a55740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acec0beed0 con 0x55acec0b3260 2024-03-21T08:33:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c63a55740 10 client.0 _ll_register_callbacks cb 0x55acebfd6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0bb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c5a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0bb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47554/0 (socket says 172.21.15.3:47554) 2024-03-21T08:33:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c59e00640 1 -- 172.21.15.3:0/3918677349 learned_addr learned my addr 172.21.15.3:0/3918677349 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.902+0000 7f2c53e00640 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.903+0000 7f2c59e00640 1 -- 172.21.15.3:0/3918677349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 msgr2=0x55acec0b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.903+0000 7f2c59e00640 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.903+0000 7f2c59e00640 1 -- 172.21.15.3:0/3918677349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 msgr2=0x55acec0b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.903+0000 7f2c59e00640 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.903+0000 7f2c59e00640 1 -- 172.21.15.3:0/3918677349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acebd2a3f0 con 0x55acec0b28b0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.903+0000 7f2c59e00640 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0bb6d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0a4420 tx=0x7f2c4c09a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adc9ed0290bfd92 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.903+0000 7f2c3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.904+0000 7f2c3fe00640 1 -- 172.21.15.3:0/3918677349 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0a0090 con 0x55acec0b28b0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.904+0000 7f2c3fe00640 1 -- 172.21.15.3:0/3918677349 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2c4c0460f0 con 0x55acec0b28b0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.904+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c44015070 con 0x55acec0b28b0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.904+0000 7f2c3fe00640 1 -- 172.21.15.3:0/3918677349 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c4c0462f0 con 0x55acec0b28b0 2024-03-21T08:33:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.904+0000 7f2c63a55740 10 client.16269 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.904+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55acebdf47d0 con 0x55acec0b28b0 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.904+0000 7f2c3fe00640 1 -- 172.21.15.3:0/3918677349 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c44015070 con 0x55acec0b28b0 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c3fe00640 1 -- 172.21.15.3:0/3918677349 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f2c4c096ab0 con 0x55acec0b28b0 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c63a55740 10 client.16269 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c63a55740 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c63a55740 20 client.16269 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec0cbb60 0x55acec0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c63a55740 4 client.16269 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acec023800 con 0x55acec0cbb60 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.905+0000 7f2c53e00640 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec0cbb60 0x55acec0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.906+0000 7f2c53e00640 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec0cbb60 0x55acec0cdf50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=354a745e8112b8c7 server_cookie=5658723bb8071883 in_seq=0 out_seq=0 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.906+0000 7f2c3fe00640 10 client.16269 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.907+0000 7f2c3fe00640 1 -- 172.21.15.3:0/3918677349 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55acec023800 con 0x55acec0cbb60 2024-03-21T08:33:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.907+0000 7f2c3fe00640 10 client.16269 handle_command_reply: tid=0 2024-03-21T08:33:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.907+0000 7f2c63a55740 1 client.16269 shutdown 2024-03-21T08:33:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.907+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 msgr2=0x55acec0bb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.907+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0bb6d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0a4420 tx=0x7f2c4c09a450 comp rx=0 tx=0).stop 2024-03-21T08:33:51.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 shutdown_connections 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0bbde0 0x55acec0b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec0cbb60 0x55acec0cdf50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3918677349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0bb6d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 >> 172.21.15.3:0/3918677349 conn(0x55acec0acf80 msgr2=0x55acec0aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c3fe00640 0 client.16269 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 shutdown_connections 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.908+0000 7f2c63a55740 1 -- 172.21.15.3:0/3918677349 wait complete. 2024-03-21T08:33:51.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.909+0000 7f2c63a55740 20 client.16269 trim_cache size 0 max 16384 2024-03-21T08:33:51.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:51 smithi003 ceph-mon[31675]: pgmap v63: 129 pgs: 129 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 513 KiB/s wr, 111 op/s 2024-03-21T08:33:51.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:51 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1046646257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:33:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.929+0000 7f2c63a55740 1 Processor -- start 2024-03-21T08:33:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.929+0000 7f2c63a55740 1 -- start start 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b3260 0x55acec0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0bbde0 0x55acec0bb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c63a55740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1b31e0 con 0x55acec0b28b0 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c63a55740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1b0cb0 con 0x55acec0b3260 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c63a55740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1aed50 con 0x55acec0bbde0 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47556/0 (socket says 172.21.15.3:47556) 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c59e00640 1 -- 172.21.15.3:0/3317570916 learned_addr learned my addr 172.21.15.3:0/3317570916 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c53e00640 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b3260 0x55acec0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c59e00640 1 -- 172.21.15.3:0/3317570916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0bbde0 msgr2=0x55acec0bb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.930+0000 7f2c59e00640 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0bbde0 0x55acec0bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.931+0000 7f2c59e00640 1 -- 172.21.15.3:0/3317570916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b3260 msgr2=0x55acec0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.931+0000 7f2c59e00640 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b3260 0x55acec0aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.931+0000 7f2c59e00640 1 -- 172.21.15.3:0/3317570916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acec0beda0 con 0x55acec0b28b0 2024-03-21T08:33:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.931+0000 7f2c59e00640 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0ac160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0983a0 tx=0x7f2c4c003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f764bd9c103f3f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.931+0000 7f2c4aa00640 1 -- 172.21.15.3:0/3317570916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c09b8a0 con 0x55acec0b28b0 2024-03-21T08:33:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.931+0000 7f2c4aa00640 1 -- 172.21.15.3:0/3317570916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c0b5060 con 0x55acec0b28b0 2024-03-21T08:33:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.931+0000 7f2c4aa00640 1 -- 172.21.15.3:0/3317570916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c09bd10 con 0x55acec0b28b0 2024-03-21T08:33:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.932+0000 7f2c63a55740 1 -- 172.21.15.3:0/3317570916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 msgr2=0x55acec0ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.932+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0ac160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0983a0 tx=0x7f2c4c003e10 comp rx=0 tx=0).stop 2024-03-21T08:33:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.932+0000 7f2c63a55740 1 -- 172.21.15.3:0/3317570916 shutdown_connections 2024-03-21T08:33:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0bbde0 0x55acec0bb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b3260 0x55acec0aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 --2- 172.21.15.3:0/3317570916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0b28b0 0x55acec0ac160 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 -- 172.21.15.3:0/3317570916 >> 172.21.15.3:0/3317570916 conn(0x55acec0acf80 msgr2=0x55acec0ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 -- 172.21.15.3:0/3317570916 shutdown_connections 2024-03-21T08:33:51.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 -- 172.21.15.3:0/3317570916 wait complete. 2024-03-21T08:33:51.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 -- start start 2024-03-21T08:33:51.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.933+0000 7f2c63a55740 1 Processor -- start 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b28b0 0x55acec0c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c63a55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0bbde0 0x55acec0c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c63a55740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1af350 con 0x55acec0bbde0 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c63a55740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1b0650 con 0x55acec0b28b0 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c63a55740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1af990 con 0x55acec0b3260 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c63a55740 10 client.0 _ll_register_callbacks cb 0x7f2c4c003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c5a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0bbde0 0x55acec0c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b28b0 0x55acec0c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c5a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0bbde0 0x55acec0c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47558/0 (socket says 172.21.15.3:47558) 2024-03-21T08:33:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c59e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b28b0 0x55acec0c2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55974/0 (socket says 172.21.15.3:55974) 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.934+0000 7f2c5a800640 1 -- 172.21.15.3:0/1812924889 learned_addr learned my addr 172.21.15.3:0/1812924889 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c53e00640 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c5a800640 1 -- 172.21.15.3:0/1812924889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 msgr2=0x55acec0c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c5a800640 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c5a800640 1 -- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b28b0 msgr2=0x55acec0c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c5a800640 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b28b0 0x55acec0c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c5a800640 1 -- 172.21.15.3:0/1812924889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acec1274e0 con 0x55acec0bbde0 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b28b0 0x55acec0c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c5a800640 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0bbde0 0x55acec0c3090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c101190 tx=0x7f2c5402cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a03c158b1072d89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.935+0000 7f2c3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.936+0000 7f2c3fe00640 1 -- 172.21.15.3:0/1812924889 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54051020 con 0x55acec0bbde0 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.936+0000 7f2c3fe00640 1 -- 172.21.15.3:0/1812924889 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2c54003970 con 0x55acec0bbde0 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.936+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c4c0b5060 con 0x55acec0bbde0 2024-03-21T08:33:51.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.936+0000 7f2c3fe00640 1 -- 172.21.15.3:0/1812924889 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c54003b70 con 0x55acec0bbde0 2024-03-21T08:33:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.936+0000 7f2c63a55740 10 client.16275 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:51.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.936+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55acec1287e0 con 0x55acec0bbde0 2024-03-21T08:33:51.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c3fe00640 1 -- 172.21.15.3:0/1812924889 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c0b5060 con 0x55acec0bbde0 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c3fe00640 1 -- 172.21.15.3:0/1812924889 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f2c54004280 con 0x55acec0bbde0 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c63a55740 10 client.16275 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c63a55740 10 client.16275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c63a55740 20 client.16275 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec128f60 0x55acec12b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c63a55740 4 client.16275 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.937+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acec01db50 con 0x55acec128f60 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.938+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec128f60 0x55acec12b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.938+0000 7f2c59e00640 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec128f60 0x55acec12b380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f80ed94f4d94438 server_cookie=1672dbd243b09b07 in_seq=0 out_seq=0 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.938+0000 7f2c3fe00640 10 client.16275 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:51.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.938+0000 7f2c3fe00640 1 -- 172.21.15.3:0/1812924889 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acec01db50 con 0x55acec128f60 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.938+0000 7f2c3fe00640 10 client.16275 handle_command_reply: tid=0 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.938+0000 7f2c63a55740 1 client.16275 shutdown 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0bbde0 msgr2=0x55acec0c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0bbde0 0x55acec0c3090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c101190 tx=0x7f2c5402cc60 comp rx=0 tx=0).stop 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 shutdown_connections 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acec0bbde0 0x55acec0c3090 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55acec128f60 0x55acec12b380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55acec0b3260 0x55acec0c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 --2- 172.21.15.3:0/1812924889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acec0b28b0 0x55acec0c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 >> 172.21.15.3:0/1812924889 conn(0x55acec0acf80 msgr2=0x55acec0abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c3fe00640 0 client.16275 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 shutdown_connections 2024-03-21T08:33:51.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 1 -- 172.21.15.3:0/1812924889 wait complete. 2024-03-21T08:33:51.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.939+0000 7f2c63a55740 20 client.16275 trim_cache size 0 max 16384 2024-03-21T08:33:51.943 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2c400a3110 msgr2=0x7f2c400a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2c400a3110 0x7f2c400a55d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c5402c430 tx=0x7f2c54009500 comp rx=0 tx=0).stop 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 msgr2=0x7f2c5c199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 0x7f2c5c199eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00e9c0 tx=0x7f2c4c00ee80 comp rx=0 tx=0).stop 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 shutdown_connections 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2c400a3110 0x7f2c400a55d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c5c10dd60 0x7f2c5c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c5c100b60 0x7f2c5c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 --2- 172.21.15.3:0/1562622311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c5c1001b0 0x7f2c5c199eb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 >> 172.21.15.3:0/1562622311 conn(0x7f2c5c077130 msgr2=0x7f2c5c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 shutdown_connections 2024-03-21T08:33:51.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:51.940+0000 7f2c5b200640 1 -- 172.21.15.3:0/1562622311 wait complete. 2024-03-21T08:33:51.960 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:33:51.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T08:33:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.175+0000 7ff777400640 1 -- 172.21.15.3:0/2969486446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770107dc0 msgr2=0x7ff770108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.175+0000 7ff777400640 1 --2- 172.21.15.3:0/2969486446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770107dc0 0x7ff770108220 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009860 tx=0x7ff75c02bb70 comp rx=0 tx=0).stop 2024-03-21T08:33:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.176+0000 7ff777400640 1 -- 172.21.15.3:0/2969486446 shutdown_connections 2024-03-21T08:33:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.176+0000 7ff777400640 1 --2- 172.21.15.3:0/2969486446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff770108760 0x7ff770114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.176+0000 7ff777400640 1 --2- 172.21.15.3:0/2969486446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770107dc0 0x7ff770108220 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.176+0000 7ff777400640 1 --2- 172.21.15.3:0/2969486446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff770109b70 0x7ff770109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.176+0000 7ff777400640 1 -- 172.21.15.3:0/2969486446 >> 172.21.15.3:0/2969486446 conn(0x7ff770077540 msgr2=0x7ff7701064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.176+0000 7ff777400640 1 -- 172.21.15.3:0/2969486446 shutdown_connections 2024-03-21T08:33:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.176+0000 7ff777400640 1 -- 172.21.15.3:0/2969486446 wait complete. 2024-03-21T08:33:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.177+0000 7ff777400640 1 Processor -- start 2024-03-21T08:33:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.177+0000 7ff777400640 1 -- start start 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.177+0000 7ff777400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff770107dc0 0x7ff77019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.177+0000 7ff777400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff770108760 0x7ff77019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff777400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770109b70 0x7ff7701a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff777400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff770117fd0 con 0x7ff770109b70 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff777400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff770117e50 con 0x7ff770107dc0 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff777400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff770118150 con 0x7ff770108760 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff775400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770109b70 0x7ff7701a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff775400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770109b70 0x7ff7701a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47578/0 (socket says 172.21.15.3:47578) 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff775400640 1 -- 172.21.15.3:0/584982050 learned_addr learned my addr 172.21.15.3:0/584982050 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff774a00640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff770107dc0 0x7ff77019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.178+0000 7ff76fe00640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff770108760 0x7ff77019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.179+0000 7ff774a00640 1 -- 172.21.15.3:0/584982050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff770108760 msgr2=0x7ff77019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.179+0000 7ff774a00640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff770108760 0x7ff77019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.179+0000 7ff774a00640 1 -- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770109b70 msgr2=0x7ff7701a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.179+0000 7ff774a00640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770109b70 0x7ff7701a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.179+0000 7ff774a00640 1 -- 172.21.15.3:0/584982050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7701a34c0 con 0x7ff770107dc0 2024-03-21T08:33:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.179+0000 7ff774a00640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff770107dc0 0x7ff77019e4d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff76400e790 tx=0x7ff76400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.180+0000 7ff76d600640 1 -- 172.21.15.3:0/584982050 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76400eea0 con 0x7ff770107dc0 2024-03-21T08:33:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.180+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7701a37b0 con 0x7ff770107dc0 2024-03-21T08:33:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.180+0000 7ff76d600640 1 -- 172.21.15.3:0/584982050 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff76400ccc0 con 0x7ff770107dc0 2024-03-21T08:33:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.180+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7701ab040 con 0x7ff770107dc0 2024-03-21T08:33:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.180+0000 7ff76d600640 1 -- 172.21.15.3:0/584982050 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764018710 con 0x7ff770107dc0 2024-03-21T08:33:52.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.184+0000 7ff77a1a4740 1 Processor -- start 2024-03-21T08:33:52.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff77a1a4740 1 -- start start 2024-03-21T08:33:52.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff76d600640 1 -- 172.21.15.3:0/584982050 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7ff764010430 con 0x7ff770107dc0 2024-03-21T08:33:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acc00050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2acb582c0 con 0x55c2acbfdc00 2024-03-21T08:33:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2acb00c90 con 0x55c2acbf5080 2024-03-21T08:33:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2acb54360 con 0x55c2acbf46d0 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.185+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acc00050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff774a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acc00050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47586/0 (socket says 172.21.15.3:47586) 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 -- 172.21.15.3:0/2035878359 learned_addr learned my addr 172.21.15.3:0/2035878359 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff775400640 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 -- 172.21.15.3:0/2035878359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 msgr2=0x55c2acbf4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbf4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 -- 172.21.15.3:0/2035878359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 msgr2=0x55c2acbfd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 -- 172.21.15.3:0/2035878359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2acb25080 con 0x55c2acbfdc00 2024-03-21T08:33:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76d600640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff7580a3110 0x7ff7580a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.186+0000 7ff76fe00640 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acc00050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009860 tx=0x7ff75c034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bbbbc6255df27f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.187+0000 7ff76d600640 1 -- 172.21.15.3:0/584982050 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7ff7640cfdc0 con 0x7ff770107dc0 2024-03-21T08:33:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.187+0000 7ff761c00640 1 -- 172.21.15.3:0/2035878359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c039040 con 0x55c2acbfdc00 2024-03-21T08:33:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.187+0000 7ff775400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff7580a3110 0x7ff7580a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.187+0000 7ff761c00640 1 -- 172.21.15.3:0/2035878359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff75c0349b0 con 0x55c2acbfdc00 2024-03-21T08:33:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.187+0000 7ff761c00640 1 -- 172.21.15.3:0/2035878359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0053a0 con 0x55c2acbfdc00 2024-03-21T08:33:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.188+0000 7ff77a1a4740 1 -- 172.21.15.3:0/2035878359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 msgr2=0x55c2acc00050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.188+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acc00050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009860 tx=0x7ff75c034440 comp rx=0 tx=0).stop 2024-03-21T08:33:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff775400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff7580a3110 0x7ff7580a55d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff768009270 tx=0x7ff7680095d0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff77a1a4740 1 -- 172.21.15.3:0/2035878359 shutdown_connections 2024-03-21T08:33:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acc00050 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/2035878359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff77a1a4740 1 -- 172.21.15.3:0/2035878359 >> 172.21.15.3:0/2035878359 conn(0x55c2acbeeda0 msgr2=0x55c2acbef180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:52.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff77a1a4740 1 -- 172.21.15.3:0/2035878359 shutdown_connections 2024-03-21T08:33:52.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.189+0000 7ff77a1a4740 1 -- 172.21.15.3:0/2035878359 wait complete. 2024-03-21T08:33:52.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.190+0000 7ff77a1a4740 1 -- start start 2024-03-21T08:33:52.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.190+0000 7ff77a1a4740 1 Processor -- start 2024-03-21T08:33:52.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acbfd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acbfad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2acc00b10 con 0x55c2acbf46d0 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2acb140d0 con 0x55c2acbf5080 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2acc00c90 con 0x55c2acbfdc00 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff77a1a4740 10 client.0 _ll_register_callbacks cb 0x55c2acb18e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff774a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acbfd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff774a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acbfd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47594/0 (socket says 172.21.15.3:47594) 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff774a00640 1 -- 172.21.15.3:0/1913149981 learned_addr learned my addr 172.21.15.3:0/1913149981 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.191+0000 7ff76fe00640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56020/0 (socket says 172.21.15.3:56020) 2024-03-21T08:33:52.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff775400640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acbfad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff774a00640 1 -- 172.21.15.3:0/1913149981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 msgr2=0x55c2acbfad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff774a00640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acbfad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff774a00640 1 -- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 msgr2=0x55c2acbfa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff774a00640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff774a00640 1 -- 172.21.15.3:0/1913149981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2ac86c3f0 con 0x55c2acbf46d0 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff76fe00640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff774a00640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acbfd470 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff764098190 tx=0x7ff76400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4214da6fc626f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.192+0000 7ff756000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.193+0000 7ff756000640 1 -- 172.21.15.3:0/1913149981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76409b870 con 0x55c2acbf46d0 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.193+0000 7ff756000640 1 -- 172.21.15.3:0/1913149981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7ff7640a44b0 con 0x55c2acbf46d0 2024-03-21T08:33:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.193+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff75c0349b0 con 0x55c2acbf46d0 2024-03-21T08:33:52.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.193+0000 7ff756000640 1 -- 172.21.15.3:0/1913149981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7640cd520 con 0x55c2acbf46d0 2024-03-21T08:33:52.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.193+0000 7ff77a1a4740 10 client.16287 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:52.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.193+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c2ac9367d0 con 0x55c2acbf46d0 2024-03-21T08:33:52.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.193+0000 7ff756000640 1 -- 172.21.15.3:0/1913149981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff75c0349b0 con 0x55c2acbf46d0 2024-03-21T08:33:52.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.194+0000 7ff756000640 1 -- 172.21.15.3:0/1913149981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7ff764097ba0 con 0x55c2acbf46d0 2024-03-21T08:33:52.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.194+0000 7ff77a1a4740 10 client.16287 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:52.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.194+0000 7ff77a1a4740 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:52.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.194+0000 7ff77a1a4740 20 client.16287 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:52.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.195+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc0d900 0x55c2acc0fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.195+0000 7ff77a1a4740 4 client.16287 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:52.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.195+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2acb65800 con 0x55c2acc0d900 2024-03-21T08:33:52.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.195+0000 7ff76fe00640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc0d900 0x55c2acc0fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.196+0000 7ff76fe00640 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc0d900 0x55c2acc0fd30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e332102c4ae2fa0 server_cookie=aa99909ae565d9fd in_seq=0 out_seq=0 2024-03-21T08:33:52.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.196+0000 7ff756000640 10 client.16287 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:52.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff756000640 1 -- 172.21.15.3:0/1913149981 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c2acb65800 con 0x55c2acc0d900 2024-03-21T08:33:52.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff756000640 10 client.16287 handle_command_reply: tid=0 2024-03-21T08:33:52.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff77a1a4740 1 client.16287 shutdown 2024-03-21T08:33:52.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 msgr2=0x55c2acbfd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acbfd470 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff764098190 tx=0x7ff76400ce70 comp rx=0 tx=0).stop 2024-03-21T08:33:52.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 shutdown_connections 2024-03-21T08:33:52.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc0d900 0x55c2acc0fd30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acbfad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.197+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.198+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1913149981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acbfd470 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.198+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 >> 172.21.15.3:0/1913149981 conn(0x55c2acbeeda0 msgr2=0x55c2acbf0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.198+0000 7ff756000640 0 client.16287 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.198+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 shutdown_connections 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.198+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1913149981 wait complete. 2024-03-21T08:33:52.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.198+0000 7ff77a1a4740 20 client.16287 trim_cache size 0 max 16384 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 Processor -- start 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 -- start start 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbfcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acbf7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750012540 con 0x55c2acbfdc00 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750010010 con 0x55c2acbf5080 2024-03-21T08:33:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.218+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75000e0b0 con 0x55c2acbf46d0 2024-03-21T08:33:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff775400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acbf7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff775400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acbf7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47598/0 (socket says 172.21.15.3:47598) 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff774a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbfcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff775400640 1 -- 172.21.15.3:0/3246328370 learned_addr learned my addr 172.21.15.3:0/3246328370 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff76fe00640 1 -- 172.21.15.3:0/3246328370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 msgr2=0x55c2acbfcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff76fe00640 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbfcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff76fe00640 1 -- 172.21.15.3:0/3246328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 msgr2=0x55c2acbf7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff76fe00640 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acbf7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff76fe00640 1 -- 172.21.15.3:0/3246328370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2acbffff0 con 0x55c2acbf5080 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.219+0000 7ff775400640 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acbf7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.220+0000 7ff76fe00640 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff75c000c00 tx=0x7ff75c02c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1cac6b0e84d69d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.220+0000 7ff761200640 1 -- 172.21.15.3:0/3246328370 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c039040 con 0x55c2acbf5080 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.220+0000 7ff761200640 1 -- 172.21.15.3:0/3246328370 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff75c0349b0 con 0x55c2acbf5080 2024-03-21T08:33:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.220+0000 7ff761200640 1 -- 172.21.15.3:0/3246328370 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c02c770 con 0x55c2acbf5080 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 -- 172.21.15.3:0/3246328370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 msgr2=0x55c2acbfd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff75c000c00 tx=0x7ff75c02c270 comp rx=0 tx=0).stop 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 -- 172.21.15.3:0/3246328370 shutdown_connections 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbfdc00 0x55c2acbf7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acbfd470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/3246328370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbf46d0 0x55c2acbfcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 -- 172.21.15.3:0/3246328370 >> 172.21.15.3:0/3246328370 conn(0x55c2acbeeda0 msgr2=0x55c2acbedeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 -- 172.21.15.3:0/3246328370 shutdown_connections 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 -- 172.21.15.3:0/3246328370 wait complete. 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 -- start start 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.221+0000 7ff77a1a4740 1 Processor -- start 2024-03-21T08:33:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.222+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acc01620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.222+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acc01b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.222+0000 7ff77a1a4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acc68200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.222+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75000e6b0 con 0x55c2acbf46d0 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.222+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75000f9b0 con 0x55c2acbf5080 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.222+0000 7ff77a1a4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75000ecf0 con 0x55c2acbfdc00 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.222+0000 7ff77a1a4740 10 client.0 _ll_register_callbacks cb 0x7ff7640d3560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acc01b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff774a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acc01620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acc01b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56024/0 (socket says 172.21.15.3:56024) 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 -- 172.21.15.3:0/1218007797 learned_addr learned my addr 172.21.15.3:0/1218007797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff775400640 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acc68200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 -- 172.21.15.3:0/1218007797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 msgr2=0x55c2acc68200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acc68200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 -- 172.21.15.3:0/1218007797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 msgr2=0x55c2acc01620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:52.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acc01620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 -- 172.21.15.3:0/1218007797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2acc02430 con 0x55c2acbf5080 2024-03-21T08:33:52.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff775400640 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acc68200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:52.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff76fe00640 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acc01b60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff75c02c180 tx=0x7ff75c0094d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d5d1bc12232ee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:52.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.223+0000 7ff756000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:52.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.224+0000 7ff756000640 1 -- 172.21.15.3:0/1218007797 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c039040 con 0x55c2acbf5080 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.224+0000 7ff756000640 1 -- 172.21.15.3:0/1218007797 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7ff75c005040 con 0x55c2acbf5080 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.224+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff75c0349b0 con 0x55c2acbf5080 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff756000640 1 -- 172.21.15.3:0/1218007797 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c005240 con 0x55c2acbf5080 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 10 client.36029 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c2acc69810 con 0x55c2acbf5080 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff756000640 1 -- 172.21.15.3:0/1218007797 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff75c0349b0 con 0x55c2acbf5080 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff756000640 1 -- 172.21.15.3:0/1218007797 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7ff75c0048c0 con 0x55c2acbf5080 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 10 client.36029 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 10 client.36029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 20 client.36029 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc69aa0 0x55c2acc6be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 4 client.36029 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.225+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c2acbfb0d0 con 0x55c2acc69aa0 2024-03-21T08:33:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.226+0000 7ff774a00640 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc69aa0 0x55c2acc6be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.226+0000 7ff774a00640 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc69aa0 0x55c2acc6be90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee83a5d64ff6af1 server_cookie=4be76528d9a88d4f in_seq=0 out_seq=0 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.226+0000 7ff756000640 10 client.36029 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.227+0000 7ff756000640 1 -- 172.21.15.3:0/1218007797 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c2acbfb0d0 con 0x55c2acc69aa0 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.227+0000 7ff756000640 10 client.36029 handle_command_reply: tid=0 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.227+0000 7ff77a1a4740 1 client.36029 shutdown 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.227+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 msgr2=0x55c2acc01b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.227+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acc01b60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff75c02c180 tx=0x7ff75c0094d0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.227+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 shutdown_connections 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.227+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c2acc69aa0 0x55c2acc6be90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2acbfdc00 0x55c2acc68200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2acbf5080 0x55c2acc01b60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff77a1a4740 1 --2- 172.21.15.3:0/1218007797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2acbf46d0 0x55c2acc01620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff756000640 0 client.36029 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:52.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 >> 172.21.15.3:0/1218007797 conn(0x55c2acbeeda0 msgr2=0x55c2acbed620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:52.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 shutdown_connections 2024-03-21T08:33:52.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff77a1a4740 1 -- 172.21.15.3:0/1218007797 wait complete. 2024-03-21T08:33:52.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.228+0000 7ff77a1a4740 20 client.36029 trim_cache size 0 max 16384 2024-03-21T08:33:52.231 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff7580a3110 msgr2=0x7ff7580a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff7580a3110 0x7ff7580a55d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff768009270 tx=0x7ff7680095d0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff770107dc0 msgr2=0x7ff77019e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff770107dc0 0x7ff77019e4d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff76400e790 tx=0x7ff76400ec50 comp rx=0 tx=0).stop 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 shutdown_connections 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff770109b70 0x7ff7701a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff7580a3110 0x7ff7580a55d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff770108760 0x7ff77019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 --2- 172.21.15.3:0/584982050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff770107dc0 0x7ff77019e4d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 >> 172.21.15.3:0/584982050 conn(0x7ff770077540 msgr2=0x7ff770077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.229+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 shutdown_connections 2024-03-21T08:33:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:52.230+0000 7ff777400640 1 -- 172.21.15.3:0/584982050 wait complete. 2024-03-21T08:33:52.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:51 smithi084 ceph-mon[35477]: pgmap v63: 129 pgs: 129 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 513 KiB/s wr, 111 op/s 2024-03-21T08:33:52.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:51 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1046646257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:33:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:51 smithi171 ceph-mon[35388]: pgmap v63: 129 pgs: 129 active+clean; 20 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 513 KiB/s wr, 111 op/s 2024-03-21T08:33:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:51 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1046646257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:33:53.253 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T08:33:53.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:33:53.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T08:33:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.500+0000 7f5fc1e00640 1 -- 172.21.15.3:0/384221424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc103840 msgr2=0x7f5fbc103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.500+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/384221424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc103840 0x7f5fbc103cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009900 tx=0x7f5fa802bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.501+0000 7f5fc1e00640 1 -- 172.21.15.3:0/384221424 shutdown_connections 2024-03-21T08:33:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.501+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/384221424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fbc104200 0x7f5fbc110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.501+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/384221424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc103840 0x7f5fbc103cc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.501+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/384221424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc109850 0x7f5fbc109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.501+0000 7f5fc1e00640 1 -- 172.21.15.3:0/384221424 >> 172.21.15.3:0/384221424 conn(0x7f5fbc0ff2f0 msgr2=0x7f5fbc101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.501+0000 7f5fc1e00640 1 -- 172.21.15.3:0/384221424 shutdown_connections 2024-03-21T08:33:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.501+0000 7f5fc1e00640 1 -- 172.21.15.3:0/384221424 wait complete. 2024-03-21T08:33:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.502+0000 7f5fc1e00640 1 Processor -- start 2024-03-21T08:33:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.502+0000 7f5fc1e00640 1 -- start start 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fc1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc103840 0x7f5fbc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fc1e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fbc104200 0x7f5fbc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fc1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc109850 0x7f5fbc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fc1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc113b20 con 0x7f5fbc103840 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fc1e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1139a0 con 0x7f5fbc109850 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fc1e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc113ca0 con 0x7f5fbc104200 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc103840 0x7f5fbc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc103840 0x7f5fbc19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47614/0 (socket says 172.21.15.3:47614) 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fbaa00640 1 -- 172.21.15.3:0/686553221 learned_addr learned my addr 172.21.15.3:0/686553221 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.503+0000 7f5fbb400640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc109850 0x7f5fbc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.504+0000 7f5fba000640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fbc104200 0x7f5fbc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.504+0000 7f5fbaa00640 1 -- 172.21.15.3:0/686553221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fbc104200 msgr2=0x7f5fbc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.504+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fbc104200 0x7f5fbc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.504+0000 7f5fbaa00640 1 -- 172.21.15.3:0/686553221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc109850 msgr2=0x7f5fbc19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.504+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc109850 0x7f5fbc19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.504+0000 7f5fbaa00640 1 -- 172.21.15.3:0/686553221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc19f070 con 0x7f5fbc103840 2024-03-21T08:33:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.504+0000 7f5fba000640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fbc104200 0x7f5fbc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.505+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc103840 0x7f5fbc19a080 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00e9f0 tx=0x7f5fac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.505+0000 7f5fc0a00640 1 -- 172.21.15.3:0/686553221 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00cd80 con 0x7f5fbc103840 2024-03-21T08:33:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.505+0000 7f5fc0a00640 1 -- 172.21.15.3:0/686553221 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fac0108e0 con 0x7f5fbc103840 2024-03-21T08:33:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.505+0000 7f5fc0a00640 1 -- 172.21.15.3:0/686553221 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac018920 con 0x7f5fbc103840 2024-03-21T08:33:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.505+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc19f360 con 0x7f5fbc103840 2024-03-21T08:33:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.505+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc1a6bf0 con 0x7f5fbc103840 2024-03-21T08:33:53.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.510+0000 7f5fc4bda740 1 Processor -- start 2024-03-21T08:33:53.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.510+0000 7f5fc0a00640 1 -- 172.21.15.3:0/686553221 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f5fac010a90 con 0x7f5fbc103840 2024-03-21T08:33:53.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.510+0000 7f5fc4bda740 1 -- start start 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa612dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa6136850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa6139180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfa6089eb0 con 0x55cfa612e1b0 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfa6032a60 con 0x55cfa612d890 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfa6085f50 con 0x55cfa6136d90 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa6136850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa612dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa612dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56050/0 (socket says 172.21.15.3:56050) 2024-03-21T08:33:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fbb400640 1 -- 172.21.15.3:0/790000070 learned_addr learned my addr 172.21.15.3:0/790000070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fbaa00640 1 -- 172.21.15.3:0/790000070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 msgr2=0x55cfa6139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/790000070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa6139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.511+0000 7f5fbaa00640 1 -- 172.21.15.3:0/790000070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 msgr2=0x55cfa612dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.512+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/790000070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa612dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.512+0000 7f5fbaa00640 1 -- 172.21.15.3:0/790000070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfa5e667e0 con 0x55cfa612e1b0 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.512+0000 7f5fc0a00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5fa00a3110 0x7f5fa00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.512+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/790000070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa6136850 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0cd1b0 tx=0x7f5fac0d8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a9ad3b2ab3ecd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.512+0000 7f5fc0a00640 1 -- 172.21.15.3:0/686553221 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5fac0cfee0 con 0x7f5fbc103840 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.513+0000 7f5fbb400640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5fa00a3110 0x7f5fa00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.512+0000 7f5fa7e00640 1 -- 172.21.15.3:0/790000070 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0980f0 con 0x55cfa612e1b0 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.513+0000 7f5fa7e00640 1 -- 172.21.15.3:0/790000070 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fac010de0 con 0x55cfa612e1b0 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.513+0000 7f5fa7e00640 1 -- 172.21.15.3:0/790000070 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0d3a80 con 0x55cfa612e1b0 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.513+0000 7f5fc4bda740 1 -- 172.21.15.3:0/790000070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 msgr2=0x55cfa6136850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.513+0000 7f5fbb400640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5fa00a3110 0x7f5fa00a55d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4005f40 tx=0x7f5fb4009600 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.513+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/790000070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa6136850 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0cd1b0 tx=0x7f5fac0d8570 comp rx=0 tx=0).stop 2024-03-21T08:33:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 -- 172.21.15.3:0/790000070 shutdown_connections 2024-03-21T08:33:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/790000070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa6139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/790000070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa6136850 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/790000070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa612dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 -- 172.21.15.3:0/790000070 >> 172.21.15.3:0/790000070 conn(0x55cfa6127fc0 msgr2=0x55cfa61283a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 -- 172.21.15.3:0/790000070 shutdown_connections 2024-03-21T08:33:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 -- 172.21.15.3:0/790000070 wait complete. 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 -- start start 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.514+0000 7f5fc4bda740 1 Processor -- start 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612d890 0x55cfa61366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa6133a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa6133f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfa6139b80 con 0x55cfa612d890 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfa6045eb0 con 0x55cfa612e1b0 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfa6139d00 con 0x55cfa6136d90 2024-03-21T08:33:53.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fc4bda740 10 client.0 _ll_register_callbacks cb 0x55cfa603a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.515+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612d890 0x55cfa61366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa6133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa6133a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56052/0 (socket says 172.21.15.3:56052) 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 -- 172.21.15.3:0/3968494585 learned_addr learned my addr 172.21.15.3:0/3968494585 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fbb400640 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa6133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 -- 172.21.15.3:0/3968494585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 msgr2=0x55cfa6133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa6133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 -- 172.21.15.3:0/3968494585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612d890 msgr2=0x55cfa61366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612d890 0x55cfa61366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fba000640 1 -- 172.21.15.3:0/3968494585 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfa5ec6f70 con 0x55cfa612e1b0 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.516+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612d890 0x55cfa61366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.517+0000 7f5fba000640 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa6133a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8036040 tx=0x7f5fa802c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122417d336a03040 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.517+0000 7f5fa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.517+0000 7f5fa4c00640 1 -- 172.21.15.3:0/3968494585 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8043020 con 0x55cfa612e1b0 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.517+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfa60973f0 con 0x55cfa612e1b0 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.517+0000 7f5fa4c00640 1 -- 172.21.15.3:0/3968494585 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5fa8005040 con 0x55cfa612e1b0 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.517+0000 7f5fa4c00640 1 -- 172.21.15.3:0/3968494585 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa8005d50 con 0x55cfa612e1b0 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 10 client.36041 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cfa5d5e170 con 0x55cfa612e1b0 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fa4c00640 1 -- 172.21.15.3:0/3968494585 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55cfa60973f0 con 0x55cfa612e1b0 2024-03-21T08:33:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fa4c00640 1 -- 172.21.15.3:0/3968494585 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f5fa8004bf0 con 0x55cfa612e1b0 2024-03-21T08:33:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 10 client.36041 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 10 client.36041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 20 client.36041 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa6146d20 0x55cfa6149160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 4 client.36041 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.518+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfa6091740 con 0x55cfa6146d20 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.519+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa6146d20 0x55cfa6149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.520+0000 7f5fa4c00640 10 client.36041 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.520+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa6146d20 0x55cfa6149160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec63c4bb6286121 server_cookie=fa351c9e8103341a in_seq=0 out_seq=0 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.520+0000 7f5fa4c00640 1 -- 172.21.15.3:0/3968494585 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cfa6091740 con 0x55cfa6146d20 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.520+0000 7f5fa4c00640 10 client.36041 handle_command_reply: tid=0 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 client.36041 shutdown 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 msgr2=0x55cfa6133a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa6133a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8036040 tx=0x7f5fa802c9b0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 shutdown_connections 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa6146d20 0x55cfa6149160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa6133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa6133a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/3968494585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612d890 0x55cfa61366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fa4c00640 0 client.36041 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.521+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 >> 172.21.15.3:0/3968494585 conn(0x55cfa6127fc0 msgr2=0x55cfa6129d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.522+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 shutdown_connections 2024-03-21T08:33:53.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.522+0000 7f5fc4bda740 1 -- 172.21.15.3:0/3968494585 wait complete. 2024-03-21T08:33:53.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.522+0000 7f5fc4bda740 20 client.36041 trim_cache size 0 max 16384 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 Processor -- start 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 -- start start 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa612d890 0x55cfa6136160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa61366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa6136d90 0x55cfa612ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1af0a0 con 0x55cfa6136d90 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1acb70 con 0x55cfa612e1b0 2024-03-21T08:33:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.545+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1aac10 con 0x55cfa612d890 2024-03-21T08:33:53.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa61366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa61366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56066/0 (socket says 172.21.15.3:56066) 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 -- 172.21.15.3:0/989787696 learned_addr learned my addr 172.21.15.3:0/989787696 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa612d890 0x55cfa6136160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fbb400640 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa6136d90 0x55cfa612ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 -- 172.21.15.3:0/989787696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa612d890 msgr2=0x55cfa6136160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa612d890 0x55cfa6136160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 -- 172.21.15.3:0/989787696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa6136d90 msgr2=0x55cfa612ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa6136d90 0x55cfa612ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fba000640 1 -- 172.21.15.3:0/989787696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfa611ac20 con 0x55cfa612e1b0 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.546+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa612d890 0x55cfa6136160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.547+0000 7f5fba000640 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa61366a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5fa802c170 tx=0x7f5fa802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6a96240ba473e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.547+0000 7f5fa7400640 1 -- 172.21.15.3:0/989787696 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8043040 con 0x55cfa612e1b0 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.547+0000 7f5fa7400640 1 -- 172.21.15.3:0/989787696 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fa8057070 con 0x55cfa612e1b0 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.547+0000 7f5fa7400640 1 -- 172.21.15.3:0/989787696 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8003e40 con 0x55cfa612e1b0 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.548+0000 7f5fc4bda740 1 -- 172.21.15.3:0/989787696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 msgr2=0x55cfa61366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.548+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa61366a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5fa802c170 tx=0x7f5fa802bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.548+0000 7f5fc4bda740 1 -- 172.21.15.3:0/989787696 shutdown_connections 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.548+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa6136d90 0x55cfa612ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.548+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612e1b0 0x55cfa61366a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.548+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/989787696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa612d890 0x55cfa6136160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.548+0000 7f5fc4bda740 1 -- 172.21.15.3:0/989787696 >> 172.21.15.3:0/989787696 conn(0x55cfa6127fc0 msgr2=0x55cfa612e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.549+0000 7f5fc4bda740 1 -- 172.21.15.3:0/989787696 shutdown_connections 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.549+0000 7f5fc4bda740 1 -- 172.21.15.3:0/989787696 wait complete. 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.549+0000 7f5fc4bda740 1 -- start start 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 1 Processor -- start 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa6131630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa613bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa613c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1ab210 con 0x55cfa612e1b0 2024-03-21T08:33:53.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1ac510 con 0x55cfa612d890 2024-03-21T08:33:53.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1ab850 con 0x55cfa6136d90 2024-03-21T08:33:53.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fc4bda740 10 client.0 _ll_register_callbacks cb 0x7f5fac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa6131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.550+0000 7f5fba000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa613bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa6131630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56078/0 (socket says 172.21.15.3:56078) 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fba000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa613bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47670/0 (socket says 172.21.15.3:47670) 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbaa00640 1 -- 172.21.15.3:0/1399133587 learned_addr learned my addr 172.21.15.3:0/1399133587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbb400640 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa613c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbaa00640 1 -- 172.21.15.3:0/1399133587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 msgr2=0x55cfa613c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa613c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbaa00640 1 -- 172.21.15.3:0/1399133587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 msgr2=0x55cfa613bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa613bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.551+0000 7f5fbaa00640 1 -- 172.21.15.3:0/1399133587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfa6139180 con 0x55cfa612d890 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fbaa00640 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa6131630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fac09bb50 tx=0x7f5fac009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d21fb516f97d86 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fa4c00640 1 -- 172.21.15.3:0/1399133587 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac09b7b0 con 0x55cfa612d890 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fa4c00640 1 -- 172.21.15.3:0/1399133587 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5fac0984c0 con 0x55cfa612d890 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa8057070 con 0x55cfa612d890 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fa4c00640 1 -- 172.21.15.3:0/1399133587 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fac09b990 con 0x55cfa612d890 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fc4bda740 10 client.36047 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:53.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.552+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cfa61a3370 con 0x55cfa612d890 2024-03-21T08:33:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fa4c00640 1 -- 172.21.15.3:0/1399133587 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fac0b6020 con 0x55cfa612d890 2024-03-21T08:33:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fa4c00640 1 -- 172.21.15.3:0/1399133587 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f5fac0cd430 con 0x55cfa612d890 2024-03-21T08:33:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fc4bda740 10 client.36047 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fc4bda740 10 client.36047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fc4bda740 20 client.36047 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa61a3660 0x55cfa61a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fc4bda740 4 client.36047 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T08:33:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cfa600fa40 con 0x55cfa61a3660 2024-03-21T08:33:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.553+0000 7f5fba000640 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa61a3660 0x55cfa61a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.554+0000 7f5fa4c00640 10 client.36047 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.554+0000 7f5fba000640 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa61a3660 0x55cfa61a5a50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166fb30624e62a66 server_cookie=fc8e5647574f5c5d in_seq=0 out_seq=0 2024-03-21T08:33:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.556+0000 7f5fa4c00640 1 -- 172.21.15.3:0/1399133587 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cfa600fa40 con 0x55cfa61a3660 2024-03-21T08:33:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.556+0000 7f5fa4c00640 10 client.36047 handle_command_reply: tid=0 2024-03-21T08:33:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.556+0000 7f5fc4bda740 1 client.36047 shutdown 2024-03-21T08:33:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.556+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 msgr2=0x55cfa6131630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.556+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa6131630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fac09bb50 tx=0x7f5fac009310 comp rx=0 tx=0).stop 2024-03-21T08:33:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 shutdown_connections 2024-03-21T08:33:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55cfa61a3660 0x55cfa61a5a50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cfa6136d90 0x55cfa613c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cfa612e1b0 0x55cfa613bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 --2- 172.21.15.3:0/1399133587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfa612d890 0x55cfa6131630 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 >> 172.21.15.3:0/1399133587 conn(0x55cfa6127fc0 msgr2=0x55cfa612ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fa4c00640 0 client.36047 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 shutdown_connections 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 1 -- 172.21.15.3:0/1399133587 wait complete. 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.557+0000 7f5fc4bda740 20 client.36047 trim_cache size 0 max 16384 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "ed2a039b-cbfc-4f4a-9c02-4e78bd2e3876", 2024-03-21T08:33:53.559 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-03-21T08:33:53.560 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5fa00a3110 msgr2=0x7f5fa00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5fa00a3110 0x7f5fa00a55d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4005f40 tx=0x7f5fb4009600 comp rx=0 tx=0).stop 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc103840 msgr2=0x7f5fbc19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc103840 0x7f5fbc19a080 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00e9f0 tx=0x7f5fac00eeb0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 shutdown_connections 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fbc109850 0x7f5fbc19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5fa00a3110 0x7f5fa00a55d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fbc104200 0x7f5fbc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 --2- 172.21.15.3:0/686553221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fbc103840 0x7f5fbc19a080 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 >> 172.21.15.3:0/686553221 conn(0x7f5fbc0ff2f0 msgr2=0x7f5fbc0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.558+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 shutdown_connections 2024-03-21T08:33:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.559+0000 7f5fc1e00640 1 -- 172.21.15.3:0/686553221 wait complete. 2024-03-21T08:33:53.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:33:53.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:33:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.828+0000 7fbf75600640 1 -- 172.21.15.3:0/4151910558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70109870 msgr2=0x7fbf70109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.828+0000 7fbf75600640 1 --2- 172.21.15.3:0/4151910558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70109870 0x7fbf70109c50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009930 tx=0x7fbf6002bc30 comp rx=0 tx=0).stop 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.828+0000 7fbf75600640 1 -- 172.21.15.3:0/4151910558 shutdown_connections 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.828+0000 7fbf75600640 1 --2- 172.21.15.3:0/4151910558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70104220 0x7fbf70110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.828+0000 7fbf75600640 1 --2- 172.21.15.3:0/4151910558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf70103860 0x7fbf70103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.828+0000 7fbf75600640 1 --2- 172.21.15.3:0/4151910558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70109870 0x7fbf70109c50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.828+0000 7fbf75600640 1 -- 172.21.15.3:0/4151910558 >> 172.21.15.3:0/4151910558 conn(0x7fbf700ff310 msgr2=0x7fbf70101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.829+0000 7fbf75600640 1 -- 172.21.15.3:0/4151910558 shutdown_connections 2024-03-21T08:33:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.829+0000 7fbf75600640 1 -- 172.21.15.3:0/4151910558 wait complete. 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 Processor -- start 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 -- start start 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70103860 0x7fbf7019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70104220 0x7fbf7019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf70109870 0x7fbf7019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70113af0 con 0x7fbf70104220 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70113970 con 0x7fbf70103860 2024-03-21T08:33:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.830+0000 7fbf75600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70113c70 con 0x7fbf70109870 2024-03-21T08:33:53.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70103860 0x7fbf7019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70103860 0x7fbf7019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56100/0 (socket says 172.21.15.3:56100) 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 -- 172.21.15.3:0/3989396732 learned_addr learned my addr 172.21.15.3:0/3989396732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6f400640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf70109870 0x7fbf7019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6e000640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70104220 0x7fbf7019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 -- 172.21.15.3:0/3989396732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf70109870 msgr2=0x7fbf7019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf70109870 0x7fbf7019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 -- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70104220 msgr2=0x7fbf7019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70104220 0x7fbf7019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6ea00640 1 -- 172.21.15.3:0/3989396732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7019f070 con 0x7fbf70103860 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.831+0000 7fbf6e000640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70104220 0x7fbf7019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.832+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70103860 0x7fbf7019a080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009230 tx=0x7fbf6002c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.833+0000 7fbf67400640 1 -- 172.21.15.3:0/3989396732 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60043020 con 0x7fbf70103860 2024-03-21T08:33:53.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.833+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7019f300 con 0x7fbf70103860 2024-03-21T08:33:53.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.833+0000 7fbf67400640 1 -- 172.21.15.3:0/3989396732 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf6002caf0 con 0x7fbf70103860 2024-03-21T08:33:53.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.833+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf701a6bf0 con 0x7fbf70103860 2024-03-21T08:33:53.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.833+0000 7fbf67400640 1 -- 172.21.15.3:0/3989396732 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60003ae0 con 0x7fbf70103860 2024-03-21T08:33:53.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.836+0000 7fbf7826e740 1 Processor -- start 2024-03-21T08:33:53.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf7826e740 1 -- start start 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf7826e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366384d250 con 0x5636638e8f20 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf7826e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636637f5a30 con 0x5636638f2290 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf7826e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636638492f0 con 0x5636638e9c00 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf6e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf67400640 1 -- 172.21.15.3:0/3989396732 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fbf60046020 con 0x7fbf70103860 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf6e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56106/0 (socket says 172.21.15.3:56106) 2024-03-21T08:33:53.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.837+0000 7fbf6e000640 1 -- 172.21.15.3:0/1480015035 learned_addr learned my addr 172.21.15.3:0/1480015035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6f400640 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6e000640 1 -- 172.21.15.3:0/1480015035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 msgr2=0x5636638e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6e000640 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638e8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6e000640 1 -- 172.21.15.3:0/1480015035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 msgr2=0x5636638e93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6e000640 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638e93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6e000640 1 -- 172.21.15.3:0/1480015035 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636637e0690 con 0x5636638f2290 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:53.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.838+0000 7fbf6e000640 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638f4680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bd30 tx=0x7fbf5c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1adfb0d33c25b4db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.839+0000 7fbf5b400640 1 -- 172.21.15.3:0/1480015035 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c015070 con 0x5636638f2290 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.839+0000 7fbf5b400640 1 -- 172.21.15.3:0/1480015035 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf5c00be20 con 0x5636638f2290 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.839+0000 7fbf5b400640 1 -- 172.21.15.3:0/1480015035 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c011970 con 0x5636638f2290 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.839+0000 7fbf67400640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbf540a3090 0x7fbf540a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.839+0000 7fbf7826e740 1 -- 172.21.15.3:0/1480015035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 msgr2=0x5636638f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.839+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638f4680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bd30 tx=0x7fbf5c00f730 comp rx=0 tx=0).stop 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.839+0000 7fbf67400640 1 -- 172.21.15.3:0/3989396732 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fbf600f1d30 con 0x7fbf70103860 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf6f400640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbf540a3090 0x7fbf540a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 -- 172.21.15.3:0/1480015035 shutdown_connections 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638f4680 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1480015035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 -- 172.21.15.3:0/1480015035 >> 172.21.15.3:0/1480015035 conn(0x5636638b0200 msgr2=0x5636638e3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 -- 172.21.15.3:0/1480015035 shutdown_connections 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 -- 172.21.15.3:0/1480015035 wait complete. 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf7826e740 1 -- start start 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.840+0000 7fbf6f400640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbf540a3090 0x7fbf540a5550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf68005fd0 tx=0x7fbf68009500 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 1 Processor -- start 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e8f20 0x5636638f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e9c00 0x5636638eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638f2290 0x5636638ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636638f5100 con 0x5636638e9c00 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x563663808e80 con 0x5636638e8f20 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636638f5280 con 0x5636638f2290 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf7826e740 10 client.0 _ll_register_callbacks cb 0x56366381b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf6e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e9c00 0x5636638eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf6e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e9c00 0x5636638eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47724/0 (socket says 172.21.15.3:47724) 2024-03-21T08:33:53.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf6e000640 1 -- 172.21.15.3:0/1184979752 learned_addr learned my addr 172.21.15.3:0/1184979752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.841+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e8f20 0x5636638f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6f400640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638f2290 0x5636638ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6e000640 1 -- 172.21.15.3:0/1184979752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638f2290 msgr2=0x5636638ef1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6e000640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638f2290 0x5636638ef1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6e000640 1 -- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e8f20 msgr2=0x5636638f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6e000640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e8f20 0x5636638f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6e000640 1 -- 172.21.15.3:0/1184979752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56366362b7d0 con 0x5636638e9c00 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e8f20 0x5636638f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf6e000640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e9c00 0x5636638eec70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005270 tx=0x7fbf5c0052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16310cb903b0b0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf4fe00640 1 -- 172.21.15.3:0/1184979752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c015070 con 0x5636638e9c00 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf4fe00640 1 -- 172.21.15.3:0/1184979752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fbf5c004020 con 0x5636638e9c00 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.842+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf5c00be20 con 0x5636638e9c00 2024-03-21T08:33:53.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf4fe00640 1 -- 172.21.15.3:0/1184979752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf5c004240 con 0x5636638e9c00 2024-03-21T08:33:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf7826e740 10 client.16320 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5636635613f0 con 0x5636638e9c00 2024-03-21T08:33:53.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf4fe00640 1 -- 172.21.15.3:0/1184979752 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf5c00be20 con 0x5636638e9c00 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf4fe00640 1 -- 172.21.15.3:0/1184979752 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fbf5c0047e0 con 0x5636638e9c00 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf7826e740 10 client.16320 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf7826e740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf7826e740 20 client.16320 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.843+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563663901fe0 0x5636639043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.844+0000 7fbf7826e740 4 client.16320 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.844+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56366385a790 con 0x563663901fe0 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.844+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563663901fe0 0x5636639043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.844+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563663901fe0 0x5636639043d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcdd3ad24d108ddc server_cookie=46830c88714cc7e2 in_seq=0 out_seq=0 2024-03-21T08:33:53.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.844+0000 7fbf4fe00640 10 client.16320 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf4fe00640 1 -- 172.21.15.3:0/1184979752 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56366385a790 con 0x563663901fe0 2024-03-21T08:33:53.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf4fe00640 10 client.16320 handle_command_reply: tid=0 2024-03-21T08:33:53.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 client.16320 shutdown 2024-03-21T08:33:53.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e9c00 msgr2=0x5636638eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e9c00 0x5636638eec70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005270 tx=0x7fbf5c0052a0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 shutdown_connections 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x563663901fe0 0x5636639043d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638f2290 0x5636638ef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e9c00 0x5636638eec70 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 --2- 172.21.15.3:0/1184979752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e8f20 0x5636638f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 >> 172.21.15.3:0/1184979752 conn(0x5636638b0200 msgr2=0x5636638e56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf4fe00640 0 client.16320 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 shutdown_connections 2024-03-21T08:33:53.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.848+0000 7fbf7826e740 1 -- 172.21.15.3:0/1184979752 wait complete. 2024-03-21T08:33:53.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.849+0000 7fbf7826e740 20 client.16320 trim_cache size 0 max 16384 2024-03-21T08:33:53.876 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:33:53 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k[64508]: 2024-03-21T08:33:53.576+0000 7f8e6ae00700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d2 [4,head] /volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/af_unix.c auth v366 ap=1 s=1155 n(v0 rc2024-03-21T08:33:39.346859+0000 b1155 1=1+0) (iversion lock) caps={15471=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55d258cf5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d2:[<0x100000002d0/af_unix.c v4>,<0x10000000007/src v796>,<0x10000000006/xfstests-dev v98>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v33>,<0x10000000002/c7ef95a4-879e-4568-8fc1-2823dfb91393 v47>,<0x10000000001/sv_0 v65>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[3]","memoryvalue":"(4)0x100000002d2:[<0x100000002d0/af_unix.c v366>,<0x10000000007/src v1259>,<0x10000000006/xfstests-dev v151>,<0x10000000005/tmp v41>,<0x10000000003/client.0 v33>,<0x10000000002/c7ef95a4-879e-4568-8fc1-2823dfb91393 v57>,<0x10000000001/sv_0 v63>,<0x10000000000/qa v49>,<0x1/volumes v54>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-21T08:33:53.876 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:33:53 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k[64508]: 2024-03-21T08:33:53.589+0000 7f8e6ae00700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d3 [...4,head] /volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/aio-dio-regress/ auth v402 ap=1 f(v0 m2024-03-21T08:33:39.426357+0000 19=19+0) n(v0 rc2024-03-21T08:33:39.427957+0000 b77976 20=19+1) (inest lock) (ifile excl) (iversion lock) caps={15471=pAsLsXsFsx/-@1},l=15471 | importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55d258cb5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000002d3:[<0x100000002d0/aio-dio-regress v44>,<0x10000000007/src v796>,<0x10000000006/xfstests-dev v98>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v33>,<0x10000000002/c7ef95a4-879e-4568-8fc1-2823dfb91393 v47>,<0x10000000001/sv_0 v65>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x100000002d3:[<0x100000002d0/aio-dio-regress v402>,<0x10000000007/src v1259>,<0x10000000006/xfstests-dev v151>,<0x10000000005/tmp v41>,<0x10000000003/client.0 v33>,<0x10000000002/c7ef95a4-879e-4568-8fc1-2823dfb91393 v57>,<0x10000000001/sv_0 v63>,<0x10000000000/qa v49>,<0x1/volumes v54>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-21T08:33:39.426357+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-03-21T08:33:39.427957+0000 b77976 20=19+1)","memory_value.dirstat":"f(v0 m2024-03-21T08:33:39.426357+0000 19=19+0)","memory_value.rstat":"n(v0 rc2024-03-21T08:33:39.427957+0000 b77976 20=19+1)","error_str":""},"return_code":0} 2024-03-21T08:33:53.876 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:33:53 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k[64508]: 2024-03-21T08:33:53.595+0000 7f8e6ae00700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d1 [4,head] /volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/Makefile auth v366 ap=1 s=3854 n(v0 rc2024-03-21T08:33:39.343015+0000 b3854 1=1+0) (iversion lock) caps={15471=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55d258d79180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d1:[<0x100000002d0/Makefile v2>,<0x10000000007/src v796>,<0x10000000006/xfstests-dev v98>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v33>,<0x10000000002/c7ef95a4-879e-4568-8fc1-2823dfb91393 v47>,<0x10000000001/sv_0 v65>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[3]","memoryvalue":"(4)0x100000002d1:[<0x100000002d0/Makefile v366>,<0x10000000007/src v1259>,<0x10000000006/xfstests-dev v151>,<0x10000000005/tmp v41>,<0x10000000003/client.0 v33>,<0x10000000002/c7ef95a4-879e-4568-8fc1-2823dfb91393 v57>,<0x10000000001/sv_0 v63>,<0x10000000000/qa v49>,<0x1/volumes v54>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-21T08:33:53.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.877+0000 7fbf7826e740 1 Processor -- start 2024-03-21T08:33:53.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.877+0000 7fbf7826e740 1 -- start start 2024-03-21T08:33:53.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.877+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e8f20 0x5636638f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e9c00 0x5636638f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638f2290 0x5636638f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf7826e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701b3320 con 0x5636638f2290 2024-03-21T08:33:53.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf7826e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701b0df0 con 0x5636638e9c00 2024-03-21T08:33:53.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf7826e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701aee90 con 0x5636638e8f20 2024-03-21T08:33:53.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e8f20 0x5636638f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e8f20 0x5636638f13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.3:54448/0 (socket says 172.21.15.3:54448) 2024-03-21T08:33:53.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf6ea00640 1 -- 172.21.15.3:0/4231553690 learned_addr learned my addr 172.21.15.3:0/4231553690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.878+0000 7fbf6f400640 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638f2290 0x5636638f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.879+0000 7fbf6e000640 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e9c00 0x5636638f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.879+0000 7fbf6e000640 1 -- 172.21.15.3:0/4231553690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e8f20 msgr2=0x5636638f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.879+0000 7fbf6e000640 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e8f20 0x5636638f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.879+0000 7fbf6e000640 1 -- 172.21.15.3:0/4231553690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638f2290 msgr2=0x5636638f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.879+0000 7fbf6e000640 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638f2290 0x5636638f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.879+0000 7fbf6e000640 1 -- 172.21.15.3:0/4231553690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563663904740 con 0x5636638e9c00 2024-03-21T08:33:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.880+0000 7fbf6e000640 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e9c00 0x5636638f18e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bc50 tx=0x7fbf5c00b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4f958832c37c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.880+0000 7fbf5aa00640 1 -- 172.21.15.3:0/4231553690 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c015040 con 0x5636638e9c00 2024-03-21T08:33:53.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.880+0000 7fbf5aa00640 1 -- 172.21.15.3:0/4231553690 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf5c00be20 con 0x5636638e9c00 2024-03-21T08:33:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.880+0000 7fbf5aa00640 1 -- 172.21.15.3:0/4231553690 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0118d0 con 0x5636638e9c00 2024-03-21T08:33:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.881+0000 7fbf7826e740 1 -- 172.21.15.3:0/4231553690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e9c00 msgr2=0x5636638f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.881+0000 7fbf7826e740 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e9c00 0x5636638f18e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bc50 tx=0x7fbf5c00b5a0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.881+0000 7fbf7826e740 1 -- 172.21.15.3:0/4231553690 shutdown_connections 2024-03-21T08:33:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.881+0000 7fbf7826e740 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638f2290 0x5636638f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.881+0000 7fbf7826e740 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638e9c00 0x5636638f18e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.881+0000 7fbf7826e740 1 --2- 172.21.15.3:0/4231553690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e8f20 0x5636638f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.881+0000 7fbf7826e740 1 -- 172.21.15.3:0/4231553690 >> 172.21.15.3:0/4231553690 conn(0x5636638b0200 msgr2=0x5636638e34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.882+0000 7fbf7826e740 1 -- 172.21.15.3:0/4231553690 shutdown_connections 2024-03-21T08:33:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.882+0000 7fbf7826e740 1 -- 172.21.15.3:0/4231553690 wait complete. 2024-03-21T08:33:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.882+0000 7fbf7826e740 1 -- start start 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.882+0000 7fbf7826e740 1 Processor -- start 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.883+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.883+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.883+0000 7fbf7826e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.883+0000 7fbf7826e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701af490 con 0x5636638e8f20 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.883+0000 7fbf7826e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701b0790 con 0x5636638f2290 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.883+0000 7fbf7826e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701afad0 con 0x5636638e9c00 2024-03-21T08:33:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.883+0000 7fbf7826e740 10 client.0 _ll_register_callbacks cb 0x7fbf5c00bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47744/0 (socket says 172.21.15.3:47744) 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 -- 172.21.15.3:0/2209319369 learned_addr learned my addr 172.21.15.3:0/2209319369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6e000640 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6f400640 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 -- 172.21.15.3:0/2209319369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 msgr2=0x5636638eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 -- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 msgr2=0x5636638eb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638eb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.884+0000 7fbf6ea00640 1 -- 172.21.15.3:0/2209319369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636635210f0 con 0x5636638e8f20 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf6ea00640 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638ec4d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006010 tx=0x7fbf600035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4843fef1b4222 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf4fe00640 1 -- 172.21.15.3:0/2209319369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600d6070 con 0x5636638e8f20 2024-03-21T08:33:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf4fe00640 1 -- 172.21.15.3:0/2209319369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fbf600dc7f0 con 0x5636638e8f20 2024-03-21T08:33:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf5c00be20 con 0x5636638e8f20 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf4fe00640 1 -- 172.21.15.3:0/2209319369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf600be4b0 con 0x5636638e8f20 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf7826e740 10 client.16326 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.885+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563663965a10 con 0x5636638e8f20 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.886+0000 7fbf4fe00640 1 -- 172.21.15.3:0/2209319369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf5c00be20 con 0x5636638e8f20 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.886+0000 7fbf4fe00640 1 -- 172.21.15.3:0/2209319369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fbf600a8040 con 0x5636638e8f20 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.886+0000 7fbf7826e740 10 client.16326 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.886+0000 7fbf7826e740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.886+0000 7fbf7826e740 20 client.16326 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.886+0000 7fbf7826e740 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5636639660f0 0x563663968510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.886+0000 7fbf7826e740 4 client.16326 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:33:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.887+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563663854ae0 con 0x5636639660f0 2024-03-21T08:33:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.887+0000 7fbf6e000640 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5636639660f0 0x563663968510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.888+0000 7fbf6e000640 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5636639660f0 0x563663968510 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5236542810c6267d server_cookie=5296e1be1522d63c in_seq=0 out_seq=0 2024-03-21T08:33:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.888+0000 7fbf4fe00640 10 client.16326 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf4fe00640 1 -- 172.21.15.3:0/2209319369 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x563663854ae0 con 0x5636639660f0 2024-03-21T08:33:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf4fe00640 10 client.16326 handle_command_reply: tid=0 2024-03-21T08:33:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 client.16326 shutdown 2024-03-21T08:33:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 msgr2=0x5636638ec4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638ec4d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006010 tx=0x7fbf600035f0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 shutdown_connections 2024-03-21T08:33:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636638f2290 0x5636638eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5636639660f0 0x563663968510 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5636638e9c00 0x5636638eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 --2- 172.21.15.3:0/2209319369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636638e8f20 0x5636638ec4d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 >> 172.21.15.3:0/2209319369 conn(0x5636638b0200 msgr2=0x5636638eea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.889+0000 7fbf4fe00640 0 client.16326 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.890+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 shutdown_connections 2024-03-21T08:33:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.890+0000 7fbf7826e740 1 -- 172.21.15.3:0/2209319369 wait complete. 2024-03-21T08:33:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.890+0000 7fbf7826e740 20 client.16326 trim_cache size 0 max 16384 2024-03-21T08:33:53.892 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "scrub active (1103 inodes in the stack)", 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: "ed2a039b-cbfc-4f4a-9c02-4e78bd2e3876": { 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "ed2a039b-cbfc-4f4a-9c02-4e78bd2e3876", 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:33:53.893 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:33:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.891+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbf540a3090 msgr2=0x7fbf540a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.891+0000 7fbf75600640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbf540a3090 0x7fbf540a5550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf68005fd0 tx=0x7fbf68009500 comp rx=0 tx=0).stop 2024-03-21T08:33:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.891+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70103860 msgr2=0x7fbf7019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.891+0000 7fbf75600640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70103860 0x7fbf7019a080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009230 tx=0x7fbf6002c710 comp rx=0 tx=0).stop 2024-03-21T08:33:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 shutdown_connections 2024-03-21T08:33:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbf540a3090 0x7fbf540a5550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf70109870 0x7fbf7019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf70104220 0x7fbf7019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 --2- 172.21.15.3:0/3989396732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf70103860 0x7fbf7019a080 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 >> 172.21.15.3:0/3989396732 conn(0x7fbf700ff310 msgr2=0x7fbf700ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 shutdown_connections 2024-03-21T08:33:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:33:53.892+0000 7fbf75600640 1 -- 172.21.15.3:0/3989396732 wait complete. 2024-03-21T08:33:53.917 INFO:tasks.cephfs.filesystem:scrub status for tag:ed2a039b-cbfc-4f4a-9c02-4e78bd2e3876 - {'path': '/', 'tag': 'ed2a039b-cbfc-4f4a-9c02-4e78bd2e3876', 'options': 'recursive,force'} 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.019+0000 7f4a7b000640 1 -- 172.21.15.3:0/968965914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74073ad0 msgr2=0x7f4a74073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.019+0000 7f4a7b000640 1 --2- 172.21.15.3:0/968965914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74073ad0 0x7f4a74073eb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a6402bcc0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.020+0000 7f4a7b000640 1 -- 172.21.15.3:0/968965914 shutdown_connections 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.020+0000 7f4a7b000640 1 --2- 172.21.15.3:0/968965914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a7410de90 0x7f4a741102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.020+0000 7f4a7b000640 1 --2- 172.21.15.3:0/968965914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a740743f0 0x7f4a7410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.020+0000 7f4a7b000640 1 --2- 172.21.15.3:0/968965914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74073ad0 0x7f4a74073eb0 secure :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a6402bcc0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.020+0000 7f4a7b000640 1 -- 172.21.15.3:0/968965914 >> 172.21.15.3:0/968965914 conn(0x7f4a740fd160 msgr2=0x7f4a740ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.020+0000 7f4a7b000640 1 -- 172.21.15.3:0/968965914 shutdown_connections 2024-03-21T08:33:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.020+0000 7f4a7b000640 1 -- 172.21.15.3:0/968965914 wait complete. 2024-03-21T08:33:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.021+0000 7f4a7b000640 1 Processor -- start 2024-03-21T08:33:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.021+0000 7f4a7b000640 1 -- start start 2024-03-21T08:33:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a7b000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a740743f0 0x7f4a7419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a7b000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a7410de90 0x7f4a7419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a7b000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a7419eb10 0x7f4a7419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a7b000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a74113aa0 con 0x7f4a7419eb10 2024-03-21T08:33:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a7b000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a74113920 con 0x7f4a7410de90 2024-03-21T08:33:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a7b000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a74113c20 con 0x7f4a740743f0 2024-03-21T08:33:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a73400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a7410de90 0x7f4a7419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a740743f0 0x7f4a7419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a73400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a7410de90 0x7f4a7419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56132/0 (socket says 172.21.15.3:56132) 2024-03-21T08:33:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.022+0000 7f4a73400640 1 -- 172.21.15.3:0/1859609708 learned_addr learned my addr 172.21.15.3:0/1859609708 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.023+0000 7f4a73400640 1 -- 172.21.15.3:0/1859609708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a740743f0 msgr2=0x7f4a7419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.023+0000 7f4a73400640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a740743f0 0x7f4a7419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.023+0000 7f4a73400640 1 -- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a7419eb10 msgr2=0x7f4a7419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.024+0000 7f4a79000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a7419eb10 0x7f4a7419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.024+0000 7f4a73400640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a7419eb10 0x7f4a7419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.024+0000 7f4a73400640 1 -- 172.21.15.3:0/1859609708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7419f600 con 0x7f4a7410de90 2024-03-21T08:33:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.024+0000 7f4a79000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a7419eb10 0x7f4a7419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.025+0000 7f4a73400640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a7410de90 0x7f4a7419a790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e9f0 tx=0x7f4a6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.025+0000 7f4a70c00640 1 -- 172.21.15.3:0/1859609708 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000cd80 con 0x7f4a7410de90 2024-03-21T08:33:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.025+0000 7f4a70c00640 1 -- 172.21.15.3:0/1859609708 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a600108e0 con 0x7f4a7410de90 2024-03-21T08:33:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.025+0000 7f4a70c00640 1 -- 172.21.15.3:0/1859609708 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60018920 con 0x7f4a7410de90 2024-03-21T08:33:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.025+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7419f8f0 con 0x7f4a7410de90 2024-03-21T08:33:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.025+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a741a7090 con 0x7f4a7410de90 2024-03-21T08:33:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.030+0000 7f4a7dd7b740 1 Processor -- start 2024-03-21T08:33:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.031+0000 7f4a7dd7b740 1 -- start start 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.031+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe6123b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.031+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe6128f0 0x561afe61b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.031+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe61dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.031+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe5d43b0 con 0x561afe611fd0 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.031+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe576830 con 0x561afe6128f0 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.031+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe51f5d0 con 0x561afe61b6c0 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe6123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe6123b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47770/0 (socket says 172.21.15.3:47770) 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 -- 172.21.15.3:0/2679859665 learned_addr learned my addr 172.21.15.3:0/2679859665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a73e00640 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe6128f0 0x561afe61b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a70c00640 1 -- 172.21.15.3:0/1859609708 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f4a60010a90 con 0x7f4a7410de90 2024-03-21T08:33:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 -- 172.21.15.3:0/2679859665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 msgr2=0x561afe61dab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T08:33:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe61dab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 -- 172.21.15.3:0/2679859665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe6128f0 msgr2=0x561afe61b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe6128f0 0x561afe61b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a79000640 1 -- 172.21.15.3:0/2679859665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561afe5372c0 con 0x561afe611fd0 2024-03-21T08:33:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.032+0000 7f4a73e00640 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe6128f0 0x561afe61b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.033+0000 7f4a79000640 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe6123b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c008de0 tx=0x7f4a6c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ad8082b8ebedc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.033+0000 7f4a69600640 1 -- 172.21.15.3:0/2679859665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00f830 con 0x561afe611fd0 2024-03-21T08:33:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.033+0000 7f4a69600640 1 -- 172.21.15.3:0/2679859665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c00de20 con 0x561afe611fd0 2024-03-21T08:33:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.033+0000 7f4a69600640 1 -- 172.21.15.3:0/2679859665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00e3b0 con 0x561afe611fd0 2024-03-21T08:33:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a70c00640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4a580a3100 0x7f4a580a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2679859665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 msgr2=0x561afe6123b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe6123b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c008de0 tx=0x7f4a6c00d710 comp rx=0 tx=0).stop 2024-03-21T08:33:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a70c00640 1 -- 172.21.15.3:0/1859609708 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f4a600cff20 con 0x7f4a7410de90 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2679859665 shutdown_connections 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe61dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe6128f0 0x561afe61b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2679859665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe6123b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.034+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2679859665 >> 172.21.15.3:0/2679859665 conn(0x561afe60c8f0 msgr2=0x561afe60ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.035+0000 7f4a73e00640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4a580a3100 0x7f4a580a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.035+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2679859665 shutdown_connections 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.035+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2679859665 wait complete. 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.036+0000 7f4a7dd7b740 1 -- start start 2024-03-21T08:33:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.036+0000 7f4a7dd7b740 1 Processor -- start 2024-03-21T08:33:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.036+0000 7f4a73e00640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4a580a3100 0x7f4a580a55c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009230 tx=0x7f4a64005e80 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe611fd0 0x561afe618790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe6128f0 0x561afe618cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe619210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe532a20 con 0x561afe6128f0 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe5728d0 con 0x561afe61b6c0 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe61e2f0 con 0x561afe611fd0 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a7dd7b740 10 client.0 _ll_register_callbacks cb 0x561afe56f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a73400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe6128f0 0x561afe618cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a73400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe6128f0 0x561afe618cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47774/0 (socket says 172.21.15.3:47774) 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a73400640 1 -- 172.21.15.3:0/592101993 learned_addr learned my addr 172.21.15.3:0/592101993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.038+0000 7f4a73e00640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe611fd0 0x561afe618790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.037+0000 7f4a79000640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe619210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.038+0000 7f4a73400640 1 -- 172.21.15.3:0/592101993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe611fd0 msgr2=0x561afe618790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.038+0000 7f4a73400640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe611fd0 0x561afe618790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.038+0000 7f4a73400640 1 -- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 msgr2=0x561afe619210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.038+0000 7f4a73400640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe619210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.038+0000 7f4a73400640 1 -- 172.21.15.3:0/592101993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561afe28b210 con 0x561afe6128f0 2024-03-21T08:33:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.038+0000 7f4a79000640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe619210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.039+0000 7f4a73e00640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe611fd0 0x561afe618790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.039+0000 7f4a73400640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe6128f0 0x561afe618cd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4a600cd300 tx=0x7f4a60010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3bbab36dd198e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.039+0000 7f4a5e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:33:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.039+0000 7f4a5e000640 1 -- 172.21.15.3:0/592101993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a600a0040 con 0x561afe6128f0 2024-03-21T08:33:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.039+0000 7f4a5e000640 1 -- 172.21.15.3:0/592101993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f4a60010430 con 0x561afe6128f0 2024-03-21T08:33:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.039+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a6c00de20 con 0x561afe6128f0 2024-03-21T08:33:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.040+0000 7f4a5e000640 1 -- 172.21.15.3:0/592101993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a600a4900 con 0x561afe6128f0 2024-03-21T08:33:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.040+0000 7f4a7dd7b740 10 client.16335 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.040+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561afe357620 con 0x561afe6128f0 2024-03-21T08:33:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.040+0000 7f4a5e000640 1 -- 172.21.15.3:0/592101993 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c00de20 con 0x561afe6128f0 2024-03-21T08:33:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.041+0000 7f4a5e000640 1 -- 172.21.15.3:0/592101993 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f4a600b8020 con 0x561afe6128f0 2024-03-21T08:33:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.041+0000 7f4a7dd7b740 10 client.16335 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.041+0000 7f4a7dd7b740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.041+0000 7f4a7dd7b740 20 client.16335 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.041+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe633f10 0x561afe636300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.041+0000 7f4a7dd7b740 4 client.16335 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.041+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561afe583d70 con 0x561afe633f10 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.042+0000 7f4a73e00640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe633f10 0x561afe636300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.042+0000 7f4a73e00640 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe633f10 0x561afe636300 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23534893d4371cab server_cookie=c03a0847c30650d5 in_seq=0 out_seq=0 2024-03-21T08:33:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.042+0000 7f4a5e000640 10 client.16335 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.045+0000 7f4a5e000640 1 -- 172.21.15.3:0/592101993 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561afe583d70 con 0x561afe633f10 2024-03-21T08:33:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.045+0000 7f4a5e000640 10 client.16335 handle_command_reply: tid=0 2024-03-21T08:33:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.045+0000 7f4a7dd7b740 1 client.16335 shutdown 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe6128f0 msgr2=0x561afe618cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe6128f0 0x561afe618cd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4a600cd300 tx=0x7f4a60010ea0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 shutdown_connections 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe619210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe6128f0 0x561afe618cd0 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4a600cd300 tx=0x7f4a60010ea0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe633f10 0x561afe636300 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/592101993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe611fd0 0x561afe618790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 >> 172.21.15.3:0/592101993 conn(0x561afe60c8f0 msgr2=0x561afe60e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.046+0000 7f4a5e000640 0 client.16335 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.047+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 shutdown_connections 2024-03-21T08:33:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.047+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/592101993 wait complete. 2024-03-21T08:33:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.047+0000 7f4a7dd7b740 20 client.16335 trim_cache size 0 max 16384 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.063+0000 7f4a7dd7b740 1 Processor -- start 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.063+0000 7f4a7dd7b740 1 -- start start 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe61aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe62f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe63bbd0 0x561afe62f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a741af590 con 0x561afe611fd0 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a741ad060 con 0x561afe61b6c0 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a741ab100 con 0x561afe63bbd0 2024-03-21T08:33:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe61aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe61aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47776/0 (socket says 172.21.15.3:47776) 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 -- 172.21.15.3:0/939483266 learned_addr learned my addr 172.21.15.3:0/939483266 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73400640 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe62f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 -- 172.21.15.3:0/939483266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe63bbd0 msgr2=0x561afe62f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a79000640 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe63bbd0 0x561afe62f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe63bbd0 0x561afe62f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 -- 172.21.15.3:0/939483266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 msgr2=0x561afe62f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe62f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73e00640 1 -- 172.21.15.3:0/939483266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561afe61de20 con 0x561afe611fd0 2024-03-21T08:33:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.064+0000 7f4a73400640 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe62f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:33:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.065+0000 7f4a73e00640 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe61aec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4a64035000 tx=0x7f4a64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be16452757d60417 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.065+0000 7f4a68c00640 1 -- 172.21.15.3:0/939483266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a640054d0 con 0x561afe611fd0 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.065+0000 7f4a68c00640 1 -- 172.21.15.3:0/939483266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a64056070 con 0x561afe611fd0 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.065+0000 7f4a68c00640 1 -- 172.21.15.3:0/939483266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64050aa0 con 0x561afe611fd0 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/939483266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 msgr2=0x561afe61aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe61aec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4a64035000 tx=0x7f4a64009260 comp rx=0 tx=0).stop 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/939483266 shutdown_connections 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe63bbd0 0x561afe62f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe61b6c0 0x561afe62f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/939483266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe611fd0 0x561afe61aec0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/939483266 >> 172.21.15.3:0/939483266 conn(0x561afe60c8f0 msgr2=0x561afe613b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/939483266 shutdown_connections 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.066+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/939483266 wait complete. 2024-03-21T08:33:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 -- start start 2024-03-21T08:33:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 Processor -- start 2024-03-21T08:33:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe611fd0 0x561afe615580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe615ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe63bbd0 0x561afe68e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a741ab700 con 0x561afe63bbd0 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a741aca00 con 0x561afe611fd0 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.067+0000 7f4a7dd7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a741abd40 con 0x561afe61b6c0 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a7dd7b740 10 client.0 _ll_register_callbacks cb 0x7f4a60009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe611fd0 0x561afe615580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe611fd0 0x561afe615580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:56176/0 (socket says 172.21.15.3:56176) 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 -- 172.21.15.3:0/2483022933 learned_addr learned my addr 172.21.15.3:0/2483022933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a79000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe63bbd0 0x561afe68e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 -- 172.21.15.3:0/2483022933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 msgr2=0x561afe615ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73400640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe615ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe615ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 -- 172.21.15.3:0/2483022933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe63bbd0 msgr2=0x561afe68e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe63bbd0 0x561afe68e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a73e00640 1 -- 172.21.15.3:0/2483022933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561afe68ecd0 con 0x561afe611fd0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.068+0000 7f4a79000640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe63bbd0 0x561afe68e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.069+0000 7f4a73400640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe615ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.069+0000 7f4a73e00640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe611fd0 0x561afe615580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a64006010 tx=0x7f4a64050100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f1220bbe220bae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.069+0000 7f4a5e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.069+0000 7f4a5e000640 1 -- 172.21.15.3:0/2483022933 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6402c870 con 0x561afe611fd0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.069+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a64056070 con 0x561afe611fd0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.069+0000 7f4a5e000640 1 -- 172.21.15.3:0/2483022933 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f4a6402ca10 con 0x561afe611fd0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a5e000640 1 -- 172.21.15.3:0/2483022933 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a64066050 con 0x561afe611fd0 2024-03-21T08:33:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 10 client.36083 fetch_fsmap learned FSMap version 26 2024-03-21T08:33:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561afe68ffb0 con 0x561afe611fd0 2024-03-21T08:33:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a5e000640 1 -- 172.21.15.3:0/2483022933 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a64056070 con 0x561afe611fd0 2024-03-21T08:33:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a5e000640 1 -- 172.21.15.3:0/2483022933 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f4a6404a040 con 0x561afe611fd0 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 10 client.36083 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 10 client.36083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 20 client.36083 populate_metadata read hostname 'smithi003' 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe6906f0 0x561afe692b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 4 client.36083 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.070+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561afe57e0c0 con 0x561afe6906f0 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.071+0000 7f4a73400640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe6906f0 0x561afe692b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.071+0000 7f4a73400640 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe6906f0 0x561afe692b10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605739f37bdfc44e server_cookie=deb34b979f8616dd in_seq=0 out_seq=0 2024-03-21T08:33:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:54.071+0000 7f4a5e000640 10 client.36083 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 776 KiB/s wr, 172 op/s 2024-03-21T08:33:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub queued for path: / 2024-03-21T08:33:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: Scrub error on inode 0x100000002d2 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/af_unix.c) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: Scrub error on inode 0x100000002d3 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/aio-dio-regress) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: Scrub error on inode 0x100000002d1 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/Makefile) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: active paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:54 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 776 KiB/s wr, 172 op/s 2024-03-21T08:33:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub queued for path: / 2024-03-21T08:33:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:33:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: Scrub error on inode 0x100000002d2 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/af_unix.c) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: Scrub error on inode 0x100000002d3 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/aio-dio-regress) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: Scrub error on inode 0x100000002d1 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/Makefile) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: active paths [/] 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:54 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 776 KiB/s wr, 172 op/s 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub queued for path: / 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: Scrub error on inode 0x100000002d2 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/af_unix.c) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: Scrub error on inode 0x100000002d3 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/aio-dio-regress) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: Scrub error on inode 0x100000002d1 (/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/Makefile) see mds.k log and `damage ls` output for details 2024-03-21T08:33:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:33:54.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: active paths [/] 2024-03-21T08:33:54.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:54.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:54 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:55.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:55 smithi084 ceph-mon[35477]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:55.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:55 smithi171 ceph-mon[35388]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:55.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:55 smithi003 ceph-mon[31675]: scrub summary: idle+waiting paths [/] 2024-03-21T08:33:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.849+0000 7f4a5e000640 1 -- 172.21.15.3:0/2483022933 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561afe57e0c0 con 0x561afe6906f0 2024-03-21T08:33:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.849+0000 7f4a5e000640 10 client.36083 handle_command_reply: tid=0 2024-03-21T08:33:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 client.36083 shutdown 2024-03-21T08:33:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe611fd0 msgr2=0x561afe615580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe611fd0 0x561afe615580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a64006010 tx=0x7f4a64050100 comp rx=0 tx=0).stop 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 shutdown_connections 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561afe63bbd0 0x561afe68e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561afe6906f0 0x561afe692b10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561afe61b6c0 0x561afe615ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 --2- 172.21.15.3:0/2483022933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561afe611fd0 0x561afe615580 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 >> 172.21.15.3:0/2483022933 conn(0x561afe60c8f0 msgr2=0x561afe60b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.850+0000 7f4a5e000640 0 client.36083 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:33:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.851+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 shutdown_connections 2024-03-21T08:33:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.851+0000 7f4a7dd7b740 1 -- 172.21.15.3:0/2483022933 wait complete. 2024-03-21T08:33:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.851+0000 7f4a7dd7b740 20 client.36083 trim_cache size 0 max 16384 2024-03-21T08:33:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:33:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:33:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:33:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:33:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.852+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4a580a3100 msgr2=0x7f4a580a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.852+0000 7f4a7b000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4a580a3100 0x7f4a580a55c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009230 tx=0x7f4a64005e80 comp rx=0 tx=0).stop 2024-03-21T08:33:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a7410de90 msgr2=0x7f4a7419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:33:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a7410de90 0x7f4a7419a790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e9f0 tx=0x7f4a6000eeb0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 shutdown_connections 2024-03-21T08:33:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a7419eb10 0x7f4a7419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a7410de90 0x7f4a7419a790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4a580a3100 0x7f4a580a55c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 --2- 172.21.15.3:0/1859609708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a740743f0 0x7f4a7419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:33:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.853+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 >> 172.21.15.3:0/1859609708 conn(0x7f4a740fd160 msgr2=0x7f4a740fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:33:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.854+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 shutdown_connections 2024-03-21T08:33:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:33:56.854+0000 7f4a7b000640 1 -- 172.21.15.3:0/1859609708 wait complete. 2024-03-21T08:33:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:56 smithi084 ceph-mon[35477]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 405 KiB/s wr, 78 op/s 2024-03-21T08:33:56.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:56 smithi171 ceph-mon[35388]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 405 KiB/s wr, 78 op/s 2024-03-21T08:33:56.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:56 smithi003 ceph-mon[31675]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 405 KiB/s wr, 78 op/s 2024-03-21T08:33:58.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:33:58 smithi084 ceph-mon[35477]: pgmap v66: 129 pgs: 129 active+clean; 25 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 709 KiB/s wr, 348 op/s 2024-03-21T08:33:58.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:33:58 smithi171 ceph-mon[35388]: pgmap v66: 129 pgs: 129 active+clean; 25 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 709 KiB/s wr, 348 op/s 2024-03-21T08:33:58.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:33:58 smithi003 ceph-mon[31675]: pgmap v66: 129 pgs: 129 active+clean; 25 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 709 KiB/s wr, 348 op/s 2024-03-21T08:34:00.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:00 smithi084 ceph-mon[35477]: pgmap v67: 129 pgs: 129 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 648 KiB/s wr, 457 op/s 2024-03-21T08:34:00.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:00 smithi171 ceph-mon[35388]: pgmap v67: 129 pgs: 129 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 648 KiB/s wr, 457 op/s 2024-03-21T08:34:00.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31671]: 2024-03-21T08:34:00.453+0000 7efc8f000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-21T08:34:00.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:00 smithi003 ceph-mon[31675]: pgmap v67: 129 pgs: 129 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 648 KiB/s wr, 457 op/s 2024-03-21T08:34:01.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:01 smithi084 ceph-mon[35477]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-21T08:34:01.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:01 smithi084 ceph-mon[35477]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:active 2024-03-21T08:34:01.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:01 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:34:01.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:01 smithi171 ceph-mon[35388]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-21T08:34:01.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:01 smithi171 ceph-mon[35388]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:active 2024-03-21T08:34:01.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:01 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:34:01.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:01 smithi003 ceph-mon[31675]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-21T08:34:01.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:01 smithi003 ceph-mon[31675]: mds.? [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] up:active 2024-03-21T08:34:01.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:01 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:34:02.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:02 smithi084 ceph-mon[35477]: pgmap v68: 129 pgs: 129 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 634 KiB/s wr, 454 op/s 2024-03-21T08:34:02.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:02 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:34:02.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:02 smithi171 ceph-mon[35388]: pgmap v68: 129 pgs: 129 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 634 KiB/s wr, 454 op/s 2024-03-21T08:34:02.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:02 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:34:02.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:02 smithi003 ceph-mon[31675]: pgmap v68: 129 pgs: 129 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 634 KiB/s wr, 454 op/s 2024-03-21T08:34:02.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:02 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:34:04.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:04 smithi084 ceph-mon[35477]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 536 op/s 2024-03-21T08:34:04.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:04 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:34:04.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:04 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:34:04.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:04 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:34:04.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:04 smithi171 ceph-mon[35388]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 536 op/s 2024-03-21T08:34:04.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:04 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:34:04.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:04 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:34:04.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:04 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:34:04.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:04 smithi003 ceph-mon[31675]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.1 MiB/s wr, 536 op/s 2024-03-21T08:34:04.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:04 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:34:04.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:04 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:34:04.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:04 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:34:06.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:06 smithi084 ceph-mon[35477]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 899 KiB/s wr, 475 op/s 2024-03-21T08:34:06.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:06 smithi084 ceph-mon[35477]: scrub summary: idle 2024-03-21T08:34:06.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:06 smithi084 ceph-mon[35477]: scrub summary: idle 2024-03-21T08:34:06.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:06 smithi084 ceph-mon[35477]: scrub summary: idle 2024-03-21T08:34:06.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:06 smithi171 ceph-mon[35388]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 899 KiB/s wr, 475 op/s 2024-03-21T08:34:06.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:06 smithi171 ceph-mon[35388]: scrub summary: idle 2024-03-21T08:34:06.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:06 smithi171 ceph-mon[35388]: scrub summary: idle 2024-03-21T08:34:06.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:06 smithi171 ceph-mon[35388]: scrub summary: idle 2024-03-21T08:34:06.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:06 smithi003 ceph-mon[31675]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 899 KiB/s wr, 475 op/s 2024-03-21T08:34:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:06 smithi003 ceph-mon[31675]: scrub summary: idle 2024-03-21T08:34:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:06 smithi003 ceph-mon[31675]: scrub summary: idle 2024-03-21T08:34:06.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:06 smithi003 ceph-mon[31675]: scrub summary: idle 2024-03-21T08:34:07.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:07 smithi084 ceph-mon[35477]: pgmap v71: 129 pgs: 129 active+clean; 29 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.2 MiB/s wr, 513 op/s 2024-03-21T08:34:07.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:07 smithi171 ceph-mon[35388]: pgmap v71: 129 pgs: 129 active+clean; 29 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.2 MiB/s wr, 513 op/s 2024-03-21T08:34:07.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:07 smithi003 ceph-mon[31675]: pgmap v71: 129 pgs: 129 active+clean; 29 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.2 MiB/s wr, 513 op/s 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.011+0000 7f0b87400640 1 -- 172.21.15.3:0/748958580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80102930 msgr2=0x7f0b80102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.011+0000 7f0b87400640 1 --2- 172.21.15.3:0/748958580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80102930 0x7f0b80102d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009900 tx=0x7f0b7002bcd0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.011+0000 7f0b87400640 1 -- 172.21.15.3:0/748958580 shutdown_connections 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.011+0000 7f0b87400640 1 --2- 172.21.15.3:0/748958580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b8010bc70 0x7f0b8010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.011+0000 7f0b87400640 1 --2- 172.21.15.3:0/748958580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b80103250 0x7f0b8010b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.011+0000 7f0b87400640 1 --2- 172.21.15.3:0/748958580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80102930 0x7f0b80102d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.011+0000 7f0b87400640 1 -- 172.21.15.3:0/748958580 >> 172.21.15.3:0/748958580 conn(0x7f0b800777b0 msgr2=0x7f0b80077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.012+0000 7f0b87400640 1 -- 172.21.15.3:0/748958580 shutdown_connections 2024-03-21T08:34:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.012+0000 7f0b87400640 1 -- 172.21.15.3:0/748958580 wait complete. 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 Processor -- start 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 -- start start 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b80102930 0x7f0b8019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80103250 0x7f0b8019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b8010bc70 0x7f0b8019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80111900 con 0x7f0b80102930 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80111780 con 0x7f0b80103250 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.013+0000 7f0b87400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80111a80 con 0x7f0b8010bc70 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b80102930 0x7f0b8019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b77e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80103250 0x7f0b8019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b80102930 0x7f0b8019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50338/0 (socket says 172.21.15.3:50338) 2024-03-21T08:34:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 -- 172.21.15.3:0/2949800630 learned_addr learned my addr 172.21.15.3:0/2949800630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b85400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b8010bc70 0x7f0b8019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 -- 172.21.15.3:0/2949800630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b8010bc70 msgr2=0x7f0b8019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b8010bc70 0x7f0b8019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 -- 172.21.15.3:0/2949800630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80103250 msgr2=0x7f0b8019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80103250 0x7f0b8019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.014+0000 7f0b84a00640 1 -- 172.21.15.3:0/2949800630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8019f000 con 0x7f0b80102930 2024-03-21T08:34:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.015+0000 7f0b84a00640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b80102930 0x7f0b8019a060 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0b70005940 tx=0x7f0b700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.015+0000 7f0b7e000640 1 -- 172.21.15.3:0/2949800630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70043020 con 0x7f0b80102930 2024-03-21T08:34:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.015+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8019f290 con 0x7f0b80102930 2024-03-21T08:34:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.015+0000 7f0b7e000640 1 -- 172.21.15.3:0/2949800630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b7002dba0 con 0x7f0b80102930 2024-03-21T08:34:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.015+0000 7f0b7e000640 1 -- 172.21.15.3:0/2949800630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70003a10 con 0x7f0b80102930 2024-03-21T08:34:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.015+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801a6b80 con 0x7f0b80102930 2024-03-21T08:34:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.018+0000 7f0b89fdf740 1 Processor -- start 2024-03-21T08:34:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b7e000640 1 -- 172.21.15.3:0/2949800630 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f0b70046020 con 0x7f0b80102930 2024-03-21T08:34:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b89fdf740 1 -- start start 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556401384fd0 0x5564013853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5564013858f0 0x55640138e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x556401390ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564013473b0 con 0x5564013858f0 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564012e9830 con 0x55640138e6c0 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564012925d0 con 0x556401384fd0 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5564013858f0 0x55640138e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b77e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x556401390ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5564013858f0 0x55640138e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50354/0 (socket says 172.21.15.3:50354) 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b84a00640 1 -- 172.21.15.3:0/304514504 learned_addr learned my addr 172.21.15.3:0/304514504 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.019+0000 7f0b77e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x556401390ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:45324/0 (socket says 172.21.15.3:45324) 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b85400640 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556401384fd0 0x5564013853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b84a00640 1 -- 172.21.15.3:0/304514504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556401384fd0 msgr2=0x5564013853b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b84a00640 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556401384fd0 0x5564013853b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b84a00640 1 -- 172.21.15.3:0/304514504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 msgr2=0x556401390ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b84a00640 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x556401390ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b84a00640 1 -- 172.21.15.3:0/304514504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564012aa2c0 con 0x5564013858f0 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b77e00640 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x556401390ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.020+0000 7f0b84a00640 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5564013858f0 0x55640138e180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0b700eeec0 tx=0x7f0b700f5c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c435d6ab446ce82e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.021+0000 7f0b75c00640 1 -- 172.21.15.3:0/304514504 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70101020 con 0x5564013858f0 2024-03-21T08:34:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.021+0000 7f0b7e000640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0b640a3080 0x7f0b640a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.021+0000 7f0b75c00640 1 -- 172.21.15.3:0/304514504 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b700f6040 con 0x5564013858f0 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.021+0000 7f0b75c00640 1 -- 172.21.15.3:0/304514504 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700ba290 con 0x5564013858f0 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.021+0000 7f0b7e000640 1 -- 172.21.15.3:0/2949800630 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f0b700f1ba0 con 0x7f0b80102930 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.021+0000 7f0b85400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0b640a3080 0x7f0b640a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 -- 172.21.15.3:0/304514504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5564013858f0 msgr2=0x55640138e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5564013858f0 0x55640138e180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0b700eeec0 tx=0x7f0b700f5c40 comp rx=0 tx=0).stop 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b85400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0b640a3080 0x7f0b640a5540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b780099a0 tx=0x7f0b78009500 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 -- 172.21.15.3:0/304514504 shutdown_connections 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x556401390ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5564013858f0 0x55640138e180 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/304514504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556401384fd0 0x5564013853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 -- 172.21.15.3:0/304514504 >> 172.21.15.3:0/304514504 conn(0x55640137f8f0 msgr2=0x55640137fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 -- 172.21.15.3:0/304514504 shutdown_connections 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.022+0000 7f0b89fdf740 1 -- 172.21.15.3:0/304514504 wait complete. 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.023+0000 7f0b89fdf740 1 -- start start 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.023+0000 7f0b89fdf740 1 Processor -- start 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556401384fd0 0x55640138b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x55640138bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55640138e6c0 0x55640138c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564012a5a20 con 0x55640138e6c0 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564012e58d0 con 0x556401384fd0 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564013910b0 con 0x5564013858f0 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b89fdf740 10 client.0 _ll_register_callbacks cb 0x5564012e21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b85400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55640138e6c0 0x55640138c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556401384fd0 0x55640138b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556401384fd0 0x55640138b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:45326/0 (socket says 172.21.15.3:45326) 2024-03-21T08:34:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b85400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55640138e6c0 0x55640138c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50366/0 (socket says 172.21.15.3:50366) 2024-03-21T08:34:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b84a00640 1 -- 172.21.15.3:0/327223720 learned_addr learned my addr 172.21.15.3:0/327223720 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.024+0000 7f0b77e00640 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x55640138bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b85400640 1 -- 172.21.15.3:0/327223720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 msgr2=0x55640138bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b85400640 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x55640138bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b85400640 1 -- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556401384fd0 msgr2=0x55640138b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b85400640 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556401384fd0 0x55640138b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b85400640 1 -- 172.21.15.3:0/327223720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556400ffe210 con 0x55640138e6c0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b84a00640 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556401384fd0 0x55640138b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b85400640 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55640138e6c0 0x55640138c130 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b780109b0 tx=0x7f0b78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ee3b28e12369ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b6e000640 1 -- 172.21.15.3:0/327223720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78004d20 con 0x55640138e6c0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b700f6040 con 0x55640138e6c0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.025+0000 7f0b6e000640 1 -- 172.21.15.3:0/327223720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f0b7801d990 con 0x55640138e6c0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.026+0000 7f0b6e000640 1 -- 172.21.15.3:0/327223720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b78032050 con 0x55640138e6c0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.026+0000 7f0b89fdf740 10 client.16353 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.026+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564010ca620 con 0x55640138e6c0 2024-03-21T08:34:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.026+0000 7f0b6e000640 1 -- 172.21.15.3:0/327223720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b700f6040 con 0x55640138e6c0 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.026+0000 7f0b6e000640 1 -- 172.21.15.3:0/327223720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f0b78005720 con 0x55640138e6c0 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.026+0000 7f0b89fdf740 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.026+0000 7f0b89fdf740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.027+0000 7f0b89fdf740 20 client.16353 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.027+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55640139e530 0x5564013a0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.027+0000 7f0b89fdf740 4 client.16353 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.027+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564012f6d70 con 0x55640139e530 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.027+0000 7f0b84a00640 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55640139e530 0x5564013a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.027+0000 7f0b84a00640 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55640139e530 0x5564013a0980 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb1cc149cd73f53 server_cookie=d2499dd5f50f8a0d in_seq=0 out_seq=0 2024-03-21T08:34:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.027+0000 7f0b6e000640 10 client.16353 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b6e000640 1 -- 172.21.15.3:0/327223720 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5564012f6d70 con 0x55640139e530 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b6e000640 10 client.16353 handle_command_reply: tid=0 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 client.16353 shutdown 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55640138e6c0 msgr2=0x55640138c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55640138e6c0 0x55640138c130 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b780109b0 tx=0x7f0b78004a40 comp rx=0 tx=0).stop 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 shutdown_connections 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55640138e6c0 0x55640138c130 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55640139e530 0x5564013a0980 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x55640138bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/327223720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556401384fd0 0x55640138b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 >> 172.21.15.3:0/327223720 conn(0x55640137f8f0 msgr2=0x556401381640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b6e000640 0 client.16353 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.029+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 shutdown_connections 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.030+0000 7f0b89fdf740 1 -- 172.21.15.3:0/327223720 wait complete. 2024-03-21T08:34:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.030+0000 7f0b89fdf740 20 client.16353 trim_cache size 0 max 16384 2024-03-21T08:34:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.046+0000 7f0b89fdf740 1 Processor -- start 2024-03-21T08:34:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.046+0000 7f0b89fdf740 1 -- start start 2024-03-21T08:34:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x55640138dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564013858f0 0x556401399730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55640138e6c0 0x556401399c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801b3310 con 0x556401384fd0 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801b0de0 con 0x5564013858f0 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801aee80 con 0x55640138e6c0 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x55640138dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b77e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564013858f0 0x556401399730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x55640138dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50368/0 (socket says 172.21.15.3:50368) 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b84a00640 1 -- 172.21.15.3:0/3869728668 learned_addr learned my addr 172.21.15.3:0/3869728668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.047+0000 7f0b77e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564013858f0 0x556401399730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:45334/0 (socket says 172.21.15.3:45334) 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b85400640 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55640138e6c0 0x556401399c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b77e00640 1 -- 172.21.15.3:0/3869728668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55640138e6c0 msgr2=0x556401399c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b77e00640 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55640138e6c0 0x556401399c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b77e00640 1 -- 172.21.15.3:0/3869728668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 msgr2=0x55640138dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b77e00640 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x55640138dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b77e00640 1 -- 172.21.15.3:0/3869728668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55640139a1b0 con 0x5564013858f0 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b84a00640 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x55640138dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b77e00640 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564013858f0 0x556401399730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55640138af10 tx=0x7f0b6800bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52aa1c48f1d06e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.048+0000 7f0b75200640 1 -- 172.21.15.3:0/3869728668 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68003d20 con 0x5564013858f0 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b75200640 1 -- 172.21.15.3:0/3869728668 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b68020070 con 0x5564013858f0 2024-03-21T08:34:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b75200640 1 -- 172.21.15.3:0/3869728668 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6801b660 con 0x5564013858f0 2024-03-21T08:34:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3869728668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564013858f0 msgr2=0x556401399730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564013858f0 0x556401399730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55640138af10 tx=0x7f0b6800bd00 comp rx=0 tx=0).stop 2024-03-21T08:34:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3869728668 shutdown_connections 2024-03-21T08:34:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55640138e6c0 0x556401399c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564013858f0 0x556401399730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3869728668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x55640138dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.049+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3869728668 >> 172.21.15.3:0/3869728668 conn(0x55640137f8f0 msgr2=0x5564013869a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.050+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3869728668 shutdown_connections 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.050+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3869728668 wait complete. 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.050+0000 7f0b89fdf740 1 -- start start 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.050+0000 7f0b89fdf740 1 Processor -- start 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x556401391ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x556401392400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b89fdf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x5564013f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801af480 con 0x556401384fd0 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801b0780 con 0x55640138e6c0 2024-03-21T08:34:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b89fdf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b801afac0 con 0x5564013858f0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b89fdf740 10 client.0 _ll_register_callbacks cb 0x7f0b78009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x556401391ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x556401391ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50382/0 (socket says 172.21.15.3:50382) 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 -- 172.21.15.3:0/3034442202 learned_addr learned my addr 172.21.15.3:0/3034442202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b85400640 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x5564013f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b77e00640 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x556401392400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 -- 172.21.15.3:0/3034442202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 msgr2=0x556401392400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x556401392400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 -- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 msgr2=0x5564013f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x5564013f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.051+0000 7f0b84a00640 1 -- 172.21.15.3:0/3034442202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564013f3480 con 0x556401384fd0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b84a00640 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x556401391ec0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0b700bd920 tx=0x7f0b70009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1604b17ec456e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b6e000640 1 -- 172.21.15.3:0/3034442202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700bd580 con 0x556401384fd0 2024-03-21T08:34:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b6e000640 1 -- 172.21.15.3:0/3034442202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f0b7002dd50 con 0x556401384fd0 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b68020070 con 0x556401384fd0 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b6e000640 1 -- 172.21.15.3:0/3034442202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b700bd760 con 0x556401384fd0 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b89fdf740 10 client.16359 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.052+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564013fa830 con 0x556401384fd0 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b6e000640 1 -- 172.21.15.3:0/3034442202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b68020070 con 0x556401384fd0 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b6e000640 1 -- 172.21.15.3:0/3034442202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f0b700ba290 con 0x556401384fd0 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b89fdf740 10 client.16359 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b89fdf740 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b89fdf740 20 client.16359 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5564013fade0 0x5564013fd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b89fdf740 4 client.16359 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5564012f10c0 con 0x5564013fade0 2024-03-21T08:34:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.053+0000 7f0b77e00640 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5564013fade0 0x5564013fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.054+0000 7f0b77e00640 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5564013fade0 0x5564013fd200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f6544704556f40 server_cookie=8ce7ea3ae7e84222 in_seq=0 out_seq=0 2024-03-21T08:34:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.054+0000 7f0b6e000640 10 client.16359 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:10.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:09 smithi084 ceph-mon[35477]: pgmap v72: 129 pgs: 129 active+clean; 30 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 934 KiB/s wr, 257 op/s 2024-03-21T08:34:10.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:09 smithi171 ceph-mon[35388]: pgmap v72: 129 pgs: 129 active+clean; 30 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 934 KiB/s wr, 257 op/s 2024-03-21T08:34:10.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:09 smithi003 ceph-mon[31675]: pgmap v72: 129 pgs: 129 active+clean; 30 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 934 KiB/s wr, 257 op/s 2024-03-21T08:34:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b6e000640 1 -- 172.21.15.3:0/3034442202 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5564012f10c0 con 0x5564013fade0 2024-03-21T08:34:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b6e000640 10 client.16359 handle_command_reply: tid=0 2024-03-21T08:34:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 client.16359 shutdown 2024-03-21T08:34:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 msgr2=0x556401391ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x556401391ec0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0b700bd920 tx=0x7f0b70009640 comp rx=0 tx=0).stop 2024-03-21T08:34:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 shutdown_connections 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55640138e6c0 0x5564013f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5564013fade0 0x5564013fd200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5564013858f0 0x556401392400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 --2- 172.21.15.3:0/3034442202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556401384fd0 0x556401391ec0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b6e000640 0 client.16359 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.360+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 >> 172.21.15.3:0/3034442202 conn(0x55640137f8f0 msgr2=0x55640138b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.361+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 shutdown_connections 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.361+0000 7f0b89fdf740 1 -- 172.21.15.3:0/3034442202 wait complete. 2024-03-21T08:34:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.361+0000 7f0b89fdf740 20 client.16359 trim_cache size 0 max 16384 2024-03-21T08:34:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:34:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:34:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:34:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:34:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0b640a3080 msgr2=0x7f0b640a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0b640a3080 0x7f0b640a5540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b780099a0 tx=0x7f0b78009500 comp rx=0 tx=0).stop 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b80102930 msgr2=0x7f0b8019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b80102930 0x7f0b8019a060 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0b70005940 tx=0x7f0b700045e0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 shutdown_connections 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f0b640a3080 0x7f0b640a5540 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b8010bc70 0x7f0b8019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b80103250 0x7f0b8019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 --2- 172.21.15.3:0/2949800630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b80102930 0x7f0b8019a060 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 >> 172.21.15.3:0/2949800630 conn(0x7f0b800777b0 msgr2=0x7f0b80107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 shutdown_connections 2024-03-21T08:34:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:10.362+0000 7f0b87400640 1 -- 172.21.15.3:0/2949800630 wait complete. 2024-03-21T08:34:11.980 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:11 smithi003 ceph-mon[31675]: pgmap v73: 129 pgs: 129 active+clean; 30 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 867 KiB/s wr, 134 op/s 2024-03-21T08:34:12.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:11 smithi084 ceph-mon[35477]: pgmap v73: 129 pgs: 129 active+clean; 30 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 867 KiB/s wr, 134 op/s 2024-03-21T08:34:12.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:11 smithi171 ceph-mon[35388]: pgmap v73: 129 pgs: 129 active+clean; 30 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 867 KiB/s wr, 134 op/s 2024-03-21T08:34:14.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:13 smithi084 ceph-mon[35477]: pgmap v74: 129 pgs: 129 active+clean; 25 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-03-21T08:34:14.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:13 smithi171 ceph-mon[35388]: pgmap v74: 129 pgs: 129 active+clean; 25 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-03-21T08:34:14.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:13 smithi003 ceph-mon[31675]: pgmap v74: 129 pgs: 129 active+clean; 25 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-03-21T08:34:16.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:15 smithi084 ceph-mon[35477]: pgmap v75: 129 pgs: 129 active+clean; 25 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 713 KiB/s wr, 240 op/s 2024-03-21T08:34:16.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:15 smithi171 ceph-mon[35388]: pgmap v75: 129 pgs: 129 active+clean; 25 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 713 KiB/s wr, 240 op/s 2024-03-21T08:34:16.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:15 smithi003 ceph-mon[31675]: pgmap v75: 129 pgs: 129 active+clean; 25 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 713 KiB/s wr, 240 op/s 2024-03-21T08:34:18.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:17 smithi084 ceph-mon[35477]: pgmap v76: 129 pgs: 129 active+clean; 26 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 775 KiB/s wr, 267 op/s 2024-03-21T08:34:18.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:17 smithi171 ceph-mon[35388]: pgmap v76: 129 pgs: 129 active+clean; 26 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 775 KiB/s wr, 267 op/s 2024-03-21T08:34:18.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:17 smithi003 ceph-mon[31675]: pgmap v76: 129 pgs: 129 active+clean; 26 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 775 KiB/s wr, 267 op/s 2024-03-21T08:34:20.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:19 smithi084 ceph-mon[35477]: pgmap v77: 129 pgs: 129 active+clean; 27 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 637 KiB/s wr, 248 op/s 2024-03-21T08:34:20.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:19 smithi171 ceph-mon[35388]: pgmap v77: 129 pgs: 129 active+clean; 27 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 637 KiB/s wr, 248 op/s 2024-03-21T08:34:20.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:19 smithi003 ceph-mon[31675]: pgmap v77: 129 pgs: 129 active+clean; 27 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 637 KiB/s wr, 248 op/s 2024-03-21T08:34:22.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:21 smithi084 ceph-mon[35477]: pgmap v78: 129 pgs: 129 active+clean; 27 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 581 KiB/s wr, 233 op/s 2024-03-21T08:34:22.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:21 smithi171 ceph-mon[35388]: pgmap v78: 129 pgs: 129 active+clean; 27 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 581 KiB/s wr, 233 op/s 2024-03-21T08:34:22.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:21 smithi003 ceph-mon[31675]: pgmap v78: 129 pgs: 129 active+clean; 27 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 581 KiB/s wr, 233 op/s 2024-03-21T08:34:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 -- 172.21.15.3:0/598441905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 msgr2=0x7f3018114900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 --2- 172.21.15.3:0/598441905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f3018114900 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3010009c10 tx=0x7f301002bee0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 -- 172.21.15.3:0/598441905 shutdown_connections 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 --2- 172.21.15.3:0/598441905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f3018114900 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 --2- 172.21.15.3:0/598441905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30180764f0 0x7f3018111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 --2- 172.21.15.3:0/598441905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3018075bd0 0x7f3018075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 -- 172.21.15.3:0/598441905 >> 172.21.15.3:0/598441905 conn(0x7f30180ff320 msgr2=0x7f3018101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 -- 172.21.15.3:0/598441905 shutdown_connections 2024-03-21T08:34:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.519+0000 7f301d800640 1 -- 172.21.15.3:0/598441905 wait complete. 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.520+0000 7f301d800640 1 Processor -- start 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f301d800640 1 -- start start 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f301d800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3018075bd0 0x7f301819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f301d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30180764f0 0x7f301819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f301d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f30181a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f301d800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018117f30 con 0x7f30180764f0 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f301d800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018117db0 con 0x7f3018112510 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f301d800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181180b0 con 0x7f3018075bd0 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f3017400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f30181a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.521+0000 7f300fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30180764f0 0x7f301819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f3017400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f30181a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57738/0 (socket says 172.21.15.3:57738) 2024-03-21T08:34:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f3017400640 1 -- 172.21.15.3:0/4099336977 learned_addr learned my addr 172.21.15.3:0/4099336977 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f3016a00640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3018075bd0 0x7f301819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f300fe00640 1 -- 172.21.15.3:0/4099336977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3018075bd0 msgr2=0x7f301819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f300fe00640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3018075bd0 0x7f301819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f300fe00640 1 -- 172.21.15.3:0/4099336977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 msgr2=0x7f30181a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f300fe00640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f30181a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f300fe00640 1 -- 172.21.15.3:0/4099336977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30181a3480 con 0x7f30180764f0 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f3017400640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f30181a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.522+0000 7f300fe00640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30180764f0 0x7f301819e9d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3000005fd0 tx=0x7f300000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.523+0000 7f300f400640 1 -- 172.21.15.3:0/4099336977 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300000cd00 con 0x7f30180764f0 2024-03-21T08:34:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.523+0000 7f300f400640 1 -- 172.21.15.3:0/4099336977 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30000035f0 con 0x7f30180764f0 2024-03-21T08:34:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.523+0000 7f300f400640 1 -- 172.21.15.3:0/4099336977 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000010690 con 0x7f30180764f0 2024-03-21T08:34:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.523+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30181a3770 con 0x7f30180764f0 2024-03-21T08:34:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.523+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30181ab000 con 0x7f30180764f0 2024-03-21T08:34:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 Processor -- start 2024-03-21T08:34:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 -- start start 2024-03-21T08:34:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f300f400640 1 -- 172.21.15.3:0/4099336977 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f30000037a0 con 0x7f30180764f0 2024-03-21T08:34:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24bbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf24723b0 con 0x558cf24affd0 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf2414830 con 0x558cf24b08f0 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f30205d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf23bd5d0 con 0x558cf24b96c0 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f3017400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.527+0000 7f3016a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f3016a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57752/0 (socket says 172.21.15.3:57752) 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f3016a00640 1 -- 172.21.15.3:0/228328632 learned_addr learned my addr 172.21.15.3:0/228328632 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f300fe00640 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24bbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f3016a00640 1 -- 172.21.15.3:0/228328632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 msgr2=0x558cf24bbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f3016a00640 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24bbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f3016a00640 1 -- 172.21.15.3:0/228328632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 msgr2=0x558cf24b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f3016a00640 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.528+0000 7f3016a00640 1 -- 172.21.15.3:0/228328632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cf23d52c0 con 0x558cf24b08f0 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.529+0000 7f3016a00640 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b9180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f300800bcd0 tx=0x7f30080107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=171b7bfa3e635f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.529+0000 7f300f400640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2ffc0a3080 0x7f2ffc0a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.529+0000 7f3007e00640 1 -- 172.21.15.3:0/228328632 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008005220 con 0x558cf24b08f0 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.529+0000 7f300f400640 1 -- 172.21.15.3:0/4099336977 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f30000cfdc0 con 0x7f30180764f0 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.529+0000 7f3007e00640 1 -- 172.21.15.3:0/228328632 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3008004c50 con 0x558cf24b08f0 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.529+0000 7f3007e00640 1 -- 172.21.15.3:0/228328632 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30080116e0 con 0x558cf24b08f0 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.529+0000 7f3017400640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2ffc0a3080 0x7f2ffc0a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 -- 172.21.15.3:0/228328632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 msgr2=0x558cf24b9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b9180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f300800bcd0 tx=0x7f30080107a0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 -- 172.21.15.3:0/228328632 shutdown_connections 2024-03-21T08:34:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24bbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b9180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 --2- 172.21.15.3:0/228328632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 -- 172.21.15.3:0/228328632 >> 172.21.15.3:0/228328632 conn(0x558cf24aa8f0 msgr2=0x558cf24aacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f3017400640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2ffc0a3080 0x7f2ffc0a5540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3010009330 tx=0x7f30100092a0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 -- 172.21.15.3:0/228328632 shutdown_connections 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.530+0000 7f30205d2740 1 -- 172.21.15.3:0/228328632 wait complete. 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 -- start start 2024-03-21T08:34:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 Processor -- start 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf23d0a20 con 0x558cf24affd0 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf24108d0 con 0x558cf24b08f0 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.531+0000 7f30205d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf24bc2f0 con 0x558cf24b96c0 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f30205d2740 10 client.0 _ll_register_callbacks cb 0x558cf240d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59598/0 (socket says 172.21.15.3:59598) 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 -- 172.21.15.3:0/4231051027 learned_addr learned my addr 172.21.15.3:0/4231051027 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f300fe00640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3017400640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 -- 172.21.15.3:0/4231051027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 msgr2=0x558cf24b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24b7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 -- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 msgr2=0x558cf24b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3016a00640 1 -- 172.21.15.3:0/4231051027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cf2129210 con 0x558cf24affd0 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.532+0000 7f3017400640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f3016a00640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b67c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3008006010 tx=0x7f30080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fdd9fcc1eed8f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f3004c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f3004c00640 1 -- 172.21.15.3:0/4231051027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008004a60 con 0x558cf24affd0 2024-03-21T08:34:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f3004c00640 1 -- 172.21.15.3:0/4231051027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3008005010 con 0x558cf24affd0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3008004c50 con 0x558cf24affd0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f3004c00640 1 -- 172.21.15.3:0/4231051027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f300802f050 con 0x558cf24affd0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f30205d2740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.533+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558cf21f5620 con 0x558cf24affd0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f3004c00640 1 -- 172.21.15.3:0/4231051027 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3008004c50 con 0x558cf24affd0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f3004c00640 1 -- 172.21.15.3:0/4231051027 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f300801b430 con 0x558cf24affd0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f30205d2740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f30205d2740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f30205d2740 20 client.16371 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f30205d2740 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf24c96e0 0x558cf24cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f30205d2740 4 client.16371 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.534+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cf2421d70 con 0x558cf24c96e0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.535+0000 7f300fe00640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf24c96e0 0x558cf24cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.535+0000 7f300fe00640 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf24c96e0 0x558cf24cbad0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeee2622ee5eb656 server_cookie=1140107844f2bfae in_seq=0 out_seq=0 2024-03-21T08:34:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.535+0000 7f3004c00640 10 client.16371 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f3004c00640 1 -- 172.21.15.3:0/4231051027 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558cf2421d70 con 0x558cf24c96e0 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f3004c00640 10 client.16371 handle_command_reply: tid=0 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 client.16371 shutdown 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 msgr2=0x558cf24b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b67c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3008006010 tx=0x7f30080048f0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 shutdown_connections 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf24c96e0 0x558cf24cbad0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b96c0 0x558cf24b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f3004c00640 0 client.16371 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 --2- 172.21.15.3:0/4231051027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24affd0 0x558cf24b67c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 >> 172.21.15.3:0/4231051027 conn(0x558cf24aa8f0 msgr2=0x558cf24ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.537+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 shutdown_connections 2024-03-21T08:34:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.538+0000 7f30205d2740 1 -- 172.21.15.3:0/4231051027 wait complete. 2024-03-21T08:34:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.538+0000 7f30205d2740 20 client.16371 trim_cache size 0 max 16384 2024-03-21T08:34:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 Processor -- start 2024-03-21T08:34:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 -- start start 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24affd0 0x558cf24b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b08f0 0x558cf24c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf24c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181b7860 con 0x558cf24b96c0 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181b5330 con 0x558cf24affd0 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.555+0000 7f30205d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181b33d0 con 0x558cf24b08f0 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf24c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf24c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59602/0 (socket says 172.21.15.3:59602) 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 -- 172.21.15.3:0/1313196032 learned_addr learned my addr 172.21.15.3:0/1313196032 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3016a00640 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24affd0 0x558cf24b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f300fe00640 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b08f0 0x558cf24c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 -- 172.21.15.3:0/1313196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b08f0 msgr2=0x558cf24c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b08f0 0x558cf24c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 -- 172.21.15.3:0/1313196032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24affd0 msgr2=0x558cf24b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24affd0 0x558cf24b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f3017400640 1 -- 172.21.15.3:0/1313196032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cf24bc1d0 con 0x558cf24b96c0 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.556+0000 7f300fe00640 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b08f0 0x558cf24c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.557+0000 7f3017400640 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf24c4f40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3010009be0 tx=0x7f3010042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe38afcefe5d95bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.557+0000 7f3007400640 1 -- 172.21.15.3:0/1313196032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010003e00 con 0x558cf24b96c0 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.557+0000 7f3007400640 1 -- 172.21.15.3:0/1313196032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f301002ce00 con 0x558cf24b96c0 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.557+0000 7f3007400640 1 -- 172.21.15.3:0/1313196032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010034900 con 0x558cf24b96c0 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 -- 172.21.15.3:0/1313196032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 msgr2=0x558cf24c4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf24c4f40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3010009be0 tx=0x7f3010042000 comp rx=0 tx=0).stop 2024-03-21T08:34:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 -- 172.21.15.3:0/1313196032 shutdown_connections 2024-03-21T08:34:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf24c4f40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24b08f0 0x558cf24c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 --2- 172.21.15.3:0/1313196032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24affd0 0x558cf24b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 -- 172.21.15.3:0/1313196032 >> 172.21.15.3:0/1313196032 conn(0x558cf24aa8f0 msgr2=0x558cf24b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 -- 172.21.15.3:0/1313196032 shutdown_connections 2024-03-21T08:34:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.558+0000 7f30205d2740 1 -- 172.21.15.3:0/1313196032 wait complete. 2024-03-21T08:34:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 -- start start 2024-03-21T08:34:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 Processor -- start 2024-03-21T08:34:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24affd0 0x558cf24bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24bd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf251dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181b39d0 con 0x558cf24b96c0 2024-03-21T08:34:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181b4cd0 con 0x558cf24b08f0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181b4010 con 0x558cf24affd0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.559+0000 7f30205d2740 10 client.0 _ll_register_callbacks cb 0x7f3010009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf251dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf251dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59612/0 (socket says 172.21.15.3:59612) 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f300fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24bd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 -- 172.21.15.3:0/758896081 learned_addr learned my addr 172.21.15.3:0/758896081 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3016a00640 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24affd0 0x558cf24bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 -- 172.21.15.3:0/758896081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24affd0 msgr2=0x558cf24bd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24affd0 0x558cf24bd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 -- 172.21.15.3:0/758896081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 msgr2=0x558cf24bd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24bd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.560+0000 7f3017400640 1 -- 172.21.15.3:0/758896081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cf251e290 con 0x558cf24b96c0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f3017400640 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf251dcd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f301002cc50 tx=0x7f3010034c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31868b782a79490 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f3004c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f3004c00640 1 -- 172.21.15.3:0/758896081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301005d030 con 0x558cf24b96c0 2024-03-21T08:34:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f301002ce00 con 0x558cf24b96c0 2024-03-21T08:34:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f3004c00640 1 -- 172.21.15.3:0/758896081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3010003e00 con 0x558cf24b96c0 2024-03-21T08:34:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f3004c00640 1 -- 172.21.15.3:0/758896081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3010004b40 con 0x558cf24b96c0 2024-03-21T08:34:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f30205d2740 10 client.16377 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.561+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558cf2525650 con 0x558cf24b96c0 2024-03-21T08:34:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f3004c00640 1 -- 172.21.15.3:0/758896081 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f301002ce00 con 0x558cf24b96c0 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f3004c00640 1 -- 172.21.15.3:0/758896081 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f30100050b0 con 0x558cf24b96c0 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f30205d2740 10 client.16377 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f30205d2740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f30205d2740 20 client.16377 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f30205d2740 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf2525c70 0x558cf2528060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f30205d2740 4 client.16377 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558cf241c0c0 con 0x558cf2525c70 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.562+0000 7f3016a00640 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf2525c70 0x558cf2528060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.563+0000 7f3016a00640 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf2525c70 0x558cf2528060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc397c78735a762 server_cookie=5de70fcbe1e15be3 in_seq=0 out_seq=0 2024-03-21T08:34:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.563+0000 7f3004c00640 10 client.16377 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f3004c00640 1 -- 172.21.15.3:0/758896081 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558cf241c0c0 con 0x558cf2525c70 2024-03-21T08:34:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f3004c00640 10 client.16377 handle_command_reply: tid=0 2024-03-21T08:34:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 client.16377 shutdown 2024-03-21T08:34:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 msgr2=0x558cf251dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf251dcd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f301002cc50 tx=0x7f3010034c90 comp rx=0 tx=0).stop 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 shutdown_connections 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cf24b96c0 0x558cf251dcd0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558cf24b08f0 0x558cf24bd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x558cf2525c70 0x558cf2528060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 --2- 172.21.15.3:0/758896081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558cf24affd0 0x558cf24bd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f3004c00640 0 client.16377 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.840+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 >> 172.21.15.3:0/758896081 conn(0x558cf24aa8f0 msgr2=0x558cf24b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.841+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 shutdown_connections 2024-03-21T08:34:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.841+0000 7f30205d2740 1 -- 172.21.15.3:0/758896081 wait complete. 2024-03-21T08:34:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.841+0000 7f30205d2740 20 client.16377 trim_cache size 0 max 16384 2024-03-21T08:34:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:34:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:34:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:34:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2ffc0a3080 msgr2=0x7f2ffc0a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2ffc0a3080 0x7f2ffc0a5540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3010009330 tx=0x7f30100092a0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30180764f0 msgr2=0x7f301819e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30180764f0 0x7f301819e9d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3000005fd0 tx=0x7f300000ee50 comp rx=0 tx=0).stop 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 shutdown_connections 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3018112510 0x7f30181a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30180764f0 0x7f301819e9d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2ffc0a3080 0x7f2ffc0a5540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 --2- 172.21.15.3:0/4099336977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3018075bd0 0x7f301819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.842+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 >> 172.21.15.3:0/4099336977 conn(0x7f30180ff320 msgr2=0x7f3018113c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.843+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 shutdown_connections 2024-03-21T08:34:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:23.843+0000 7f301d800640 1 -- 172.21.15.3:0/4099336977 wait complete. 2024-03-21T08:34:23.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T08:34:23.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T08:34:24.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:23 smithi084 ceph-mon[35477]: pgmap v79: 129 pgs: 129 active+clean; 31 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 896 KiB/s wr, 284 op/s 2024-03-21T08:34:24.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:23 smithi171 ceph-mon[35388]: pgmap v79: 129 pgs: 129 active+clean; 31 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 896 KiB/s wr, 284 op/s 2024-03-21T08:34:24.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.142+0000 7f11fb200640 1 -- 172.21.15.3:0/3188073251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc102730 msgr2=0x7f11fc102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.142+0000 7f11fb200640 1 --2- 172.21.15.3:0/3188073251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc102730 0x7f11fc102b10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009900 tx=0x7f11ec02bcc0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.142+0000 7f11fb200640 1 -- 172.21.15.3:0/3188073251 shutdown_connections 2024-03-21T08:34:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.142+0000 7f11fb200640 1 --2- 172.21.15.3:0/3188073251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11fc10ba70 0x7f11fc10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.142+0000 7f11fb200640 1 --2- 172.21.15.3:0/3188073251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc103050 0x7f11fc10b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.142+0000 7f11fb200640 1 --2- 172.21.15.3:0/3188073251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc102730 0x7f11fc102b10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.142+0000 7f11fb200640 1 -- 172.21.15.3:0/3188073251 >> 172.21.15.3:0/3188073251 conn(0x7f11fc077130 msgr2=0x7f11fc0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.143+0000 7f11fb200640 1 -- 172.21.15.3:0/3188073251 shutdown_connections 2024-03-21T08:34:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.143+0000 7f11fb200640 1 -- 172.21.15.3:0/3188073251 wait complete. 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.143+0000 7f11fb200640 1 Processor -- start 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11fb200640 1 -- start start 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11fb200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc102730 0x7f11fc199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11fb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc103050 0x7f11fc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11fb200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11fc10ba70 0x7f11fc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11fb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc111720 con 0x7f11fc103050 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11fb200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1115a0 con 0x7f11fc102730 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11fb200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1118a0 con 0x7f11fc10ba70 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc103050 0x7f11fc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc102730 0x7f11fc199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc103050 0x7f11fc19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59626/0 (socket says 172.21.15.3:59626) 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11f9400640 1 -- 172.21.15.3:0/1581420006 learned_addr learned my addr 172.21.15.3:0/1581420006 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.144+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc102730 0x7f11fc199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57786/0 (socket says 172.21.15.3:57786) 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.145+0000 7f11fa800640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11fc10ba70 0x7f11fc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.145+0000 7f11f9e00640 1 -- 172.21.15.3:0/1581420006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11fc10ba70 msgr2=0x7f11fc19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.145+0000 7f11f9e00640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11fc10ba70 0x7f11fc19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.145+0000 7f11f9e00640 1 -- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc103050 msgr2=0x7f11fc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.145+0000 7f11f9e00640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc103050 0x7f11fc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.145+0000 7f11f9e00640 1 -- 172.21.15.3:0/1581420006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc19ee30 con 0x7f11fc102730 2024-03-21T08:34:24.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.145+0000 7f11f9e00640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc102730 0x7f11fc199e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11ec02c200 tx=0x7f11ec0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.146+0000 7f11f2a00640 1 -- 172.21.15.3:0/1581420006 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec043020 con 0x7f11fc102730 2024-03-21T08:34:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.146+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc19f0c0 con 0x7f11fc102730 2024-03-21T08:34:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.146+0000 7f11f2a00640 1 -- 172.21.15.3:0/1581420006 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11ec02dc10 con 0x7f11fc102730 2024-03-21T08:34:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.146+0000 7f11f2a00640 1 -- 172.21.15.3:0/1581420006 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec003970 con 0x7f11fc102730 2024-03-21T08:34:24.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.146+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc1a69b0 con 0x7f11fc102730 2024-03-21T08:34:24.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.149+0000 7f1203a1f740 1 Processor -- start 2024-03-21T08:34:24.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.149+0000 7f1203a1f740 1 -- start start 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f11f2a00640 1 -- 172.21.15.3:0/1581420006 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f11ec046020 con 0x7f11fc102730 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b49b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b53460 0x55d2f7b55850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f1203a1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7aae250 con 0x55d2f7b53460 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f1203a1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7a56a30 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f1203a1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7aaa2f0 con 0x55d2f7b4add0 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f11f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b53460 0x55d2f7b55850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f11f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b53460 0x55d2f7b55850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59640/0 (socket says 172.21.15.3:59640) 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f11f9400640 1 -- 172.21.15.3:0/2850379627 learned_addr learned my addr 172.21.15.3:0/2850379627 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.150+0000 7f11f9e00640 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11fa800640 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b49b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11f9400640 1 -- 172.21.15.3:0/2850379627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 msgr2=0x55d2f7b49b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11f9400640 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b49b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11f9400640 1 -- 172.21.15.3:0/2850379627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4a0f0 msgr2=0x55d2f7b4a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11f9400640 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11f9400640 1 -- 172.21.15.3:0/2850379627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f7a41690 con 0x55d2f7b53460 2024-03-21T08:34:24.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11fa800640 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b49b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.151+0000 7f11f9400640 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b53460 0x55d2f7b55850 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009da0 tx=0x7f11e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193091fc22bec3fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.152+0000 7f11e7000640 1 -- 172.21.15.3:0/2850379627 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e801e070 con 0x55d2f7b53460 2024-03-21T08:34:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.152+0000 7f11e7000640 1 -- 172.21.15.3:0/2850379627 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11e8019ae0 con 0x55d2f7b53460 2024-03-21T08:34:24.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.152+0000 7f11e7000640 1 -- 172.21.15.3:0/2850379627 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800fa50 con 0x55d2f7b53460 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.152+0000 7f11f2a00640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11e00a3090 0x7f11e00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.152+0000 7f11f2a00640 1 -- 172.21.15.3:0/1581420006 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f11ec0f1b50 con 0x7f11fc102730 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.152+0000 7f11fa800640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11e00a3090 0x7f11e00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 -- 172.21.15.3:0/2850379627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b53460 msgr2=0x55d2f7b55850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b53460 0x55d2f7b55850 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009da0 tx=0x7f11e8011810 comp rx=0 tx=0).stop 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 -- 172.21.15.3:0/2850379627 shutdown_connections 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b53460 0x55d2f7b55850 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2850379627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b49b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 -- 172.21.15.3:0/2850379627 >> 172.21.15.3:0/2850379627 conn(0x55d2f7b11200 msgr2=0x55d2f7b44b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 -- 172.21.15.3:0/2850379627 shutdown_connections 2024-03-21T08:34:24.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f11fa800640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11e00a3090 0x7f11e00a5550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11f40092f0 tx=0x7f11f4009280 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.153+0000 7f1203a1f740 1 -- 172.21.15.3:0/2850379627 wait complete. 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.154+0000 7f1203a1f740 1 -- start start 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.154+0000 7f1203a1f740 1 Processor -- start 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.154+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b52a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.154+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.154+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b53460 0x55d2f7b503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.154+0000 7f1203a1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7b562d0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f1203a1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7a69e80 con 0x55d2f7b4add0 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f1203a1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7b56450 con 0x55d2f7b53460 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f1203a1f740 10 client.0 _ll_register_callbacks cb 0x55d2f7a7c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b52a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b52a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59652/0 (socket says 172.21.15.3:59652) 2024-03-21T08:34:24.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 -- 172.21.15.3:0/3419394710 learned_addr learned my addr 172.21.15.3:0/3419394710 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9400640 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 -- 172.21.15.3:0/3419394710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b53460 msgr2=0x55d2f7b503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b53460 0x55d2f7b503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 -- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4add0 msgr2=0x55d2f7b4fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 -- 172.21.15.3:0/3419394710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2f788c7d0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.155+0000 7f11f9e00640 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b52a90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0eef90 tx=0x7f11ec009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba211572bba90db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f11db400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f11db400640 1 -- 172.21.15.3:0/3419394710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0bd710 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f11db400640 1 -- 172.21.15.3:0/3419394710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f11ec0c6150 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11e8019ae0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f11db400640 1 -- 172.21.15.3:0/3419394710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11ec0bd530 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f1203a1f740 10 client.16389 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d2f77c23f0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f11db400640 1 -- 172.21.15.3:0/3419394710 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11e8019ae0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.156+0000 7f11db400640 1 -- 172.21.15.3:0/3419394710 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f11ec003b10 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.157+0000 7f1203a1f740 10 client.16389 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.157+0000 7f1203a1f740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.157+0000 7f1203a1f740 20 client.16389 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:24.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.157+0000 7f1203a1f740 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7b63200 0x55d2f7b65680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.157+0000 7f1203a1f740 4 client.16389 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.157+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2f7abb790 con 0x55d2f7b63200 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.157+0000 7f11f9400640 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7b63200 0x55d2f7b65680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.158+0000 7f11f9400640 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7b63200 0x55d2f7b65680 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e834be161cab169 server_cookie=aab0d899d5f826cb in_seq=0 out_seq=0 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.158+0000 7f11db400640 10 client.16389 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f11db400640 1 -- 172.21.15.3:0/3419394710 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d2f7abb790 con 0x55d2f7b63200 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f11db400640 10 client.16389 handle_command_reply: tid=0 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 client.16389 shutdown 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 msgr2=0x55d2f7b52a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b52a90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0eef90 tx=0x7f11ec009620 comp rx=0 tx=0).stop 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 shutdown_connections 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7b63200 0x55d2f7b65680 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b53460 0x55d2f7b503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 --2- 172.21.15.3:0/3419394710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b52a90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f11db400640 0 client.16389 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 >> 172.21.15.3:0/3419394710 conn(0x55d2f7b11200 msgr2=0x55d2f7b46890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 shutdown_connections 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 1 -- 172.21.15.3:0/3419394710 wait complete. 2024-03-21T08:34:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.159+0000 7f1203a1f740 20 client.16389 trim_cache size 0 max 16384 2024-03-21T08:34:24.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:23 smithi003 ceph-mon[31675]: pgmap v79: 129 pgs: 129 active+clean; 31 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 896 KiB/s wr, 284 op/s 2024-03-21T08:34:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.180+0000 7f1203a1f740 1 Processor -- start 2024-03-21T08:34:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f1203a1f740 1 -- start start 2024-03-21T08:34:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f1203a1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1b30e0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f1203a1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1b0bb0 con 0x55d2f7b53460 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f1203a1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1aec50 con 0x55d2f7b4add0 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f11f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59654/0 (socket says 172.21.15.3:59654) 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.181+0000 7f11f9e00640 1 -- 172.21.15.3:0/2111121805 learned_addr learned my addr 172.21.15.3:0/2111121805 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11fa800640 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11f9400640 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11f9e00640 1 -- 172.21.15.3:0/2111121805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 msgr2=0x55d2f7b466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11f9e00640 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11f9e00640 1 -- 172.21.15.3:0/2111121805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 msgr2=0x55d2f7b525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11f9e00640 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11f9e00640 1 -- 172.21.15.3:0/2111121805 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f77820f0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11fa800640 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.182+0000 7f11f9e00640 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4f7f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0bdb60 tx=0x7f11ec0bda80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49245008b99fa0e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.183+0000 7f11e6600640 1 -- 172.21.15.3:0/2111121805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0bd710 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.183+0000 7f11e6600640 1 -- 172.21.15.3:0/2111121805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11ec0db020 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.183+0000 7f11e6600640 1 -- 172.21.15.3:0/2111121805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0f52f0 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.183+0000 7f1203a1f740 1 -- 172.21.15.3:0/2111121805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 msgr2=0x55d2f7b4f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.183+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4f7f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0bdb60 tx=0x7f11ec0bda80 comp rx=0 tx=0).stop 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 -- 172.21.15.3:0/2111121805 shutdown_connections 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4add0 0x55d2f7b466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2111121805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4f7f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 -- 172.21.15.3:0/2111121805 >> 172.21.15.3:0/2111121805 conn(0x55d2f7b11200 msgr2=0x55d2f7b45f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 -- 172.21.15.3:0/2111121805 shutdown_connections 2024-03-21T08:34:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 -- 172.21.15.3:0/2111121805 wait complete. 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 -- start start 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.184+0000 7f1203a1f740 1 Processor -- start 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f1203a1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b4d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f1203a1f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1af250 con 0x55d2f7b4add0 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f1203a1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1b0550 con 0x55d2f7b53460 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f1203a1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1af890 con 0x55d2f7b4a0f0 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f1203a1f740 10 client.0 _ll_register_callbacks cb 0x7f11ec005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f11f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f11f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59662/0 (socket says 172.21.15.3:59662) 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f11f9400640 1 -- 172.21.15.3:0/2943658611 learned_addr learned my addr 172.21.15.3:0/2943658611 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.185+0000 7f11fa800640 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11f9400640 1 -- 172.21.15.3:0/2943658611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4a0f0 msgr2=0x55d2f7b4c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11f9400640 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11f9400640 1 -- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 msgr2=0x55d2f7b4d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11f9400640 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b4d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11f9400640 1 -- 172.21.15.3:0/2943658611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2f7bbdb50 con 0x55d2f7b4add0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11fa800640 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b4d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11f9400640 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4cc40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f11e8011d10 tx=0x7f11e8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6889332a6d174c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11db400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11db400640 1 -- 172.21.15.3:0/2943658611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e801e040 con 0x55d2f7b4add0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f11db400640 1 -- 172.21.15.3:0/2943658611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f11e80051b0 con 0x55d2f7b4add0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.186+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11ec0db020 con 0x55d2f7b4add0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f11db400640 1 -- 172.21.15.3:0/2943658611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e803b050 con 0x55d2f7b4add0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f1203a1f740 10 client.16395 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d2f7bbee20 con 0x55d2f7b4add0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f11db400640 1 -- 172.21.15.3:0/2943658611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11ec0db020 con 0x55d2f7b4add0 2024-03-21T08:34:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f11db400640 1 -- 172.21.15.3:0/2943658611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f11e8019430 con 0x55d2f7b4add0 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f1203a1f740 10 client.16395 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f1203a1f740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f1203a1f740 20 client.16395 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.187+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7bbf400 0x55d2f7bc1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.188+0000 7f1203a1f740 4 client.16395 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.188+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d2f7ab5ae0 con 0x55d2f7bbf400 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.188+0000 7f11f9e00640 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7bbf400 0x55d2f7bc1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.188+0000 7f11f9e00640 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7bbf400 0x55d2f7bc1820 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e503d7832a6811ea server_cookie=1b5d57b92cdfa5d8 in_seq=0 out_seq=0 2024-03-21T08:34:24.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.188+0000 7f11db400640 10 client.16395 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f11db400640 1 -- 172.21.15.3:0/2943658611 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d2f7ab5ae0 con 0x55d2f7bbf400 2024-03-21T08:34:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f11db400640 10 client.16395 handle_command_reply: tid=0 2024-03-21T08:34:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 client.16395 shutdown 2024-03-21T08:34:24.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4add0 msgr2=0x55d2f7b4cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4cc40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f11e8011d10 tx=0x7f11e8011e50 comp rx=0 tx=0).stop 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 shutdown_connections 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2f7b53460 0x55d2f7b4d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d2f7b4add0 0x55d2f7b4cc40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55d2f7bbf400 0x55d2f7bc1820 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 --2- 172.21.15.3:0/2943658611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d2f7b4a0f0 0x55d2f7b4c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f11db400640 0 client.16395 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 >> 172.21.15.3:0/2943658611 conn(0x55d2f7b11200 msgr2=0x55d2f7b43bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 shutdown_connections 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 1 -- 172.21.15.3:0/2943658611 wait complete. 2024-03-21T08:34:24.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.189+0000 7f1203a1f740 20 client.16395 trim_cache size 0 max 16384 2024-03-21T08:34:24.196 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:34:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-03-21T08:34:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-03-21T08:34:24.196 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:34:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11e00a3090 msgr2=0x7f11e00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11e00a3090 0x7f11e00a5550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11f40092f0 tx=0x7f11f4009280 comp rx=0 tx=0).stop 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc102730 msgr2=0x7f11fc199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc102730 0x7f11fc199e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11ec02c200 tx=0x7f11ec0045e0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 shutdown_connections 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f11e00a3090 0x7f11e00a5550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11fc10ba70 0x7f11fc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11fc103050 0x7f11fc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 --2- 172.21.15.3:0/1581420006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11fc102730 0x7f11fc199e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 >> 172.21.15.3:0/1581420006 conn(0x7f11fc077130 msgr2=0x7f11fc100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 shutdown_connections 2024-03-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.190+0000 7f11fb200640 1 -- 172.21.15.3:0/1581420006 wait complete. 2024-03-21T08:34:24.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T08:34:24.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:34:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.438+0000 7f98b3600640 1 -- 172.21.15.3:0/3620149898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 msgr2=0x7f98ac1104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.438+0000 7f98b3600640 1 --2- 172.21.15.3:0/3620149898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 0x7f98ac1104a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f989c009930 tx=0x7f989c02bc20 comp rx=0 tx=0).stop 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.438+0000 7f98b3600640 1 -- 172.21.15.3:0/3620149898 shutdown_connections 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.438+0000 7f98b3600640 1 --2- 172.21.15.3:0/3620149898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 0x7f98ac116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.438+0000 7f98b3600640 1 --2- 172.21.15.3:0/3620149898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98ac1086a0 0x7f98ac108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.438+0000 7f98b3600640 1 --2- 172.21.15.3:0/3620149898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 0x7f98ac1104a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.438+0000 7f98b3600640 1 -- 172.21.15.3:0/3620149898 >> 172.21.15.3:0/3620149898 conn(0x7f98ac06b660 msgr2=0x7f98ac06da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.439+0000 7f98b3600640 1 -- 172.21.15.3:0/3620149898 shutdown_connections 2024-03-21T08:34:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.439+0000 7f98b3600640 1 -- 172.21.15.3:0/3620149898 wait complete. 2024-03-21T08:34:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.440+0000 7f98b3600640 1 Processor -- start 2024-03-21T08:34:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.440+0000 7f98b3600640 1 -- start start 2024-03-21T08:34:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.440+0000 7f98b3600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98ac1086a0 0x7f98ac1a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.440+0000 7f98b3600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 0x7f98ac1a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.440+0000 7f98b3600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 0x7f98ac1a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b3600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac11a100 con 0x7f98ac1100c0 2024-03-21T08:34:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b3600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac119f80 con 0x7f98ac109060 2024-03-21T08:34:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b3600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac11a280 con 0x7f98ac1086a0 2024-03-21T08:34:24.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 0x7f98ac1a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98abe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 0x7f98ac1a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b0c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98ac1086a0 0x7f98ac1a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98abe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 0x7f98ac1a8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57820/0 (socket says 172.21.15.3:57820) 2024-03-21T08:34:24.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98abe00640 1 -- 172.21.15.3:0/3877906023 learned_addr learned my addr 172.21.15.3:0/3877906023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:24.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b1600640 1 -- 172.21.15.3:0/3877906023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98ac1086a0 msgr2=0x7f98ac1a85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b1600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98ac1086a0 0x7f98ac1a85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.441+0000 7f98b1600640 1 -- 172.21.15.3:0/3877906023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 msgr2=0x7f98ac1a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98b1600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 0x7f98ac1a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98b1600640 1 -- 172.21.15.3:0/3877906023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac1a2f60 con 0x7f98ac1100c0 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98abe00640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 0x7f98ac1a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98b1600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 0x7f98ac1a26a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009fd0 tx=0x7f98a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98a9600640 1 -- 172.21.15.3:0/3877906023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400ce20 con 0x7f98ac1100c0 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98a9600640 1 -- 172.21.15.3:0/3877906023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98a4005020 con 0x7f98ac1100c0 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98a9600640 1 -- 172.21.15.3:0/3877906023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4010690 con 0x7f98ac1100c0 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.442+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac1a3250 con 0x7f98ac1100c0 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.443+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac1af560 con 0x7f98ac1100c0 2024-03-21T08:34:24.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.444+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac0699c0 con 0x7f98ac1100c0 2024-03-21T08:34:24.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.446+0000 7f98a9600640 1 -- 172.21.15.3:0/3877906023 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f98a40051d0 con 0x7f98ac1100c0 2024-03-21T08:34:24.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.448+0000 7f98a9600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f98940a3110 0x7f98940a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:24.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.448+0000 7f98a9600640 1 -- 172.21.15.3:0/3877906023 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f98a40cfe00 con 0x7f98ac1100c0 2024-03-21T08:34:24.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.448+0000 7f98b0c00640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f98940a3110 0x7f98940a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.449+0000 7f98b0c00640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f98940a3110 0x7f98940a55d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c00 tx=0x7f989c005fd0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.454+0000 7f98a9600640 1 -- 172.21.15.3:0/3877906023 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f98a409c4d0 con 0x7f98ac1100c0 2024-03-21T08:34:24.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.739+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98ac1a3580 con 0x7f98ac1100c0 2024-03-21T08:34:24.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.741+0000 7f98a9600640 1 -- 172.21.15.3:0/3877906023 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f98ac1a3580 con 0x7f98ac1100c0 2024-03-21T08:34:24.743 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:34:24.743 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3089491924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3089491924},{"type":"v1","addr":"172.21.15.3:6835","nonce":3089491924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4111852519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4111852519},{"type":"v1","addr":"172.21.15.3:6837","nonce":4111852519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3720466130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3720466130},{"type":"v1","addr":"172.21.15.3:6839","nonce":3720466130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2348469157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2348469157},{"type":"v1","addr":"172.21.15.3:6841","nonce":2348469157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":14673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/1820502310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":1820502310},{"type":"v1","addr":"172.21.15.84:6839","nonce":1820502310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3477321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3477321197},{"type":"v1","addr":"172.21.15.171:6833","nonce":3477321197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:28:36.113879+0000","modified":"2024-03-21T08:34:00.343604+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34472,"mds_1":34439,"mds_2":34409},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/1859190507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1859190507},{"type":"v1","addr":"172.21.15.171:6835","nonce":1859190507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2662792128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2662792128},{"type":"v1","addr":"172.21.15.171:6837","nonce":2662792128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/1010033316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1010033316},{"type":"v1","addr":"172.21.15.171:6839","nonce":1010033316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34409":{"gid":34409,"name":"k","rank":2,"incarnation":20,"state":"up:active","state_seq":94,"addr":"172.21.15.84:6833/4091779255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4091779255},{"type":"v1","addr":"172.21.15.84:6833","nonce":4091779255}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34439":{"gid":34439,"name":"e","rank":1,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.84:6835/2576104068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2576104068},{"type":"v1","addr":"172.21.15.84:6835","nonce":2576104068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.84:6837/3332280865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3332280865},{"type":"v1","addr":"172.21.15.84:6837","nonce":3332280865}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34439,34472,34409]},"id":1}]} 2024-03-21T08:34:24.744 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-03-21T08:34:24.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.744+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f98940a3110 msgr2=0x7f98940a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.744+0000 7f98b3600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f98940a3110 0x7f98940a55d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c00 tx=0x7f989c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.744+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 msgr2=0x7f98ac1a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:24.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.744+0000 7f98b3600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 0x7f98ac1a26a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009fd0 tx=0x7f98a400ef10 comp rx=0 tx=0).stop 2024-03-21T08:34:24.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.745+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 shutdown_connections 2024-03-21T08:34:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.745+0000 7f98b3600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98ac1100c0 0x7f98ac1a26a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.745+0000 7f98b3600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98ac109060 0x7f98ac1a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.745+0000 7f98b3600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f98940a3110 0x7f98940a55d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.745+0000 7f98b3600640 1 --2- 172.21.15.3:0/3877906023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98ac1086a0 0x7f98ac1a85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.745+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 >> 172.21.15.3:0/3877906023 conn(0x7f98ac06b660 msgr2=0x7f98ac06db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.745+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 shutdown_connections 2024-03-21T08:34:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:24.746+0000 7f98b3600640 1 -- 172.21.15.3:0/3877906023 wait complete. 2024-03-21T08:34:24.768 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:34:24.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T08:34:25.035 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:24 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3877906023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 -- 172.21.15.3:0/3563661838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 msgr2=0x7f62bc114900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 --2- 172.21.15.3:0/3563661838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc114900 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009be0 tx=0x7f62b402bf80 comp rx=0 tx=0).stop 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 -- 172.21.15.3:0/3563661838 shutdown_connections 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 --2- 172.21.15.3:0/3563661838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc114900 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 --2- 172.21.15.3:0/3563661838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62bc107aa0 0x7f62bc107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 --2- 172.21.15.3:0/3563661838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc10c980 0x7f62bc10cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 -- 172.21.15.3:0/3563661838 >> 172.21.15.3:0/3563661838 conn(0x7f62bc0772a0 msgr2=0x7f62bc0796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 -- 172.21.15.3:0/3563661838 shutdown_connections 2024-03-21T08:34:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.045+0000 7f62c1e00640 1 -- 172.21.15.3:0/3563661838 wait complete. 2024-03-21T08:34:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 Processor -- start 2024-03-21T08:34:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 -- start start 2024-03-21T08:34:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc107aa0 0x7f62bc19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62bc10c980 0x7f62bc1a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc117d20 con 0x7f62bc108440 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc117ba0 con 0x7f62bc107aa0 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.046+0000 7f62c1e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc117ea0 con 0x7f62bc10c980 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62c0a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc107aa0 0x7f62bc19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59694/0 (socket says 172.21.15.3:59694) 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 -- 172.21.15.3:0/62960657 learned_addr learned my addr 172.21.15.3:0/62960657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62c1400640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62bc10c980 0x7f62bc1a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 -- 172.21.15.3:0/62960657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62bc10c980 msgr2=0x7f62bc1a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62bc10c980 0x7f62bc1a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 -- 172.21.15.3:0/62960657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc107aa0 msgr2=0x7f62bc19e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc107aa0 0x7f62bc19e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62bbe00640 1 -- 172.21.15.3:0/62960657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc1a31e0 con 0x7f62bc108440 2024-03-21T08:34:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62c1400640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62bc10c980 0x7f62bc1a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:34:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.047+0000 7f62c0a00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc107aa0 0x7f62bc19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.048+0000 7f62bbe00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc19e780 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9f0 tx=0x7f62a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.048+0000 7f62b9600640 1 -- 172.21.15.3:0/62960657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a800cd80 con 0x7f62bc108440 2024-03-21T08:34:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.048+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc1a34d0 con 0x7f62bc108440 2024-03-21T08:34:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.048+0000 7f62b9600640 1 -- 172.21.15.3:0/62960657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62a80108e0 con 0x7f62bc108440 2024-03-21T08:34:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.048+0000 7f62b9600640 1 -- 172.21.15.3:0/62960657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8018920 con 0x7f62bc108440 2024-03-21T08:34:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.048+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc1aad60 con 0x7f62bc108440 2024-03-21T08:34:25.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 Processor -- start 2024-03-21T08:34:25.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 -- start start 2024-03-21T08:34:25.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62b9600640 1 -- 172.21.15.3:0/62960657 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f62a8010a90 con 0x7f62bc108440 2024-03-21T08:34:25.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247f2252c0 con 0x56247f2c2260 2024-03-21T08:34:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247f1cdc90 con 0x56247f2c18b0 2024-03-21T08:34:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.052+0000 7f62c61d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247f221360 con 0x56247f2cade0 2024-03-21T08:34:25.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57854/0 (socket says 172.21.15.3:57854) 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 -- 172.21.15.3:0/3362433289 learned_addr learned my addr 172.21.15.3:0/3362433289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62bbe00640 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 -- 172.21.15.3:0/3362433289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 msgr2=0x56247f2cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 -- 172.21.15.3:0/3362433289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 msgr2=0x56247f2ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c1400640 1 -- 172.21.15.3:0/3362433289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247f1f2080 con 0x56247f2c18b0 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.053+0000 7f62c0a00640 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.054+0000 7f62c1400640 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c1c90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009370 tx=0x7f62b4009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4730117eacda19af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.054+0000 7f62b9600640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f62a40a3090 0x7f62a40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.054+0000 7f62b9600640 1 -- 172.21.15.3:0/62960657 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f62a80cfff0 con 0x7f62bc108440 2024-03-21T08:34:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.054+0000 7f62adc00640 1 -- 172.21.15.3:0/3362433289 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4042070 con 0x56247f2c18b0 2024-03-21T08:34:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.054+0000 7f62adc00640 1 -- 172.21.15.3:0/3362433289 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62b4033dd0 con 0x56247f2c18b0 2024-03-21T08:34:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.054+0000 7f62c0a00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f62a40a3090 0x7f62a40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.054+0000 7f62adc00640 1 -- 172.21.15.3:0/3362433289 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b402cc10 con 0x56247f2c18b0 2024-03-21T08:34:25.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.055+0000 7f62c61d0740 1 -- 172.21.15.3:0/3362433289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 msgr2=0x56247f2c1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.055+0000 7f62c61d0740 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c1c90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009370 tx=0x7f62b4009950 comp rx=0 tx=0).stop 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c61d0740 1 -- 172.21.15.3:0/3362433289 shutdown_connections 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c61d0740 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c61d0740 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c61d0740 1 --2- 172.21.15.3:0/3362433289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c1c90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c61d0740 1 -- 172.21.15.3:0/3362433289 >> 172.21.15.3:0/3362433289 conn(0x56247f2bbf80 msgr2=0x56247f2bc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c0a00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f62a40a3090 0x7f62a40a5550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56247f2bcff0 tx=0x7f62b0009260 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c61d0740 1 -- 172.21.15.3:0/3362433289 shutdown_connections 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.056+0000 7f62c61d0740 1 -- 172.21.15.3:0/3362433289 wait complete. 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.057+0000 7f62c61d0740 1 -- start start 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.057+0000 7f62c61d0740 1 Processor -- start 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.057+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c18b0 0x56247f2ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.057+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2cade0 0x56247f2c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c61d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247f2cdcf0 con 0x56247f2c2260 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c61d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247f1e10d0 con 0x56247f2cade0 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c61d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247f2cde70 con 0x56247f2c18b0 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c61d0740 10 client.0 _ll_register_callbacks cb 0x56247f1e5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2cade0 0x56247f2c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2cade0 0x56247f2c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57860/0 (socket says 172.21.15.3:57860) 2024-03-21T08:34:25.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c1400640 1 -- 172.21.15.3:0/821394740 learned_addr learned my addr 172.21.15.3:0/821394740 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62c0a00640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c18b0 0x56247f2ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.058+0000 7f62bbe00640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62c1400640 1 -- 172.21.15.3:0/821394740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c18b0 msgr2=0x56247f2ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62c1400640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c18b0 0x56247f2ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62c1400640 1 -- 172.21.15.3:0/821394740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 msgr2=0x56247f2c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62c1400640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62c1400640 1 -- 172.21.15.3:0/821394740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56247ef393f0 con 0x56247f2cade0 2024-03-21T08:34:25.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62bbe00640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:25.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62c1400640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2cade0 0x56247f2c8260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009b30 tx=0x7f62b4033980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930e6d2164061292 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.059+0000 7f62a2000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.060+0000 7f62a2000640 1 -- 172.21.15.3:0/821394740 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4042040 con 0x56247f2cade0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.060+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62b4033dd0 con 0x56247f2cade0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.060+0000 7f62a2000640 1 -- 172.21.15.3:0/821394740 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f62b402cc20 con 0x56247f2cade0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.060+0000 7f62a2000640 1 -- 172.21.15.3:0/821394740 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62b4056050 con 0x56247f2cade0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.060+0000 7f62c61d0740 10 client.36152 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.060+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56247f0037d0 con 0x56247f2cade0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.060+0000 7f62a2000640 1 -- 172.21.15.3:0/821394740 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62b4033dd0 con 0x56247f2cade0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62a2000640 1 -- 172.21.15.3:0/821394740 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f62b4004580 con 0x56247f2cade0 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62c61d0740 10 client.36152 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62c61d0740 10 client.36152 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62c61d0740 20 client.36152 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62c61d0740 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f2dab40 0x56247f2dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62c61d0740 4 client.36152 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56247f232800 con 0x56247f2dab40 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.061+0000 7f62c0a00640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f2dab40 0x56247f2dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.062+0000 7f62a2000640 10 client.36152 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.062+0000 7f62c0a00640 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f2dab40 0x56247f2dcf30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=139677d4a8d0b6ab server_cookie=4ef27f376933841f in_seq=0 out_seq=0 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62a2000640 1 -- 172.21.15.3:0/821394740 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56247f232800 con 0x56247f2dab40 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62a2000640 10 client.36152 handle_command_reply: tid=0 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 client.36152 shutdown 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2cade0 msgr2=0x56247f2c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2cade0 0x56247f2c8260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009b30 tx=0x7f62b4033980 comp rx=0 tx=0).stop 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 shutdown_connections 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f2dab40 0x56247f2dcf30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2cade0 0x56247f2c8260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 --2- 172.21.15.3:0/821394740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c18b0 0x56247f2ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62a2000640 0 client.36152 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:34:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 >> 172.21.15.3:0/821394740 conn(0x56247f2bbf80 msgr2=0x56247f2bdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.063+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 shutdown_connections 2024-03-21T08:34:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.064+0000 7f62c61d0740 1 -- 172.21.15.3:0/821394740 wait complete. 2024-03-21T08:34:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.064+0000 7f62c61d0740 20 client.36152 trim_cache size 0 max 16384 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.079+0000 7f62c61d0740 1 Processor -- start 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.079+0000 7f62c61d0740 1 -- start start 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c2260 0x56247f2ca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2cade0 0x56247f2ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c61d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1b7500 con 0x56247f2cade0 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c61d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1b4fd0 con 0x56247f2c18b0 2024-03-21T08:34:25.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c61d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1b3070 con 0x56247f2c2260 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2cade0 0x56247f2ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2cade0 0x56247f2ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59716/0 (socket says 172.21.15.3:59716) 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 -- 172.21.15.3:0/1360159493 learned_addr learned my addr 172.21.15.3:0/1360159493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c0a00640 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62bbe00640 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c2260 0x56247f2ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 -- 172.21.15.3:0/1360159493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c2260 msgr2=0x56247f2ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c2260 0x56247f2ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 -- 172.21.15.3:0/1360159493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 msgr2=0x56247f2c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c1400640 1 -- 172.21.15.3:0/1360159493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247f2dd2a0 con 0x56247f2cade0 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.080+0000 7f62c0a00640 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.081+0000 7f62c1400640 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2cade0 0x56247f2ca790 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f62b4033ff0 tx=0x7f62b4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2747cece99961ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.081+0000 7f62ad200640 1 -- 172.21.15.3:0/1360159493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4042040 con 0x56247f2cade0 2024-03-21T08:34:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.081+0000 7f62ad200640 1 -- 172.21.15.3:0/1360159493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62b404b020 con 0x56247f2cade0 2024-03-21T08:34:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.081+0000 7f62ad200640 1 -- 172.21.15.3:0/1360159493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4047720 con 0x56247f2cade0 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 -- 172.21.15.3:0/1360159493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2cade0 msgr2=0x56247f2ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2cade0 0x56247f2ca790 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f62b4033ff0 tx=0x7f62b4005fd0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 -- 172.21.15.3:0/1360159493 shutdown_connections 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2cade0 0x56247f2ca790 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2c2260 0x56247f2ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1360159493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 -- 172.21.15.3:0/1360159493 >> 172.21.15.3:0/1360159493 conn(0x56247f2bbf80 msgr2=0x56247f2bc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 -- 172.21.15.3:0/1360159493 shutdown_connections 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.082+0000 7f62c61d0740 1 -- 172.21.15.3:0/1360159493 wait complete. 2024-03-21T08:34:25.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 -- start start 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 Processor -- start 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1b3670 con 0x56247f2c2260 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1b4970 con 0x56247f2c18b0 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.083+0000 7f62c61d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1b3cb0 con 0x56247f2cade0 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62c61d0740 10 client.0 _ll_register_callbacks cb 0x7f62b0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62c0a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62c0a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57890/0 (socket says 172.21.15.3:57890) 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62c0a00640 1 -- 172.21.15.3:0/1311722124 learned_addr learned my addr 172.21.15.3:0/1311722124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59720/0 (socket says 172.21.15.3:59720) 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62c1400640 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62bbe00640 1 -- 172.21.15.3:0/1311722124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 msgr2=0x56247f2d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62bbe00640 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62bbe00640 1 -- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 msgr2=0x56247f2d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62bbe00640 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62bbe00640 1 -- 172.21.15.3:0/1311722124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56247f335200 con 0x56247f2c2260 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.084+0000 7f62c0a00640 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62bbe00640 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2d0c20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f62a80cd2c0 tx=0x7f62a809a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3063088629fdbdaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62a2000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62a2000640 1 -- 172.21.15.3:0/1311722124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8010630 con 0x56247f2c2260 2024-03-21T08:34:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62a2000640 1 -- 172.21.15.3:0/1311722124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f62a809b3b0 con 0x56247f2c2260 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62b404b020 con 0x56247f2c2260 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62a2000640 1 -- 172.21.15.3:0/1311722124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a809c660 con 0x56247f2c2260 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62c61d0740 10 client.16419 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.085+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56247f3364e0 con 0x56247f2c2260 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62a2000640 1 -- 172.21.15.3:0/1311722124 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62b404b020 con 0x56247f2c2260 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62a2000640 1 -- 172.21.15.3:0/1311722124 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f62a80b7070 con 0x56247f2c2260 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62c61d0740 10 client.16419 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62c61d0740 10 client.16419 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.k 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62c61d0740 20 client.16419 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f336bf0 0x56247f339010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62c61d0740 4 client.16419 mds_command: new command op to 34409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 --> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56247f22cb50 con 0x56247f336bf0 2024-03-21T08:34:25.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.086+0000 7f62c0a00640 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f336bf0 0x56247f339010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.087+0000 7f62c0a00640 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f336bf0 0x56247f339010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ee0f7246b80758f server_cookie=fa27b345a03a76f8 in_seq=0 out_seq=0 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.087+0000 7f62a2000640 10 client.16419 ms_handle_connect on v2:172.21.15.84:6832/4091779255 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.087+0000 7f62a2000640 1 -- 172.21.15.3:0/1311722124 <== mds.2 v2:172.21.15.84:6832/4091779255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+659 (crc 0 0 0) 0x56247f22cb50 con 0x56247f336bf0 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.087+0000 7f62a2000640 10 client.16419 handle_command_reply: tid=0 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.087+0000 7f62c61d0740 1 client.16419 shutdown 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 msgr2=0x56247f2d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2d0c20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f62a80cd2c0 tx=0x7f62a809a470 comp rx=0 tx=0).stop 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 shutdown_connections 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:6832/4091779255,v1:172.21.15.84:6833/4091779255] conn(0x56247f336bf0 0x56247f339010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56247f2cade0 0x56247f2d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56247f2c2260 0x56247f2d0c20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 --2- 172.21.15.3:0/1311722124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56247f2c18b0 0x56247f2d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 >> 172.21.15.3:0/1311722124 conn(0x56247f2bbf80 msgr2=0x56247f2bdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62a2000640 0 client.16419 ms_handle_reset on v2:172.21.15.84:6832/4091779255 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 shutdown_connections 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 1 -- 172.21.15.3:0/1311722124 wait complete. 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.088+0000 7f62c61d0740 20 client.16419 trim_cache size 0 max 16384 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:34:25.094 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1217033934, 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628498, 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/af_unix.c" 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2450867245, 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628497, 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/Makefile" 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-03-21T08:34:25.095 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3957951097, 2024-03-21T08:34:25.096 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628499, 2024-03-21T08:34:25.096 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/aio-dio-regress" 2024-03-21T08:34:25.096 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:34:25.096 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-03-21T08:34:25.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f62a40a3090 msgr2=0x7f62a40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f62a40a3090 0x7f62a40a5550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56247f2bcff0 tx=0x7f62b0009260 comp rx=0 tx=0).stop 2024-03-21T08:34:25.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 msgr2=0x7f62bc19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc19e780 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9f0 tx=0x7f62a800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 shutdown_connections 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f62a40a3090 0x7f62a40a5550 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62bc10c980 0x7f62bc1a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62bc108440 0x7f62bc19e780 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 --2- 172.21.15.3:0/62960657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc107aa0 0x7f62bc19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 >> 172.21.15.3:0/62960657 conn(0x7f62bc0772a0 msgr2=0x7f62bc077c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 shutdown_connections 2024-03-21T08:34:25.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.089+0000 7f62c1e00640 1 -- 172.21.15.3:0/62960657 wait complete. 2024-03-21T08:34:25.107 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:24 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3877906023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:34:25.110 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:34:25.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T08:34:25.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:24 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3877906023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:34:25.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 -- 172.21.15.3:0/4206736721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0103660 msgr2=0x7f20c0103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 --2- 172.21.15.3:0/4206736721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0103660 0x7f20c0103ae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009930 tx=0x7f20ac02bc30 comp rx=0 tx=0).stop 2024-03-21T08:34:25.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 -- 172.21.15.3:0/4206736721 shutdown_connections 2024-03-21T08:34:25.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 --2- 172.21.15.3:0/4206736721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20c0104020 0x7f20c0110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 --2- 172.21.15.3:0/4206736721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0103660 0x7f20c0103ae0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 --2- 172.21.15.3:0/4206736721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0109670 0x7f20c0109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 -- 172.21.15.3:0/4206736721 >> 172.21.15.3:0/4206736721 conn(0x7f20c00ff150 msgr2=0x7f20c0101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 -- 172.21.15.3:0/4206736721 shutdown_connections 2024-03-21T08:34:25.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.341+0000 7f20c6000640 1 -- 172.21.15.3:0/4206736721 wait complete. 2024-03-21T08:34:25.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.342+0000 7f20c6000640 1 Processor -- start 2024-03-21T08:34:25.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c6000640 1 -- start start 2024-03-21T08:34:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c6000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0103660 0x7f20c0199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c6000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20c0104020 0x7f20c019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0109670 0x7f20c019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c6000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0113930 con 0x7f20c0109670 2024-03-21T08:34:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c6000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01137b0 con 0x7f20c0103660 2024-03-21T08:34:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c6000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0113ab0 con 0x7f20c0104020 2024-03-21T08:34:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0103660 0x7f20c0199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0109670 0x7f20c019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0103660 0x7f20c0199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57908/0 (socket says 172.21.15.3:57908) 2024-03-21T08:34:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.343+0000 7f20c4c00640 1 -- 172.21.15.3:0/2516089230 learned_addr learned my addr 172.21.15.3:0/2516089230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.344+0000 7f20bfe00640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20c0104020 0x7f20c019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.344+0000 7f20c4c00640 1 -- 172.21.15.3:0/2516089230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20c0104020 msgr2=0x7f20c019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.344+0000 7f20c4c00640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20c0104020 0x7f20c019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.344+0000 7f20c4c00640 1 -- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0109670 msgr2=0x7f20c019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.344+0000 7f20c4c00640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0109670 0x7f20c019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.344+0000 7f20c4c00640 1 -- 172.21.15.3:0/2516089230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c019eea0 con 0x7f20c0103660 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.344+0000 7f20c4c00640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0103660 0x7f20c0199eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f20b400e9f0 tx=0x7f20b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.345+0000 7f20bd600640 1 -- 172.21.15.3:0/2516089230 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b400cd80 con 0x7f20c0103660 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.345+0000 7f20bd600640 1 -- 172.21.15.3:0/2516089230 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20b40108e0 con 0x7f20c0103660 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.345+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c019f190 con 0x7f20c0103660 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.345+0000 7f20bd600640 1 -- 172.21.15.3:0/2516089230 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4018920 con 0x7f20c0103660 2024-03-21T08:34:25.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.345+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c01a6a20 con 0x7f20c0103660 2024-03-21T08:34:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.349+0000 7f20ca412740 1 Processor -- start 2024-03-21T08:34:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.349+0000 7f20ca412740 1 -- start start 2024-03-21T08:34:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20ca412740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b7399238b0 0x55b739923c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20bd600640 1 -- 172.21.15.3:0/2516089230 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f20b4010a90 con 0x7f20c0103660 2024-03-21T08:34:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20ca412740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b73992c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20ca412740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b73992cde0 0x55b73992f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20ca412740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7398872c0 con 0x55b739924260 2024-03-21T08:34:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20ca412740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73982fc90 con 0x55b7399238b0 2024-03-21T08:34:25.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20ca412740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739883360 con 0x55b73992cde0 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20c5600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b7399238b0 0x55b739923c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20c5600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b7399238b0 0x55b739923c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57912/0 (socket says 172.21.15.3:57912) 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.350+0000 7f20c5600640 1 -- 172.21.15.3:0/3659145721 learned_addr learned my addr 172.21.15.3:0/3659145721 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20c4c00640 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b73992c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20bfe00640 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b73992cde0 0x55b73992f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20c5600640 1 -- 172.21.15.3:0/3659145721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b73992cde0 msgr2=0x55b73992f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20c5600640 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b73992cde0 0x55b73992f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20c5600640 1 -- 172.21.15.3:0/3659145721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 msgr2=0x55b73992c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20c5600640 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b73992c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20c5600640 1 -- 172.21.15.3:0/3659145721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b739854080 con 0x55b7399238b0 2024-03-21T08:34:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.351+0000 7f20c5600640 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b7399238b0 0x55b739923c90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f20b8010370 tx=0x7f20b8010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b1056a78f131d1b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20b1c00640 1 -- 172.21.15.3:0/3659145721 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8016070 con 0x55b7399238b0 2024-03-21T08:34:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20bd600640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f20a80a3110 0x7f20a80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20b1c00640 1 -- 172.21.15.3:0/3659145721 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20b8004d20 con 0x55b7399238b0 2024-03-21T08:34:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20b1c00640 1 -- 172.21.15.3:0/3659145721 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8005390 con 0x55b7399238b0 2024-03-21T08:34:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20bd600640 1 -- 172.21.15.3:0/2516089230 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f20b40cff00 con 0x7f20c0103660 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20c4c00640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f20a80a3110 0x7f20a80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20ca412740 1 -- 172.21.15.3:0/3659145721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b7399238b0 msgr2=0x55b739923c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.352+0000 7f20ca412740 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b7399238b0 0x55b739923c90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f20b8010370 tx=0x7f20b8010830 comp rx=0 tx=0).stop 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 -- 172.21.15.3:0/3659145721 shutdown_connections 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b73992cde0 0x55b73992f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b73992c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 --2- 172.21.15.3:0/3659145721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b7399238b0 0x55b739923c90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 -- 172.21.15.3:0/3659145721 >> 172.21.15.3:0/3659145721 conn(0x55b73991df80 msgr2=0x55b73991e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 -- 172.21.15.3:0/3659145721 shutdown_connections 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 -- 172.21.15.3:0/3659145721 wait complete. 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20c4c00640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f20a80a3110 0x7f20a80a55d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b739923380 tx=0x7f20b4000bc0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.353+0000 7f20ca412740 1 -- start start 2024-03-21T08:34:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 1 Processor -- start 2024-03-21T08:34:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b7399238b0 0x55b73992c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b739929a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739929fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73992fd50 con 0x55b739924260 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7398430d0 con 0x55b73992cde0 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73992fed0 con 0x55b7399238b0 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.354+0000 7f20ca412740 10 client.0 _ll_register_callbacks cb 0x55b739847e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b739929a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b739929a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59758/0 (socket says 172.21.15.3:59758) 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 -- 172.21.15.3:0/302155950 learned_addr learned my addr 172.21.15.3:0/302155950 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20c5600640 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739929fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20c5600640 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b7399238b0 0x55b73992c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 -- 172.21.15.3:0/302155950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b7399238b0 msgr2=0x55b73992c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b7399238b0 0x55b73992c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 -- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 msgr2=0x55b739929fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739929fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.355+0000 7f20bfe00640 1 -- 172.21.15.3:0/302155950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b73959b3f0 con 0x55b739924260 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20bfe00640 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b739929a40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009340 tx=0x7f20ac02ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c1eac64333870e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20a6000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20a6000640 1 -- 172.21.15.3:0/302155950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac005d50 con 0x55b739924260 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20a6000640 1 -- 172.21.15.3:0/302155950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f20ac005450 con 0x55b739924260 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20b8004d20 con 0x55b739924260 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20a6000640 1 -- 172.21.15.3:0/302155950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ac02cc80 con 0x55b739924260 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20ca412740 10 client.16431 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.356+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b7396657d0 con 0x55b739924260 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20a6000640 1 -- 172.21.15.3:0/302155950 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20b8004d20 con 0x55b739924260 2024-03-21T08:34:25.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20a6000640 1 -- 172.21.15.3:0/302155950 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f20ac0035f0 con 0x55b739924260 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20ca412740 10 client.16431 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20ca412740 10 client.16431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20ca412740 20 client.16431 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20ca412740 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b73993cc20 0x55b73993f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20ca412740 4 client.16431 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.357+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b739894800 con 0x55b73993cc20 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.358+0000 7f20c5600640 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b73993cc20 0x55b73993f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.358+0000 7f20c5600640 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b73993cc20 0x55b73993f010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=690658ff9b622c54 server_cookie=b451ebf78b70a232 in_seq=0 out_seq=0 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.358+0000 7f20a6000640 10 client.16431 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20a6000640 1 -- 172.21.15.3:0/302155950 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b739894800 con 0x55b73993cc20 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20a6000640 10 client.16431 handle_command_reply: tid=0 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 client.16431 shutdown 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 msgr2=0x55b739929a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b739929a40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009340 tx=0x7f20ac02ca90 comp rx=0 tx=0).stop 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 shutdown_connections 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739929fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b739924260 0x55b739929a40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b73993cc20 0x55b73993f010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 --2- 172.21.15.3:0/302155950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b7399238b0 0x55b73992c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20a6000640 0 client.16431 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.359+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 >> 172.21.15.3:0/302155950 conn(0x55b73991df80 msgr2=0x55b73991fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.360+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 shutdown_connections 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.360+0000 7f20ca412740 1 -- 172.21.15.3:0/302155950 wait complete. 2024-03-21T08:34:25.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.360+0000 7f20ca412740 20 client.16431 trim_cache size 0 max 16384 2024-03-21T08:34:25.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 Processor -- start 2024-03-21T08:34:25.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 -- start start 2024-03-21T08:34:25.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b73991d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b73992c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b73992c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01b3220 con 0x55b7399238b0 2024-03-21T08:34:25.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01b0cf0 con 0x55b73992cde0 2024-03-21T08:34:25.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.376+0000 7f20ca412740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01aed90 con 0x55b739924260 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b73991d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b73991d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59766/0 (socket says 172.21.15.3:59766) 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 -- 172.21.15.3:0/3095996487 learned_addr learned my addr 172.21.15.3:0/3095996487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b73992c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20bfe00640 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b73992c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 -- 172.21.15.3:0/3095996487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 msgr2=0x55b73992c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b73992c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 -- 172.21.15.3:0/3095996487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 msgr2=0x55b73992c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b73992c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.377+0000 7f20c5600640 1 -- 172.21.15.3:0/3095996487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b739927ff0 con 0x55b7399238b0 2024-03-21T08:34:25.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.378+0000 7f20c5600640 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b73991d3a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f20b8015040 tx=0x7f20b8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1111018e22b67034 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.378+0000 7f20b1200640 1 -- 172.21.15.3:0/3095996487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8016040 con 0x55b7399238b0 2024-03-21T08:34:25.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.378+0000 7f20b1200640 1 -- 172.21.15.3:0/3095996487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20b802ac50 con 0x55b7399238b0 2024-03-21T08:34:25.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.378+0000 7f20b1200640 1 -- 172.21.15.3:0/3095996487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b80042a0 con 0x55b7399238b0 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 -- 172.21.15.3:0/3095996487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 msgr2=0x55b73991d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b73991d3a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f20b8015040 tx=0x7f20b8010830 comp rx=0 tx=0).stop 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 -- 172.21.15.3:0/3095996487 shutdown_connections 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b73992c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b73992c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 --2- 172.21.15.3:0/3095996487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b73991d3a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 -- 172.21.15.3:0/3095996487 >> 172.21.15.3:0/3095996487 conn(0x55b73991df80 msgr2=0x55b73991fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 -- 172.21.15.3:0/3095996487 shutdown_connections 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 -- 172.21.15.3:0/3095996487 wait complete. 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.379+0000 7f20ca412740 1 -- start start 2024-03-21T08:34:25.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 1 Processor -- start 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b739931580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b739931ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739932000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01af390 con 0x55b7399238b0 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01b0690 con 0x55b73992cde0 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01af9d0 con 0x55b739924260 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20ca412740 10 client.0 _ll_register_callbacks cb 0x7f20b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.380+0000 7f20c5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b739931580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b739931580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59778/0 (socket says 172.21.15.3:59778) 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 -- 172.21.15.3:0/1112982326 learned_addr learned my addr 172.21.15.3:0/1112982326 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739932000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20bfe00640 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b739931ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 -- 172.21.15.3:0/1112982326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 msgr2=0x55b739931ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b739931ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 -- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 msgr2=0x55b739932000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739932000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 -- 172.21.15.3:0/1112982326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7399a0cd0 con 0x55b7399238b0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20c5600640 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b739931580 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009c20 tx=0x7f20b802d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b032a19ad7ff80 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.381+0000 7f20a6000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20a6000640 1 -- 172.21.15.3:0/1112982326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8016040 con 0x55b7399238b0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20b802ac50 con 0x55b7399238b0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20a6000640 1 -- 172.21.15.3:0/1112982326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f20b802a7a0 con 0x55b7399238b0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20a6000640 1 -- 172.21.15.3:0/1112982326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20b803c050 con 0x55b7399238b0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20ca412740 10 client.16437 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b7399a1f90 con 0x55b7399238b0 2024-03-21T08:34:25.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20a6000640 1 -- 172.21.15.3:0/1112982326 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20b802ac50 con 0x55b7399238b0 2024-03-21T08:34:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.382+0000 7f20a6000640 1 -- 172.21.15.3:0/1112982326 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f20b800f040 con 0x55b7399238b0 2024-03-21T08:34:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.383+0000 7f20ca412740 10 client.16437 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.383+0000 7f20ca412740 10 client.16437 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.e 2024-03-21T08:34:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.383+0000 7f20ca412740 20 client.16437 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.383+0000 7f20ca412740 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b7399a23c0 0x55b7399a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.383+0000 7f20ca412740 4 client.16437 mds_command: new command op to 34439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.383+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 --> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b73988eb50 con 0x55b7399a23c0 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.383+0000 7f20bfe00640 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b7399a23c0 0x55b7399a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20a6000640 10 client.16437 ms_handle_connect on v2:172.21.15.84:6834/2576104068 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20bfe00640 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b7399a23c0 0x55b7399a47b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=837f94cb4c07361b server_cookie=ad06b3c6f1f18b32 in_seq=0 out_seq=0 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20a6000640 1 -- 172.21.15.3:0/1112982326 <== mds.1 v2:172.21.15.84:6834/2576104068 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b73988eb50 con 0x55b7399a23c0 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20a6000640 10 client.16437 handle_command_reply: tid=0 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 client.16437 shutdown 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 msgr2=0x55b739931580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b739931580 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009c20 tx=0x7f20b802d040 comp rx=0 tx=0).stop 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 shutdown_connections 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b73992cde0 0x55b739932000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.84:6834/2576104068,v1:172.21.15.84:6835/2576104068] conn(0x55b7399a23c0 0x55b7399a47b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b739924260 0x55b739931ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 --2- 172.21.15.3:0/1112982326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7399238b0 0x55b739931580 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20a6000640 0 client.16437 ms_handle_reset on v2:172.21.15.84:6834/2576104068 2024-03-21T08:34:25.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 >> 172.21.15.3:0/1112982326 conn(0x55b73991df80 msgr2=0x55b73991f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 shutdown_connections 2024-03-21T08:34:25.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.384+0000 7f20ca412740 1 -- 172.21.15.3:0/1112982326 wait complete. 2024-03-21T08:34:25.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.385+0000 7f20ca412740 20 client.16437 trim_cache size 0 max 16384 2024-03-21T08:34:25.390 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.385+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f20a80a3110 msgr2=0x7f20a80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.385+0000 7f20c6000640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f20a80a3110 0x7f20a80a55d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b739923380 tx=0x7f20b4000bc0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.385+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0103660 msgr2=0x7f20c0199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.385+0000 7f20c6000640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0103660 0x7f20c0199eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f20b400e9f0 tx=0x7f20b400eeb0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 shutdown_connections 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20c0109670 0x7f20c019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f20a80a3110 0x7f20a80a55d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20c0104020 0x7f20c019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 --2- 172.21.15.3:0/2516089230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20c0103660 0x7f20c0199eb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 >> 172.21.15.3:0/2516089230 conn(0x7f20c00ff150 msgr2=0x7f20c00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 shutdown_connections 2024-03-21T08:34:25.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.386+0000 7f20c6000640 1 -- 172.21.15.3:0/2516089230 wait complete. 2024-03-21T08:34:25.405 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T08:34:25.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.634+0000 7fb20f200640 1 -- 172.21.15.3:0/2879731151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 msgr2=0x7fb2101100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.634+0000 7fb20f200640 1 --2- 172.21.15.3:0/2879731151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb2101100d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb208009ca0 tx=0x7fb20802c180 comp rx=0 tx=0).stop 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.635+0000 7fb20f200640 1 -- 172.21.15.3:0/2879731151 shutdown_connections 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.635+0000 7fb20f200640 1 --2- 172.21.15.3:0/2879731151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb2101100d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.635+0000 7fb20f200640 1 --2- 172.21.15.3:0/2879731151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2100743f0 0x7fb21010d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.635+0000 7fb20f200640 1 --2- 172.21.15.3:0/2879731151 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb210073ad0 0x7fb210073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.635+0000 7fb20f200640 1 -- 172.21.15.3:0/2879731151 >> 172.21.15.3:0/2879731151 conn(0x7fb2100fcf30 msgr2=0x7fb2100ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.635+0000 7fb20f200640 1 -- 172.21.15.3:0/2879731151 shutdown_connections 2024-03-21T08:34:25.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.635+0000 7fb20f200640 1 -- 172.21.15.3:0/2879731151 wait complete. 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 Processor -- start 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 -- start start 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb210073ad0 0x7fb210199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2100743f0 0x7fb21019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb21019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101138d0 con 0x7fb21010dc80 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210113750 con 0x7fb210073ad0 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.636+0000 7fb20f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210113a50 con 0x7fb2100743f0 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb21019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb210073ad0 0x7fb210199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb21019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59796/0 (socket says 172.21.15.3:59796) 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 -- 172.21.15.3:0/2355239229 learned_addr learned my addr 172.21.15.3:0/2355239229 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20d400640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2100743f0 0x7fb21019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 -- 172.21.15.3:0/2355239229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2100743f0 msgr2=0x7fb21019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2100743f0 0x7fb21019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 -- 172.21.15.3:0/2355239229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb210073ad0 msgr2=0x7fb210199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb210073ad0 0x7fb210199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 -- 172.21.15.3:0/2355239229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21019ee40 con 0x7fb21010dc80 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20de00640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb210073ad0 0x7fb210199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.637+0000 7fb20e800640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb21019e710 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb20802c790 tx=0x7fb208035ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.638+0000 7fb206a00640 1 -- 172.21.15.3:0/2355239229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208039040 con 0x7fb21010dc80 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.638+0000 7fb206a00640 1 -- 172.21.15.3:0/2355239229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb208044070 con 0x7fb21010dc80 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.638+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21019f0d0 con 0x7fb21010dc80 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.638+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2101a69c0 con 0x7fb21010dc80 2024-03-21T08:34:25.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.638+0000 7fb206a00640 1 -- 172.21.15.3:0/2355239229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208032850 con 0x7fb21010dc80 2024-03-21T08:34:25.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.642+0000 7fb217bc9740 1 Processor -- start 2024-03-21T08:34:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb206a00640 1 -- 172.21.15.3:0/2355239229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fb2080329f0 con 0x7fb21010dc80 2024-03-21T08:34:25.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb217bc9740 1 -- start start 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb217bc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27d5512c0 con 0x55b27d5ee260 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb217bc9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27d4f9c90 con 0x55b27d5f6de0 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.643+0000 7fb217bc9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27d54d360 con 0x55b27d5ed8b0 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59802/0 (socket says 172.21.15.3:59802) 2024-03-21T08:34:25.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 -- 172.21.15.3:0/3668128414 learned_addr learned my addr 172.21.15.3:0/3668128414 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20d400640 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20d400640 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 -- 172.21.15.3:0/3668128414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 msgr2=0x55b27d5edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 -- 172.21.15.3:0/3668128414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 msgr2=0x55b27d5f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb20de00640 1 -- 172.21.15.3:0/3668128414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b27d51e080 con 0x55b27d5ee260 2024-03-21T08:34:25.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.644+0000 7fb206a00640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb1f40a3090 0x7fb1f40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.645+0000 7fb206a00640 1 -- 172.21.15.3:0/2355239229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fb2080f1a50 con 0x7fb21010dc80 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.645+0000 7fb20de00640 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f6810 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb200011540 tx=0x7fb200011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f62a5a20d4770ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.645+0000 7fb20d400640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb1f40a3090 0x7fb1f40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.645+0000 7fb1fb400640 1 -- 172.21.15.3:0/3668128414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000f640 con 0x55b27d5ee260 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.645+0000 7fb1fb400640 1 -- 172.21.15.3:0/3668128414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb20000fc90 con 0x55b27d5ee260 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.645+0000 7fb1fb400640 1 -- 172.21.15.3:0/3668128414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200019cc0 con 0x55b27d5ee260 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb20d400640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb1f40a3090 0x7fb1f40a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2100ff630 tx=0x7fb1fc0046c0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 -- 172.21.15.3:0/3668128414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 msgr2=0x55b27d5f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f6810 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb200011540 tx=0x7fb200011a00 comp rx=0 tx=0).stop 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 -- 172.21.15.3:0/3668128414 shutdown_connections 2024-03-21T08:34:25.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f6810 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 --2- 172.21.15.3:0/3668128414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 -- 172.21.15.3:0/3668128414 >> 172.21.15.3:0/3668128414 conn(0x55b27d5e7f80 msgr2=0x55b27d5e8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 -- 172.21.15.3:0/3668128414 shutdown_connections 2024-03-21T08:34:25.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.646+0000 7fb217bc9740 1 -- 172.21.15.3:0/3668128414 wait complete. 2024-03-21T08:34:25.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.647+0000 7fb217bc9740 1 -- start start 2024-03-21T08:34:25.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.647+0000 7fb217bc9740 1 Processor -- start 2024-03-21T08:34:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb217bc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27d5f9d50 con 0x55b27d5ee260 2024-03-21T08:34:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb217bc9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27d50d0d0 con 0x55b27d5f6de0 2024-03-21T08:34:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb217bc9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27d5f9ed0 con 0x55b27d5ed8b0 2024-03-21T08:34:25.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb217bc9740 10 client.0 _ll_register_callbacks cb 0x55b27d511e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59816/0 (socket says 172.21.15.3:59816) 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20d400640 1 -- 172.21.15.3:0/2294336437 learned_addr learned my addr 172.21.15.3:0/2294336437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20e800640 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20de00640 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20d400640 1 -- 172.21.15.3:0/2294336437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 msgr2=0x55b27d5f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20d400640 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20d400640 1 -- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 msgr2=0x55b27d5f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.648+0000 7fb20d400640 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.649+0000 7fb20d400640 1 -- 172.21.15.3:0/2294336437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b27d2653f0 con 0x55b27d5ee260 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.649+0000 7fb20d400640 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f3a80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc011b50 tx=0x7fb1fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3aed4866781169 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.649+0000 7fb1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.649+0000 7fb1efe00640 1 -- 172.21.15.3:0/2294336437 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc019070 con 0x55b27d5ee260 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.649+0000 7fb1efe00640 1 -- 172.21.15.3:0/2294336437 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fb1fc010e20 con 0x55b27d5ee260 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.649+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb20000fc90 con 0x55b27d5ee260 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb1efe00640 1 -- 172.21.15.3:0/2294336437 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1fc031050 con 0x55b27d5ee260 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 10 client.16449 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b27d32f7d0 con 0x55b27d5ee260 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb1efe00640 1 -- 172.21.15.3:0/2294336437 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb20000fc90 con 0x55b27d5ee260 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb1efe00640 1 -- 172.21.15.3:0/2294336437 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fb1fc020490 con 0x55b27d5ee260 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 10 client.16449 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 20 client.16449 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d606d40 0x55b27d609150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 4 client.16449 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.650+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b27d55e800 con 0x55b27d606d40 2024-03-21T08:34:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.651+0000 7fb20de00640 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d606d40 0x55b27d609150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.651+0000 7fb20de00640 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d606d40 0x55b27d609150 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2934a7d1db4d735 server_cookie=b3b39ad75a29ce50 in_seq=0 out_seq=0 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.652+0000 7fb1efe00640 10 client.16449 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.652+0000 7fb1efe00640 1 -- 172.21.15.3:0/2294336437 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b27d55e800 con 0x55b27d606d40 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.652+0000 7fb1efe00640 10 client.16449 handle_command_reply: tid=0 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 client.16449 shutdown 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 msgr2=0x55b27d5f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f3a80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc011b50 tx=0x7fb1fc004730 comp rx=0 tx=0).stop 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 shutdown_connections 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5f3a80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d606d40 0x55b27d609150 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2294336437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 >> 172.21.15.3:0/2294336437 conn(0x55b27d5e7f80 msgr2=0x55b27d5e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb1efe00640 0 client.16449 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:25.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 shutdown_connections 2024-03-21T08:34:25.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 1 -- 172.21.15.3:0/2294336437 wait complete. 2024-03-21T08:34:25.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.653+0000 7fb217bc9740 20 client.16449 trim_cache size 0 max 16384 2024-03-21T08:34:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.669+0000 7fb217bc9740 1 Processor -- start 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.669+0000 7fb217bc9740 1 -- start start 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5ee260 0x55b27d5f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5f6de0 0x55b27d5f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb217bc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101b31f0 con 0x55b27d5f6de0 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb217bc9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101b0cc0 con 0x55b27d5ee260 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb217bc9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101aed60 con 0x55b27d5ed8b0 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5f6de0 0x55b27d5f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5ee260 0x55b27d5f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5f6de0 0x55b27d5f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59818/0 (socket says 172.21.15.3:59818) 2024-03-21T08:34:25.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 -- 172.21.15.3:0/267823522 learned_addr learned my addr 172.21.15.3:0/267823522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20de00640 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 -- 172.21.15.3:0/267823522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 msgr2=0x55b27d5eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 -- 172.21.15.3:0/267823522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5ee260 msgr2=0x55b27d5f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5ee260 0x55b27d5f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20e800640 1 -- 172.21.15.3:0/267823522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b27d5f9c30 con 0x55b27d5f6de0 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.670+0000 7fb20d400640 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5ee260 0x55b27d5f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.671+0000 7fb20e800640 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5f6de0 0x55b27d5f66f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb208004af0 tx=0x7fb208035f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7964114a4bc7e3cc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.671+0000 7fb1faa00640 1 -- 172.21.15.3:0/267823522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208003c70 con 0x55b27d5f6de0 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.671+0000 7fb1faa00640 1 -- 172.21.15.3:0/267823522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb2080bdad0 con 0x55b27d5f6de0 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.671+0000 7fb1faa00640 1 -- 172.21.15.3:0/267823522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080ef130 con 0x55b27d5f6de0 2024-03-21T08:34:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 -- 172.21.15.3:0/267823522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5f6de0 msgr2=0x55b27d5f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5f6de0 0x55b27d5f66f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb208004af0 tx=0x7fb208035f90 comp rx=0 tx=0).stop 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 -- 172.21.15.3:0/267823522 shutdown_connections 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5f6de0 0x55b27d5f66f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5ee260 0x55b27d5f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 --2- 172.21.15.3:0/267823522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 -- 172.21.15.3:0/267823522 >> 172.21.15.3:0/267823522 conn(0x55b27d5e7f80 msgr2=0x55b27d5e7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 -- 172.21.15.3:0/267823522 shutdown_connections 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.672+0000 7fb217bc9740 1 -- 172.21.15.3:0/267823522 wait complete. 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 -- start start 2024-03-21T08:34:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 Processor -- start 2024-03-21T08:34:25.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101af360 con 0x55b27d5ee260 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101b0660 con 0x55b27d5f6de0 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101af9a0 con 0x55b27d5ed8b0 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.673+0000 7fb217bc9740 10 client.0 _ll_register_callbacks cb 0x7fb1fc00cd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59820/0 (socket says 172.21.15.3:59820) 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 -- 172.21.15.3:0/2884030413 learned_addr learned my addr 172.21.15.3:0/2884030413 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20de00640 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 -- 172.21.15.3:0/2884030413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 msgr2=0x55b27d5ff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5ff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 -- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 msgr2=0x55b27d5fdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5fdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20d400640 1 -- 172.21.15.3:0/2884030413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b27d66ae10 con 0x55b27d5ee260 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.674+0000 7fb20e800640 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb20d400640 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5fd450 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c3f0 tx=0x7fb1fc020d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffc5941009fcae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb1efe00640 1 -- 172.21.15.3:0/2884030413 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc03b070 con 0x55b27d5ee260 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb1efe00640 1 -- 172.21.15.3:0/2884030413 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fb1fc020600 con 0x55b27d5ee260 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2080bdad0 con 0x55b27d5ee260 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb1efe00640 1 -- 172.21.15.3:0/2884030413 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1fc019040 con 0x55b27d5ee260 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb217bc9740 10 client.16455 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.675+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b27d66c040 con 0x55b27d5ee260 2024-03-21T08:34:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb1efe00640 1 -- 172.21.15.3:0/2884030413 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb2080bdad0 con 0x55b27d5ee260 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb1efe00640 1 -- 172.21.15.3:0/2884030413 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fb1fc028cd0 con 0x55b27d5ee260 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb217bc9740 10 client.16455 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb217bc9740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb217bc9740 20 client.16455 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d66c760 0x55b27d66eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb217bc9740 4 client.16455 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b27d558b50 con 0x55b27d66c760 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.676+0000 7fb20de00640 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d66c760 0x55b27d66eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb20de00640 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d66c760 0x55b27d66eb80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7add2d65322bd61d server_cookie=89facfa23ac243e7 in_seq=0 out_seq=0 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb1efe00640 10 client.16455 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb1efe00640 1 -- 172.21.15.3:0/2884030413 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b27d558b50 con 0x55b27d66c760 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb1efe00640 10 client.16455 handle_command_reply: tid=0 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 client.16455 shutdown 2024-03-21T08:34:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 msgr2=0x55b27d5fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5fd450 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c3f0 tx=0x7fb1fc020d00 comp rx=0 tx=0).stop 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 shutdown_connections 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b27d5f6de0 0x55b27d5fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b27d5ee260 0x55b27d5fd450 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b27d66c760 0x55b27d66eb80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 --2- 172.21.15.3:0/2884030413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b27d5ed8b0 0x55b27d5ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 >> 172.21.15.3:0/2884030413 conn(0x55b27d5e7f80 msgr2=0x55b27d5eef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.677+0000 7fb1efe00640 0 client.16455 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 shutdown_connections 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb217bc9740 1 -- 172.21.15.3:0/2884030413 wait complete. 2024-03-21T08:34:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb217bc9740 20 client.16455 trim_cache size 0 max 16384 2024-03-21T08:34:25.685 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-03-21T08:34:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb1f40a3090 msgr2=0x7fb1f40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb20f200640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb1f40a3090 0x7fb1f40a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2100ff630 tx=0x7fb1fc0046c0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 msgr2=0x7fb21019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb20f200640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb21019e710 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb20802c790 tx=0x7fb208035ea0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.678+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 shutdown_connections 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.679+0000 7fb20f200640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21010dc80 0x7fb21019e710 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.679+0000 7fb20f200640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb1f40a3090 0x7fb1f40a5550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.679+0000 7fb20f200640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2100743f0 0x7fb21019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.679+0000 7fb20f200640 1 --2- 172.21.15.3:0/2355239229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb210073ad0 0x7fb210199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.679+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 >> 172.21.15.3:0/2355239229 conn(0x7fb2100fcf30 msgr2=0x7fb2100fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.679+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 shutdown_connections 2024-03-21T08:34:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:34:25.679+0000 7fb20f200640 1 -- 172.21.15.3:0/2355239229 wait complete. 2024-03-21T08:34:25.699 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 1217033934, 'ino': 1099511628498, 'path': '/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/af_unix.c'}, {'damage_type': 'backtrace', 'id': 2450867245, 'ino': 1099511628497, 'path': '/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/Makefile'}, {'damage_type': 'backtrace', 'id': 3957951097, 'ino': 1099511628499, 'path': '/volumes/qa/sv_0/c7ef95a4-879e-4568-8fc1-2823dfb91393/client.0/tmp/xfstests-dev/src/aio-dio-regress'}] 2024-03-21T08:34:25.699 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-21T08:34:26.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:25 smithi084 ceph-mon[35477]: pgmap v80: 129 pgs: 129 active+clean; 31 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 523 KiB/s wr, 95 op/s 2024-03-21T08:34:26.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:25 smithi171 ceph-mon[35388]: pgmap v80: 129 pgs: 129 active+clean; 31 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 523 KiB/s wr, 95 op/s 2024-03-21T08:34:26.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:25 smithi003 ceph-mon[31675]: pgmap v80: 129 pgs: 129 active+clean; 31 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 523 KiB/s wr, 95 op/s 2024-03-21T08:34:28.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:27 smithi084 ceph-mon[35477]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 676 KiB/s wr, 220 op/s 2024-03-21T08:34:28.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:27 smithi171 ceph-mon[35388]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 676 KiB/s wr, 220 op/s 2024-03-21T08:34:28.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:27 smithi003 ceph-mon[31675]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 676 KiB/s wr, 220 op/s 2024-03-21T08:34:30.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:29 smithi084 ceph-mon[35477]: pgmap v82: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 703 KiB/s wr, 254 op/s 2024-03-21T08:34:30.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:29 smithi171 ceph-mon[35388]: pgmap v82: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 703 KiB/s wr, 254 op/s 2024-03-21T08:34:30.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:29 smithi003 ceph-mon[31675]: pgmap v82: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 703 KiB/s wr, 254 op/s 2024-03-21T08:34:32.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:31 smithi084 ceph-mon[35477]: pgmap v83: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 557 KiB/s wr, 236 op/s 2024-03-21T08:34:32.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:31 smithi171 ceph-mon[35388]: pgmap v83: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 557 KiB/s wr, 236 op/s 2024-03-21T08:34:32.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:31 smithi003 ceph-mon[31675]: pgmap v83: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 557 KiB/s wr, 236 op/s 2024-03-21T08:34:34.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:33 smithi084 ceph-mon[35477]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 867 KiB/s wr, 296 op/s 2024-03-21T08:34:34.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:33 smithi171 ceph-mon[35388]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 867 KiB/s wr, 296 op/s 2024-03-21T08:34:34.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:33 smithi003 ceph-mon[31675]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 867 KiB/s wr, 296 op/s 2024-03-21T08:34:36.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:35 smithi084 ceph-mon[35477]: pgmap v85: 129 pgs: 129 active+clean; 32 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 553 KiB/s wr, 245 op/s 2024-03-21T08:34:36.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:35 smithi171 ceph-mon[35388]: pgmap v85: 129 pgs: 129 active+clean; 32 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 553 KiB/s wr, 245 op/s 2024-03-21T08:34:36.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:35 smithi003 ceph-mon[31675]: pgmap v85: 129 pgs: 129 active+clean; 32 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 553 KiB/s wr, 245 op/s 2024-03-21T08:34:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 -- 172.21.15.3:0/3667931054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10109890 msgr2=0x7f5e10109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 --2- 172.21.15.3:0/3667931054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10109890 0x7f5e10109c70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009cd0 tx=0x7f5e0402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 -- 172.21.15.3:0/3667931054 shutdown_connections 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 --2- 172.21.15.3:0/3667931054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e10104240 0x7f5e10110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 --2- 172.21.15.3:0/3667931054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e10103880 0x7f5e10103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 --2- 172.21.15.3:0/3667931054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10109890 0x7f5e10109c70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 -- 172.21.15.3:0/3667931054 >> 172.21.15.3:0/3667931054 conn(0x7f5e100ff390 msgr2=0x7f5e101017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 -- 172.21.15.3:0/3667931054 shutdown_connections 2024-03-21T08:34:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.998+0000 7f5e17600640 1 -- 172.21.15.3:0/3667931054 wait complete. 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.999+0000 7f5e17600640 1 Processor -- start 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:36.999+0000 7f5e17600640 1 -- start start 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e17600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e10103880 0x7f5e1019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e17600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10104240 0x7f5e1019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e17600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e10109890 0x7f5e1019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e17600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e10113b50 con 0x7f5e10104240 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e17600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101139d0 con 0x7f5e10109890 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e17600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e10113cd0 con 0x7f5e10103880 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10104240 0x7f5e1019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10104240 0x7f5e1019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36158/0 (socket says 172.21.15.3:36158) 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 -- 172.21.15.3:0/4141455334 learned_addr learned my addr 172.21.15.3:0/4141455334 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e15600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e10109890 0x7f5e1019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e14c00640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e10103880 0x7f5e1019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 -- 172.21.15.3:0/4141455334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e10103880 msgr2=0x7f5e1019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e10103880 0x7f5e1019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 -- 172.21.15.3:0/4141455334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e10109890 msgr2=0x7f5e1019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e10109890 0x7f5e1019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.000+0000 7f5e0fe00640 1 -- 172.21.15.3:0/4141455334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1019f0b0 con 0x7f5e10104240 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.001+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10104240 0x7f5e1019a600 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9f0 tx=0x7f5dfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.001+0000 7f5e0d600640 1 -- 172.21.15.3:0/4141455334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00cd80 con 0x7f5e10104240 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.001+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1019f3a0 con 0x7f5e10104240 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.001+0000 7f5e0d600640 1 -- 172.21.15.3:0/4141455334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc0108e0 con 0x7f5e10104240 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.001+0000 7f5e0d600640 1 -- 172.21.15.3:0/4141455334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc018920 con 0x7f5e10104240 2024-03-21T08:34:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.001+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e101a6c30 con 0x7f5e10104240 2024-03-21T08:34:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.004+0000 7f5e1a3ee740 1 Processor -- start 2024-03-21T08:34:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e1a3ee740 1 -- start start 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92ea8f0 0x5589f92f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e0d600640 1 -- 172.21.15.3:0/4141455334 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f5dfc010a90 con 0x7f5e10104240 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589f92ac3b0 con 0x5589f92e9fd0 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589f924e830 con 0x5589f92ea8f0 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589f91f75d0 con 0x5589f92f36c0 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e15600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e14c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92ea8f0 0x5589f92f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e14c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92ea8f0 0x5589f92f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:54774/0 (socket says 172.21.15.3:54774) 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e15600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ea3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36172/0 (socket says 172.21.15.3:36172) 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.005+0000 7f5e14c00640 1 -- 172.21.15.3:0/2543920203 learned_addr learned my addr 172.21.15.3:0/2543920203 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.006+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.006+0000 7f5e15600640 1 -- 172.21.15.3:0/2543920203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 msgr2=0x5589f92f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.006+0000 7f5e15600640 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.006+0000 7f5e15600640 1 -- 172.21.15.3:0/2543920203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92ea8f0 msgr2=0x5589f92f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.006+0000 7f5e15600640 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92ea8f0 0x5589f92f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.006+0000 7f5e15600640 1 -- 172.21.15.3:0/2543920203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589f920f2c0 con 0x5589f92e9fd0 2024-03-21T08:34:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.007+0000 7f5e15600640 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ea3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800aa30 tx=0x7f5e0800aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a88195e706e8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.007+0000 7f5e01c00640 1 -- 172.21.15.3:0/2543920203 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0801d070 con 0x5589f92e9fd0 2024-03-21T08:34:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.007+0000 7f5e01c00640 1 -- 172.21.15.3:0/2543920203 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e080188f0 con 0x5589f92e9fd0 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.007+0000 7f5e01c00640 1 -- 172.21.15.3:0/2543920203 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08010910 con 0x5589f92e9fd0 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e0d600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5df80a3080 0x7f5df80a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2543920203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 msgr2=0x5589f92ea3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ea3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800aa30 tx=0x7f5e0800aef0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2543920203 shutdown_connections 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92ea8f0 0x5589f92f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2543920203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ea3b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2543920203 >> 172.21.15.3:0/2543920203 conn(0x5589f92e48f0 msgr2=0x5589f92e4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e0d600640 1 -- 172.21.15.3:0/4141455334 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5dfc0cfff0 con 0x7f5e10104240 2024-03-21T08:34:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2543920203 shutdown_connections 2024-03-21T08:34:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e14c00640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5df80a3080 0x7f5df80a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.008+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2543920203 wait complete. 2024-03-21T08:34:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.009+0000 7f5e1a3ee740 1 -- start start 2024-03-21T08:34:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.009+0000 7f5e1a3ee740 1 Processor -- start 2024-03-21T08:34:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.009+0000 7f5e14c00640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5df80a3080 0x7f5df80a5540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009430 tx=0x7f5e04005d80 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92f0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589f920aa20 con 0x5589f92e9fd0 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589f924a8d0 con 0x5589f92f36c0 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589f92f62f0 con 0x5589f92ea8f0 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e1a3ee740 10 client.0 _ll_register_callbacks cb 0x5589f92471c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e14c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92f0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e14c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92f0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36188/0 (socket says 172.21.15.3:36188) 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e14c00640 1 -- 172.21.15.3:0/2871254528 learned_addr learned my addr 172.21.15.3:0/2871254528 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.010+0000 7f5e15600640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e15600640 1 -- 172.21.15.3:0/2871254528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 msgr2=0x5589f92f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e15600640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e15600640 1 -- 172.21.15.3:0/2871254528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 msgr2=0x5589f92f0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e15600640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92f0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e15600640 1 -- 172.21.15.3:0/2871254528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589f8f63210 con 0x5589f92f36c0 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e14c00640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92f0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5e15600640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92f11d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800aa30 tx=0x7f5e0800ff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bdb90b1c883fc37 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.011+0000 7f5df6000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.012+0000 7f5df6000640 1 -- 172.21.15.3:0/2871254528 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0801d040 con 0x5589f92f36c0 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.012+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e080188f0 con 0x5589f92f36c0 2024-03-21T08:34:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.012+0000 7f5df6000640 1 -- 172.21.15.3:0/2871254528 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5e08018d60 con 0x5589f92f36c0 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.012+0000 7f5df6000640 1 -- 172.21.15.3:0/2871254528 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e08030050 con 0x5589f92f36c0 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.012+0000 7f5e1a3ee740 10 client.36206 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.012+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5589f902f620 con 0x5589f92f36c0 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.012+0000 7f5df6000640 1 -- 172.21.15.3:0/2871254528 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e080188f0 con 0x5589f92f36c0 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5df6000640 1 -- 172.21.15.3:0/2871254528 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f5e08004a20 con 0x5589f92f36c0 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5e1a3ee740 10 client.36206 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5e1a3ee740 10 client.36206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5e1a3ee740 20 client.36206 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f9303700 0x5589f9305af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5e1a3ee740 4 client.36206 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589f925bd70 con 0x5589f9303700 2024-03-21T08:34:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.013+0000 7f5e14c00640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f9303700 0x5589f9305af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.014+0000 7f5e14c00640 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f9303700 0x5589f9305af0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0572820f2bcec8d server_cookie=d9fd16a905c477f1 in_seq=0 out_seq=0 2024-03-21T08:34:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.014+0000 7f5df6000640 10 client.36206 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.015+0000 7f5df6000640 1 -- 172.21.15.3:0/2871254528 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5589f925bd70 con 0x5589f9303700 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.015+0000 7f5df6000640 10 client.36206 handle_command_reply: tid=0 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.015+0000 7f5e1a3ee740 1 client.36206 shutdown 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.015+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 msgr2=0x5589f92f11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.015+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92f11d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800aa30 tx=0x7f5e0800ff60 comp rx=0 tx=0).stop 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 shutdown_connections 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92f11d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f9303700 0x5589f9305af0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2871254528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92f0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5df6000640 0 client.36206 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 >> 172.21.15.3:0/2871254528 conn(0x5589f92e48f0 msgr2=0x5589f92e6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 shutdown_connections 2024-03-21T08:34:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2871254528 wait complete. 2024-03-21T08:34:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.016+0000 7f5e1a3ee740 20 client.36206 trim_cache size 0 max 16384 2024-03-21T08:34:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.033+0000 7f5e1a3ee740 1 Processor -- start 2024-03-21T08:34:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.033+0000 7f5e1a3ee740 1 -- start start 2024-03-21T08:34:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.033+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92e9fd0 0x5589f92f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.033+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92ea8f0 0x5589f92ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.033+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.033+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101b3340 con 0x5589f92ea8f0 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.033+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101b0e10 con 0x5589f92e9fd0 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101aeeb0 con 0x5589f92f36c0 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92ea8f0 0x5589f92ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e14c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92e9fd0 0x5589f92f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92ea8f0 0x5589f92ef8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36204/0 (socket says 172.21.15.3:36204) 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 -- 172.21.15.3:0/2576456654 learned_addr learned my addr 172.21.15.3:0/2576456654 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e15600640 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 -- 172.21.15.3:0/2576456654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 msgr2=0x5589f92e37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92e37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 -- 172.21.15.3:0/2576456654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92e9fd0 msgr2=0x5589f92f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92e9fd0 0x5589f92f2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e0fe00640 1 -- 172.21.15.3:0/2576456654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589f92e4120 con 0x5589f92ea8f0 2024-03-21T08:34:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.034+0000 7f5e14c00640 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92e9fd0 0x5589f92f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.035+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92ea8f0 0x5589f92ef8f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc09bef0 tx=0x7f5dfc09b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800e9dbc7ef35cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.035+0000 7f5e01200640 1 -- 172.21.15.3:0/2576456654 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09bf60 con 0x5589f92ea8f0 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.035+0000 7f5e01200640 1 -- 172.21.15.3:0/2576456654 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc0a00b0 con 0x5589f92ea8f0 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.035+0000 7f5e01200640 1 -- 172.21.15.3:0/2576456654 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09b3b0 con 0x5589f92ea8f0 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2576456654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92ea8f0 msgr2=0x5589f92ef8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92ea8f0 0x5589f92ef8f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc09bef0 tx=0x7f5dfc09b8c0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2576456654 shutdown_connections 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92f36c0 0x5589f92e37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92ea8f0 0x5589f92ef8f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/2576456654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92e9fd0 0x5589f92f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2576456654 >> 172.21.15.3:0/2576456654 conn(0x5589f92e48f0 msgr2=0x5589f92e30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2576456654 shutdown_connections 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.036+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/2576456654 wait complete. 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 -- start start 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 Processor -- start 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101af4b0 con 0x5589f92e9fd0 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101b07b0 con 0x5589f92f36c0 2024-03-21T08:34:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101afaf0 con 0x5589f92ea8f0 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.037+0000 7f5e1a3ee740 10 client.0 _ll_register_callbacks cb 0x7f5e0801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ed0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36212/0 (socket says 172.21.15.3:36212) 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 -- 172.21.15.3:0/4032909852 learned_addr learned my addr 172.21.15.3:0/4032909852 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e15600640 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 -- 172.21.15.3:0/4032909852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 msgr2=0x5589f92ed620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92ed620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 -- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 msgr2=0x5589f92edb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92edb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 -- 172.21.15.3:0/4032909852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589f935dfe0 con 0x5589f92e9fd0 2024-03-21T08:34:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5e14c00640 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ed0e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5e0402c630 tx=0x7f5e040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3d186a612af253 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.038+0000 7f5df6000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5df6000640 1 -- 172.21.15.3:0/4032909852 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04039040 con 0x5589f92e9fd0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5df6000640 1 -- 172.21.15.3:0/4032909852 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5e04052860 con 0x5589f92e9fd0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dfc0a00b0 con 0x5589f92e9fd0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5df6000640 1 -- 172.21.15.3:0/4032909852 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e04052a60 con 0x5589f92e9fd0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5e1a3ee740 10 client.16473 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5589f935f2f0 con 0x5589f92e9fd0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5df6000640 1 -- 172.21.15.3:0/4032909852 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc0a00b0 con 0x5589f92e9fd0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.039+0000 7f5df6000640 1 -- 172.21.15.3:0/4032909852 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f5e040045d0 con 0x5589f92e9fd0 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e1a3ee740 10 client.16473 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e1a3ee740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e1a3ee740 20 client.16473 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f935f970 0x5589f9361d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e1a3ee740 4 client.16473 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:34:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5589f92560c0 con 0x5589f935f970 2024-03-21T08:34:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f935f970 0x5589f9361d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5e0fe00640 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f935f970 0x5589f9361d90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f029dc46d7ee26a server_cookie=2dddc0b4bbb6c405 in_seq=0 out_seq=0 2024-03-21T08:34:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.040+0000 7f5df6000640 10 client.16473 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5df6000640 1 -- 172.21.15.3:0/4032909852 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5589f92560c0 con 0x5589f935f970 2024-03-21T08:34:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5df6000640 10 client.16473 handle_command_reply: tid=0 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 client.16473 shutdown 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 msgr2=0x5589f92ed0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ed0e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5e0402c630 tx=0x7f5e040095c0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 shutdown_connections 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589f92f36c0 0x5589f92edb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5589f935f970 0x5589f9361d90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589f92ea8f0 0x5589f92ed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 --2- 172.21.15.3:0/4032909852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5589f92e9fd0 0x5589f92ed0e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 >> 172.21.15.3:0/4032909852 conn(0x5589f92e48f0 msgr2=0x5589f92e3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5df6000640 0 client.16473 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 shutdown_connections 2024-03-21T08:34:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 1 -- 172.21.15.3:0/4032909852 wait complete. 2024-03-21T08:34:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.130+0000 7f5e1a3ee740 20 client.16473 trim_cache size 0 max 16384 2024-03-21T08:34:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:34:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:34:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:34:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5df80a3080 msgr2=0x7f5df80a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5df80a3080 0x7f5df80a5540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009430 tx=0x7f5e04005d80 comp rx=0 tx=0).stop 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10104240 msgr2=0x7f5e1019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10104240 0x7f5e1019a600 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9f0 tx=0x7f5dfc00eeb0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 shutdown_connections 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e10109890 0x7f5e1019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e10104240 0x7f5e1019a600 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5df80a3080 0x7f5df80a5540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 --2- 172.21.15.3:0/4141455334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e10103880 0x7f5e1019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 >> 172.21.15.3:0/4141455334 conn(0x7f5e100ff390 msgr2=0x7f5e100ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 shutdown_connections 2024-03-21T08:34:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:37.132+0000 7f5e17600640 1 -- 172.21.15.3:0/4141455334 wait complete. 2024-03-21T08:34:38.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:37 smithi084 ceph-mon[35477]: pgmap v86: 129 pgs: 129 active+clean; 34 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 769 KiB/s wr, 278 op/s 2024-03-21T08:34:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:37 smithi171 ceph-mon[35388]: pgmap v86: 129 pgs: 129 active+clean; 34 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 769 KiB/s wr, 278 op/s 2024-03-21T08:34:38.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:37 smithi003 ceph-mon[31675]: pgmap v86: 129 pgs: 129 active+clean; 34 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 769 KiB/s wr, 278 op/s 2024-03-21T08:34:40.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:39 smithi084 ceph-mon[35477]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 829 KiB/s wr, 168 op/s 2024-03-21T08:34:40.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:39 smithi171 ceph-mon[35388]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 829 KiB/s wr, 168 op/s 2024-03-21T08:34:40.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:39 smithi003 ceph-mon[31675]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 829 KiB/s wr, 168 op/s 2024-03-21T08:34:42.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:41 smithi084 ceph-mon[35477]: pgmap v88: 129 pgs: 129 active+clean; 36 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 739 KiB/s wr, 108 op/s 2024-03-21T08:34:42.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:41 smithi171 ceph-mon[35388]: pgmap v88: 129 pgs: 129 active+clean; 36 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 739 KiB/s wr, 108 op/s 2024-03-21T08:34:42.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:41 smithi003 ceph-mon[31675]: pgmap v88: 129 pgs: 129 active+clean; 36 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 739 KiB/s wr, 108 op/s 2024-03-21T08:34:42.966 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 2% (119/4277) Updating files: 3% (129/4277) Updating files: 4% (172/4277) Updating files: 5% (214/4277) Updating files: 6% (257/4277) Updating files: 6% (275/4277) Updating files: 7% (300/4277) Updating files: 8% (343/4277) Updating files: 9% (385/4277) Updating files: 9% (388/4277) Updating files: 10% (428/4277) Updating files: 11% (471/4277) Updating files: 11% (489/4277) Updating files: 11% (496/4277) Updating files: 12% (514/4277) Updating files: 13% (557/4277) Updating files: 14% (599/4277) Updating files: 15% (642/4277) Updating files: 16% (685/4277) Updating files: 16% (707/4277) Updating files: 17% (728/4277) Updating files: 18% (770/4277) Updating files: 19% (813/4277) Updating files: 19% (817/4277) Updating files: 20% (856/4277) Updating files: 21% (899/4277) Updating files: 21% (928/4277) Updating files: 22% (941/4277) Updating files: 23% (984/4277) Updating files: 24% (1027/4277) Updating files: 25% (1070/4277) Updating files: 26% (1113/4277) Updating files: 26% (1122/4277) Updating files: 27% (1155/4277) Updating files: 28% (1198/4277) Updating files: 29% (1241/4277) Updating files: 30% (1284/4277) Updating files: 30% (1307/4277) Updating files: 31% (1326/4277) Updating files: 32% (1369/4277) Updating files: 33% (1412/4277) Updating files: 33% (1422/4277) Updating files: 34% (1455/4277) Updating files: 35% (1497/4277) Updating files: 36% (1540/4277) Updating files: 37% (1583/4277) Updating files: 37% (1598/4277) Updating files: 38% (1626/4277) Updating files: 39% (1669/4277) Updating files: 39% (1706/4277) Updating files: 40% (1711/4277) Updating files: 41% (1754/4277) Updating files: 41% (1794/4277) Updating files: 42% (1797/4277) Updating files: 43% (1840/4277) Updating files: 44% (1882/4277) Updating files: 44% (1906/4277) Updating files: 45% (1925/4277) Updating files: 46% (1968/4277) Updating files: 47% (2011/4277) Updating files: 47% (2012/4277) Updating files: 48% (2053/4277) Updating files: 49% (2096/4277) Updating files: 49% (2115/4277) Updating files: 50% (2139/4277) Updating files: 51% (2182/4277) Updating files: 51% (2214/4277) Updating files: 52% (2225/4277) Updating files: 52% (2261/4277) Updating files: 53% (2267/4277) Updating files: 53% (2284/4277) Updating files: 54% (2310/4277) Updating files: 54% (2328/4277) Updating files: 55% (2353/4277) Updating files: 55% (2367/4277) Updating files: 56% (2396/4277) Updating files: 56% (2403/4277) Updating files: 56% (2428/4277) Updating files: 57% (2438/4277) Updating files: 57% (2466/4277) Updating files: 58% (2481/4277) Updating files: 58% (2501/4277) Updating files: 59% (2524/4277) Updating files: 59% (2539/4277) Updating files: 60% (2567/4277) Updating files: 60% (2570/4277) Updating files: 60% (2604/4277) Updating files: 61% (2609/4277) Updating files: 62% (2652/4277) Updating files: 62% (2690/4277) Updating files: 63% (2695/4277) Updating files: 64% (2738/4277) Updating files: 64% (2773/4277) Updating files: 64% (2778/4277) Updating files: 65% (2781/4277) Updating files: 66% (2823/4277) Updating files: 67% (2866/4277) Updating files: 67% (2881/4277) Updating files: 68% (2909/4277) Updating files: 69% (2952/4277) Updating files: 70% (2994/4277) Updating files: 70% (2997/4277) Updating files: 71% (3037/4277) Updating files: 72% (3080/4277) Updating files: 72% (3102/4277) Updating files: 73% (3123/4277) Updating files: 74% (3165/4277) Updating files: 75% (3208/4277) Updating files: 75% (3212/4277) Updating files: 76% (3251/4277) Updating files: 77% (3294/4277) Updating files: 77% (3315/4277) Updating files: 78% (3337/4277) Updating files: 79% (3379/4277) Updating files: 80% (3422/4277) Updating files: 80% (3424/4277) Updating files: 81% (3465/4277) Updating files: 82% (3508/4277) Updating files: 82% (3511/4277) Updating files: 83% (3550/4277) Updating files: 83% (3590/4277) Updating files: 84% (3593/4277) Updating files: 84% (3627/4277) Updating files: 85% (3636/4277) Updating files: 85% (3677/4277) Updating files: 86% (3679/4277) Updating files: 87% (3721/4277) Updating files: 87% (3722/4277) Updating files: 87% (3762/4277) Updating files: 88% (3764/4277) Updating files: 88% (3802/4277) Updating files: 89% (3807/4277) Updating files: 89% (3840/4277) Updating files: 90% (3850/4277) Updating files: 90% (3878/4277) Updating files: 91% (3893/4277) Updating files: 91% (3913/4277) Updating files: 92% (3935/4277) Updating files: 92% (3947/4277) Updating files: 92% (3977/4277) Updating files: 93% (3978/4277) Updating files: 93% (4009/4277) Updating files: 94% (4021/4277) Updating files: 94% (4040/4277) Updating files: 95% (4064/4277) Updating files: 95% (4069/4277) Updating files: 95% (4099/4277) Updating files: 96% (4106/4277) Updating files: 96% (4128/4277) Updating files: 97% (4149/4277) Updating files: 97% (4157/4277) Updating files: 97% (4186/4277) Updating files: 98% (4192/4277) Updating files: 98% (4215/4277) Updating files: 99% (4235/4277) Updating files: 99% (4246/4277) Updating files: 100% (4277/4277) Updating files: 100% (4277/4277), done. 2024-03-21T08:34:43.048 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd xfstests-dev 2024-03-21T08:34:43.053 INFO:tasks.workunit.client.0.smithi003.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-03-21T08:34:44.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:43 smithi084 ceph-mon[35477]: pgmap v89: 129 pgs: 129 active+clean; 37 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-03-21T08:34:44.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:43 smithi171 ceph-mon[35388]: pgmap v89: 129 pgs: 129 active+clean; 37 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-03-21T08:34:44.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:43 smithi003 ceph-mon[31675]: pgmap v89: 129 pgs: 129 active+clean; 37 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-03-21T08:34:46.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:45 smithi084 ceph-mon[35477]: pgmap v90: 129 pgs: 129 active+clean; 37 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 835 KiB/s wr, 121 op/s 2024-03-21T08:34:46.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:45 smithi171 ceph-mon[35388]: pgmap v90: 129 pgs: 129 active+clean; 37 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 835 KiB/s wr, 121 op/s 2024-03-21T08:34:46.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:45 smithi003 ceph-mon[31675]: pgmap v90: 129 pgs: 129 active+clean; 37 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 835 KiB/s wr, 121 op/s 2024-03-21T08:34:48.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:47 smithi084 ceph-mon[35477]: pgmap v91: 129 pgs: 129 active+clean; 38 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 970 KiB/s wr, 132 op/s 2024-03-21T08:34:48.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:47 smithi171 ceph-mon[35388]: pgmap v91: 129 pgs: 129 active+clean; 38 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 970 KiB/s wr, 132 op/s 2024-03-21T08:34:48.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:47 smithi003 ceph-mon[31675]: pgmap v91: 129 pgs: 129 active+clean; 38 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 970 KiB/s wr, 132 op/s 2024-03-21T08:34:50.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:49 smithi084 ceph-mon[35477]: pgmap v92: 129 pgs: 129 active+clean; 39 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 809 KiB/s wr, 108 op/s 2024-03-21T08:34:50.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:49 smithi171 ceph-mon[35388]: pgmap v92: 129 pgs: 129 active+clean; 39 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 809 KiB/s wr, 108 op/s 2024-03-21T08:34:50.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:49 smithi003 ceph-mon[31675]: pgmap v92: 129 pgs: 129 active+clean; 39 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 809 KiB/s wr, 108 op/s 2024-03-21T08:34:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 -- 172.21.15.3:0/525931808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef0102f70 msgr2=0x7fbef010b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 --2- 172.21.15.3:0/525931808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef0102f70 0x7fbef010b450 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009900 tx=0x7fbee002bc30 comp rx=0 tx=0).stop 2024-03-21T08:34:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 -- 172.21.15.3:0/525931808 shutdown_connections 2024-03-21T08:34:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 --2- 172.21.15.3:0/525931808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbef010bac0 0x7fbef010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 --2- 172.21.15.3:0/525931808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef0102f70 0x7fbef010b450 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 --2- 172.21.15.3:0/525931808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbef0102650 0x7fbef0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 -- 172.21.15.3:0/525931808 >> 172.21.15.3:0/525931808 conn(0x7fbef00fbe90 msgr2=0x7fbef00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 -- 172.21.15.3:0/525931808 shutdown_connections 2024-03-21T08:34:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.290+0000 7fbef7200640 1 -- 172.21.15.3:0/525931808 wait complete. 2024-03-21T08:34:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.291+0000 7fbef7200640 1 Processor -- start 2024-03-21T08:34:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.291+0000 7fbef7200640 1 -- start start 2024-03-21T08:34:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbef0102650 0x7fbef0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbef0102f70 0x7fbef019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef010bac0 0x7fbef019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0111710 con 0x7fbef010bac0 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0111590 con 0x7fbef0102f70 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0111890 con 0x7fbef0102650 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef010bac0 0x7fbef019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef010bac0 0x7fbef019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50058/0 (socket says 172.21.15.3:50058) 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef6800640 1 -- 172.21.15.3:0/4133471831 learned_addr learned my addr 172.21.15.3:0/4133471831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef5400640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbef0102f70 0x7fbef019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.292+0000 7fbef5e00640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbef0102650 0x7fbef0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef6800640 1 -- 172.21.15.3:0/4133471831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbef0102650 msgr2=0x7fbef0199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef6800640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbef0102650 0x7fbef0199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef6800640 1 -- 172.21.15.3:0/4133471831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbef0102f70 msgr2=0x7fbef019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef6800640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbef0102f70 0x7fbef019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef6800640 1 -- 172.21.15.3:0/4133471831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef019eef0 con 0x7fbef010bac0 2024-03-21T08:34:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef5400640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbef0102f70 0x7fbef019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef6800640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef010bac0 0x7fbef019e7c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00d750 tx=0x7fbeec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbee6a00640 1 -- 172.21.15.3:0/4133471831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec017070 con 0x7fbef010bac0 2024-03-21T08:34:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbee6a00640 1 -- 172.21.15.3:0/4133471831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbeec0054d0 con 0x7fbef010bac0 2024-03-21T08:34:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef019f1e0 con 0x7fbef010bac0 2024-03-21T08:34:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbee6a00640 1 -- 172.21.15.3:0/4133471831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004c60 con 0x7fbef010bac0 2024-03-21T08:34:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.293+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef01a6a70 con 0x7fbef010bac0 2024-03-21T08:34:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.298+0000 7fbefb599740 1 Processor -- start 2024-03-21T08:34:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.298+0000 7fbee6a00640 1 -- 172.21.15.3:0/4133471831 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fbeec004e00 con 0x7fbef010bac0 2024-03-21T08:34:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.298+0000 7fbefb599740 1 -- start start 2024-03-21T08:34:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbefb599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x5611464765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbefb599740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x56114647f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbefb599740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146481cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbefb599740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611464383b0 con 0x56114647f8d0 2024-03-21T08:34:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbefb599740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611463da830 con 0x561146476b00 2024-03-21T08:34:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbefb599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611463835d0 con 0x5611464761e0 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbef5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146481cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbef5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146481cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50060/0 (socket says 172.21.15.3:50060) 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbef5400640 1 -- 172.21.15.3:0/313678101 learned_addr learned my addr 172.21.15.3:0/313678101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbef5e00640 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x56114647f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.299+0000 7fbef5400640 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x5611464765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.300+0000 7fbef5e00640 1 -- 172.21.15.3:0/313678101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 msgr2=0x5611464765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.300+0000 7fbef5e00640 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x5611464765c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.300+0000 7fbef5e00640 1 -- 172.21.15.3:0/313678101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 msgr2=0x561146481cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.300+0000 7fbef5e00640 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146481cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.300+0000 7fbef5e00640 1 -- 172.21.15.3:0/313678101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114639b2c0 con 0x561146476b00 2024-03-21T08:34:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.300+0000 7fbef5400640 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146481cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.300+0000 7fbef5e00640 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x56114647f390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009f50 tx=0x7fbee8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90da098bf1ebc698 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.301+0000 7fbee6a00640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbedc0a3070 0x7fbedc0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.301+0000 7fbee6a00640 1 -- 172.21.15.3:0/4133471831 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fbeec0cf590 con 0x7fbef010bac0 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.301+0000 7fbedb000640 1 -- 172.21.15.3:0/313678101 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003b70 con 0x561146476b00 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.301+0000 7fbef5400640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbedc0a3070 0x7fbedc0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.301+0000 7fbedb000640 1 -- 172.21.15.3:0/313678101 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbee8015070 con 0x561146476b00 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.301+0000 7fbedb000640 1 -- 172.21.15.3:0/313678101 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011720 con 0x561146476b00 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 -- 172.21.15.3:0/313678101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 msgr2=0x56114647f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x56114647f390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009f50 tx=0x7fbee8010860 comp rx=0 tx=0).stop 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbef5400640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbedc0a3070 0x7fbedc0a5530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x561146471f20 tx=0x7fbee0005b50 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 -- 172.21.15.3:0/313678101 shutdown_connections 2024-03-21T08:34:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146481cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x56114647f390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 --2- 172.21.15.3:0/313678101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x5611464765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 -- 172.21.15.3:0/313678101 >> 172.21.15.3:0/313678101 conn(0x561146470b00 msgr2=0x561146470ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 -- 172.21.15.3:0/313678101 shutdown_connections 2024-03-21T08:34:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.302+0000 7fbefb599740 1 -- 172.21.15.3:0/313678101 wait complete. 2024-03-21T08:34:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 -- start start 2024-03-21T08:34:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 Processor -- start 2024-03-21T08:34:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611464761e0 0x56114647c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561146476b00 0x56114647ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114647d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561146396a20 con 0x5611464761e0 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611463d68d0 con 0x56114647f8d0 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.303+0000 7fbefb599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561146482500 con 0x561146476b00 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbefb599740 10 client.0 _ll_register_callbacks cb 0x5611463d31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611464761e0 0x56114647c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611464761e0 0x56114647c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50076/0 (socket says 172.21.15.3:50076) 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 -- 172.21.15.3:0/129752106 learned_addr learned my addr 172.21.15.3:0/129752106 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef6800640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114647d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5400640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561146476b00 0x56114647ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 -- 172.21.15.3:0/129752106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561146476b00 msgr2=0x56114647ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561146476b00 0x56114647ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 -- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 msgr2=0x56114647d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114647d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5e00640 1 -- 172.21.15.3:0/129752106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611460ef210 con 0x5611464761e0 2024-03-21T08:34:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.304+0000 7fbef5400640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561146476b00 0x56114647ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbef5e00640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611464761e0 0x56114647c920 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbee80048f0 tx=0x7fbee8004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1901efaf386dd6a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbecf400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbecf400640 1 -- 172.21.15.3:0/129752106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004c50 con 0x5611464761e0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbecf400640 1 -- 172.21.15.3:0/129752106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fbee8004df0 con 0x5611464761e0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbee8015070 con 0x5611464761e0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbecf400640 1 -- 172.21.15.3:0/129752106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbee8004050 con 0x5611464761e0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbefb599740 10 client.16485 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.305+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5611461bb620 con 0x5611464761e0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbecf400640 1 -- 172.21.15.3:0/129752106 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbee8015070 con 0x5611464761e0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbecf400640 1 -- 172.21.15.3:0/129752106 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fbee8011500 con 0x5611464761e0 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbefb599740 10 client.16485 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbefb599740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbefb599740 20 client.16485 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbefb599740 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56114648f900 0x561146491cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbefb599740 4 client.16485 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:34:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611463e7d70 con 0x56114648f900 2024-03-21T08:34:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.306+0000 7fbef5400640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56114648f900 0x561146491cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.307+0000 7fbef5400640 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56114648f900 0x561146491cf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1511f32446da5c4 server_cookie=92d8ed56742a40ad in_seq=0 out_seq=0 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.307+0000 7fbecf400640 10 client.16485 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.308+0000 7fbecf400640 1 -- 172.21.15.3:0/129752106 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5611463e7d70 con 0x56114648f900 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.308+0000 7fbecf400640 10 client.16485 handle_command_reply: tid=0 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.308+0000 7fbefb599740 1 client.16485 shutdown 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.308+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611464761e0 msgr2=0x56114647c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.308+0000 7fbefb599740 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611464761e0 0x56114647c920 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbee80048f0 tx=0x7fbee8004920 comp rx=0 tx=0).stop 2024-03-21T08:34:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 shutdown_connections 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114647d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56114648f900 0x561146491cf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561146476b00 0x56114647ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 --2- 172.21.15.3:0/129752106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611464761e0 0x56114647c920 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 >> 172.21.15.3:0/129752106 conn(0x561146470b00 msgr2=0x561146472850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbecf400640 0 client.16485 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 shutdown_connections 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 1 -- 172.21.15.3:0/129752106 wait complete. 2024-03-21T08:34:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.309+0000 7fbefb599740 20 client.16485 trim_cache size 0 max 16384 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 Processor -- start 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 -- start start 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561146476b00 0x56114648ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114648b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01b3210 con 0x561146476b00 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01b0ce0 con 0x56114647f8d0 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.326+0000 7fbefb599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01aed80 con 0x5611464761e0 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561146476b00 0x56114648ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561146476b00 0x56114648ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50088/0 (socket says 172.21.15.3:50088) 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 -- 172.21.15.3:0/700041383 learned_addr learned my addr 172.21.15.3:0/700041383 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef6800640 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114648b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5e00640 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 -- 172.21.15.3:0/700041383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 msgr2=0x56114647f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 -- 172.21.15.3:0/700041383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 msgr2=0x56114648b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114648b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.327+0000 7fbef5400640 1 -- 172.21.15.3:0/700041383 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561146481cc0 con 0x561146476b00 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.328+0000 7fbef5400640 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561146476b00 0x56114648ac40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009230 tx=0x7fbee0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1cbed3f5f86304 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.328+0000 7fbeda600640 1 -- 172.21.15.3:0/700041383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0059070 con 0x561146476b00 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.328+0000 7fbeda600640 1 -- 172.21.15.3:0/700041383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbee002cb50 con 0x561146476b00 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.328+0000 7fbeda600640 1 -- 172.21.15.3:0/700041383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0003850 con 0x561146476b00 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 -- 172.21.15.3:0/700041383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561146476b00 msgr2=0x56114648ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561146476b00 0x56114648ac40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009230 tx=0x7fbee0009260 comp rx=0 tx=0).stop 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 -- 172.21.15.3:0/700041383 shutdown_connections 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56114647f8d0 0x56114648b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561146476b00 0x56114648ac40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 --2- 172.21.15.3:0/700041383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 -- 172.21.15.3:0/700041383 >> 172.21.15.3:0/700041383 conn(0x561146470b00 msgr2=0x561146477c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 -- 172.21.15.3:0/700041383 shutdown_connections 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.329+0000 7fbefb599740 1 -- 172.21.15.3:0/700041383 wait complete. 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 -- start start 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 Processor -- start 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x561146478fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146479700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01af380 con 0x56114647f8d0 2024-03-21T08:34:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01b0680 con 0x561146476b00 2024-03-21T08:34:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01af9c0 con 0x5611464761e0 2024-03-21T08:34:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.330+0000 7fbefb599740 10 client.0 _ll_register_callbacks cb 0x7fbee80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:34:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x561146478fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146479700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x561146478fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:41242/0 (socket says 172.21.15.3:41242) 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 -- 172.21.15.3:0/2663654991 learned_addr learned my addr 172.21.15.3:0/2663654991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5e00640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 -- 172.21.15.3:0/2663654991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 msgr2=0x56114647a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 -- 172.21.15.3:0/2663654991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 msgr2=0x561146479700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:34:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146479700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5400640 1 -- 172.21.15.3:0/2663654991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611464ea090 con 0x561146476b00 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef6800640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146479700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.331+0000 7fbef5e00640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.332+0000 7fbef5400640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x561146478fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbee0036040 tx=0x7fbee0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4bdd1800118f9d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.332+0000 7fbecf400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.332+0000 7fbecf400640 1 -- 172.21.15.3:0/2663654991 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0059030 con 0x561146476b00 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.332+0000 7fbecf400640 1 -- 172.21.15.3:0/2663654991 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fbee0053040 con 0x561146476b00 2024-03-21T08:34:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.332+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbee002cb50 con 0x561146476b00 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.332+0000 7fbecf400640 1 -- 172.21.15.3:0/2663654991 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbee0005410 con 0x561146476b00 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.332+0000 7fbefb599740 10 client.36230 fetch_fsmap learned FSMap version 27 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5611464eb240 con 0x561146476b00 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbecf400640 1 -- 172.21.15.3:0/2663654991 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbee002cb50 con 0x561146476b00 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbecf400640 1 -- 172.21.15.3:0/2663654991 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fbee004b040 con 0x561146476b00 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbefb599740 10 client.36230 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbefb599740 10 client.36230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:34:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbefb599740 20 client.36230 populate_metadata read hostname 'smithi003' 2024-03-21T08:34:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbefb599740 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5611464eb940 0x5611464edd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:34:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbefb599740 4 client.36230 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:34:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.333+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611463e20c0 con 0x5611464eb940 2024-03-21T08:34:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.334+0000 7fbef5e00640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5611464eb940 0x5611464edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:34:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.334+0000 7fbef5e00640 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5611464eb940 0x5611464edd60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6e0ae4bd62b2fc server_cookie=13a513bda6d20314 in_seq=0 out_seq=0 2024-03-21T08:34:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.334+0000 7fbecf400640 10 client.36230 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbecf400640 1 -- 172.21.15.3:0/2663654991 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611463e20c0 con 0x5611464eb940 2024-03-21T08:34:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbecf400640 10 client.36230 handle_command_reply: tid=0 2024-03-21T08:34:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 client.36230 shutdown 2024-03-21T08:34:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 msgr2=0x561146478fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x561146478fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbee0036040 tx=0x7fbee0005dd0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 shutdown_connections 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56114647f8d0 0x561146479700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561146476b00 0x561146478fc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5611464eb940 0x5611464edd60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 --2- 172.21.15.3:0/2663654991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5611464761e0 0x56114647a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbecf400640 0 client.36230 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 >> 172.21.15.3:0/2663654991 conn(0x561146470b00 msgr2=0x561146471f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 shutdown_connections 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 1 -- 172.21.15.3:0/2663654991 wait complete. 2024-03-21T08:34:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.402+0000 7fbefb599740 20 client.36230 trim_cache size 0 max 16384 2024-03-21T08:34:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:34:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:34:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:34:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:34:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbedc0a3070 msgr2=0x7fbedc0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbedc0a3070 0x7fbedc0a5530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x561146471f20 tx=0x7fbee0005b50 comp rx=0 tx=0).stop 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef010bac0 msgr2=0x7fbef019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef010bac0 0x7fbef019e7c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00d750 tx=0x7fbeec00dc10 comp rx=0 tx=0).stop 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 shutdown_connections 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef010bac0 0x7fbef019e7c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbef0102f70 0x7fbef019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fbedc0a3070 0x7fbedc0a5530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 --2- 172.21.15.3:0/4133471831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbef0102650 0x7fbef0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 >> 172.21.15.3:0/4133471831 conn(0x7fbef00fbe90 msgr2=0x7fbef00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 shutdown_connections 2024-03-21T08:34:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:34:50.403+0000 7fbef7200640 1 -- 172.21.15.3:0/4133471831 wait complete. 2024-03-21T08:34:52.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:51 smithi084 ceph-mon[35477]: pgmap v93: 129 pgs: 129 active+clean; 39 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 596 KiB/s wr, 93 op/s 2024-03-21T08:34:52.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:51 smithi171 ceph-mon[35388]: pgmap v93: 129 pgs: 129 active+clean; 39 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 596 KiB/s wr, 93 op/s 2024-03-21T08:34:52.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:51 smithi003 ceph-mon[31675]: pgmap v93: 129 pgs: 129 active+clean; 39 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 596 KiB/s wr, 93 op/s 2024-03-21T08:34:54.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:53 smithi084 ceph-mon[35477]: pgmap v94: 129 pgs: 129 active+clean; 40 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 712 KiB/s wr, 135 op/s 2024-03-21T08:34:54.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:53 smithi171 ceph-mon[35388]: pgmap v94: 129 pgs: 129 active+clean; 40 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 712 KiB/s wr, 135 op/s 2024-03-21T08:34:54.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:53 smithi003 ceph-mon[31675]: pgmap v94: 129 pgs: 129 active+clean; 40 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 712 KiB/s wr, 135 op/s 2024-03-21T08:34:56.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:55 smithi084 ceph-mon[35477]: pgmap v95: 129 pgs: 129 active+clean; 40 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 306 KiB/s wr, 61 op/s 2024-03-21T08:34:56.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:55 smithi171 ceph-mon[35388]: pgmap v95: 129 pgs: 129 active+clean; 40 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 306 KiB/s wr, 61 op/s 2024-03-21T08:34:56.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:55 smithi003 ceph-mon[31675]: pgmap v95: 129 pgs: 129 active+clean; 40 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 306 KiB/s wr, 61 op/s 2024-03-21T08:34:58.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:57 smithi171 ceph-mon[35388]: pgmap v96: 129 pgs: 129 active+clean; 43 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 531 KiB/s wr, 136 op/s 2024-03-21T08:34:58.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:57 smithi003 ceph-mon[31675]: pgmap v96: 129 pgs: 129 active+clean; 43 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 531 KiB/s wr, 136 op/s 2024-03-21T08:34:58.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:57 smithi084 ceph-mon[35477]: pgmap v96: 129 pgs: 129 active+clean; 43 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 531 KiB/s wr, 136 op/s 2024-03-21T08:35:00.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:34:59 smithi003 ceph-mon[31675]: pgmap v97: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 396 KiB/s wr, 130 op/s 2024-03-21T08:35:00.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:34:59 smithi084 ceph-mon[35477]: pgmap v97: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 396 KiB/s wr, 130 op/s 2024-03-21T08:35:00.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:34:59 smithi171 ceph-mon[35388]: pgmap v97: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 396 KiB/s wr, 130 op/s 2024-03-21T08:35:02.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:01 smithi003 ceph-mon[31675]: pgmap v98: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 341 KiB/s wr, 121 op/s 2024-03-21T08:35:02.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:01 smithi084 ceph-mon[35477]: pgmap v98: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 341 KiB/s wr, 121 op/s 2024-03-21T08:35:02.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:01 smithi171 ceph-mon[35388]: pgmap v98: 129 pgs: 129 active+clean; 43 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 341 KiB/s wr, 121 op/s 2024-03-21T08:35:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 -- 172.21.15.3:0/2764680324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04074360 msgr2=0x7f5a04111f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 --2- 172.21.15.3:0/2764680324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04074360 0x7f5a04111f50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009900 tx=0x7f59f002bcc0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 -- 172.21.15.3:0/2764680324 shutdown_connections 2024-03-21T08:35:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 --2- 172.21.15.3:0/2764680324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a04112490 0x7f5a04114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 --2- 172.21.15.3:0/2764680324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04074360 0x7f5a04111f50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 --2- 172.21.15.3:0/2764680324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a04073a40 0x7f5a04073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 -- 172.21.15.3:0/2764680324 >> 172.21.15.3:0/2764680324 conn(0x7f5a040fcf50 msgr2=0x7f5a040ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 -- 172.21.15.3:0/2764680324 shutdown_connections 2024-03-21T08:35:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.553+0000 7f5a08c00640 1 -- 172.21.15.3:0/2764680324 wait complete. 2024-03-21T08:35:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.554+0000 7f5a08c00640 1 Processor -- start 2024-03-21T08:35:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.554+0000 7f5a08c00640 1 -- start start 2024-03-21T08:35:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.554+0000 7f5a08c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a04073a40 0x7f5a0419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a08c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a04074360 0x7f5a0419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a08c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04112490 0x7f5a041a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a08c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04117da0 con 0x7f5a04112490 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a08c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04117c20 con 0x7f5a04073a40 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a08c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04117f20 con 0x7f5a04074360 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04112490 0x7f5a041a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a03400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a04073a40 0x7f5a0419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04112490 0x7f5a041a2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41394/0 (socket says 172.21.15.3:41394) 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a03e00640 1 -- 172.21.15.3:0/867184709 learned_addr learned my addr 172.21.15.3:0/867184709 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a02a00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a04074360 0x7f5a0419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a03e00640 1 -- 172.21.15.3:0/867184709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a04074360 msgr2=0x7f5a0419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.555+0000 7f5a03e00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a04074360 0x7f5a0419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f5a03e00640 1 -- 172.21.15.3:0/867184709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a04073a40 msgr2=0x7f5a0419e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f5a03e00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a04073a40 0x7f5a0419e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f5a03e00640 1 -- 172.21.15.3:0/867184709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a041a3300 con 0x7f5a04112490 2024-03-21T08:35:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f5a03e00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04112490 0x7f5a041a2bd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d720 tx=0x7f59fc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f59fbe00640 1 -- 172.21.15.3:0/867184709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc017070 con 0x7f5a04112490 2024-03-21T08:35:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f59fbe00640 1 -- 172.21.15.3:0/867184709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59fc00de30 con 0x7f5a04112490 2024-03-21T08:35:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f59fbe00640 1 -- 172.21.15.3:0/867184709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc004cc0 con 0x7f5a04112490 2024-03-21T08:35:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.556+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a041a35f0 con 0x7f5a04112490 2024-03-21T08:35:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.557+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a041aae80 con 0x7f5a04112490 2024-03-21T08:35:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.561+0000 7f5a0d00c740 1 Processor -- start 2024-03-21T08:35:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.561+0000 7f59fbe00640 1 -- 172.21.15.3:0/867184709 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f59fc004e60 con 0x7f5a04112490 2024-03-21T08:35:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.561+0000 7f5a0d00c740 1 -- start start 2024-03-21T08:35:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c35281f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c352828390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3527e13b0 con 0x55c3528288d0 2024-03-21T08:35:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c352783830 con 0x55c35281fb00 2024-03-21T08:35:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35272c5d0 con 0x55c35281f1e0 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a03400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c352828390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a03400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c352828390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:48562/0 (socket says 172.21.15.3:48562) 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a03400640 1 -- 172.21.15.3:0/2801809972 learned_addr learned my addr 172.21.15.3:0/2801809972 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.562+0000 7f5a02a00640 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f5a02a00640 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c35281f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f5a03400640 1 -- 172.21.15.3:0/2801809972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 msgr2=0x55c35281f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f5a03400640 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c35281f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f5a03400640 1 -- 172.21.15.3:0/2801809972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 msgr2=0x55c35282acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f5a03400640 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f5a03400640 1 -- 172.21.15.3:0/2801809972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3527442c0 con 0x55c35281fb00 2024-03-21T08:35:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f5a03400640 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c352828390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f59f400a4b0 tx=0x7f59f40096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eacec2a5ad664e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.563+0000 7f59fbe00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f59e80a3070 0x7f59e80a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f59fbe00640 1 -- 172.21.15.3:0/867184709 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f59fc0cf520 con 0x7f5a04112490 2024-03-21T08:35:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f59efe00640 1 -- 172.21.15.3:0/2801809972 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4018020 con 0x55c35281fb00 2024-03-21T08:35:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f59efe00640 1 -- 172.21.15.3:0/2801809972 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59f4014cc0 con 0x55c35281fb00 2024-03-21T08:35:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f59efe00640 1 -- 172.21.15.3:0/2801809972 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4013d10 con 0x55c35281fb00 2024-03-21T08:35:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f5a02a00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f59e80a3070 0x7f59e80a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f5a02a00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f59e80a3070 0x7f59e80a5530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a040ff6b0 tx=0x7f59f00058b0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2801809972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 msgr2=0x55c352828390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.564+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c352828390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f59f400a4b0 tx=0x7f59f40096d0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2801809972 shutdown_connections 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c352828390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2801809972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c35281f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2801809972 >> 172.21.15.3:0/2801809972 conn(0x55c352819b00 msgr2=0x55c352819ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2801809972 shutdown_connections 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2801809972 wait complete. 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 -- start start 2024-03-21T08:35:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.565+0000 7f5a0d00c740 1 Processor -- start 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281f1e0 0x55c352825940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281fb00 0x55c352825e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c3528288d0 0x55c3528263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35273fa20 con 0x55c35281fb00 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35277f8d0 con 0x55c35281f1e0 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35282b500 con 0x55c3528288d0 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a0d00c740 10 client.0 _ll_register_callbacks cb 0x55c35277c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281fb00 0x55c352825e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281fb00 0x55c352825e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41406/0 (socket says 172.21.15.3:41406) 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a02a00640 1 -- 172.21.15.3:0/2289784087 learned_addr learned my addr 172.21.15.3:0/2289784087 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.566+0000 7f5a03400640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281f1e0 0x55c352825940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a03e00640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c3528288d0 0x55c3528263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a02a00640 1 -- 172.21.15.3:0/2289784087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c3528288d0 msgr2=0x55c3528263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a02a00640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c3528288d0 0x55c3528263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a02a00640 1 -- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281f1e0 msgr2=0x55c352825940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a02a00640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281f1e0 0x55c352825940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a02a00640 1 -- 172.21.15.3:0/2289784087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c352498210 con 0x55c35281fb00 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a03400640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281f1e0 0x55c352825940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f5a02a00640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281fb00 0x55c352825e80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59f0035000 tx=0x7f59f002bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7937e5f324d5d779 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f59ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f59ecc00640 1 -- 172.21.15.3:0/2289784087 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0052020 con 0x55c35281fb00 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.567+0000 7f59ecc00640 1 -- 172.21.15.3:0/2289784087 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f59f0003d70 con 0x55c35281fb00 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.568+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59f4014cc0 con 0x55c35281fb00 2024-03-21T08:35:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.568+0000 7f59ecc00640 1 -- 172.21.15.3:0/2289784087 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f005d050 con 0x55c35281fb00 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.568+0000 7f5a0d00c740 10 client.16503 fetch_fsmap learned FSMap version 27 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.568+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c352564620 con 0x55c35281fb00 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.568+0000 7f59ecc00640 1 -- 172.21.15.3:0/2289784087 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59f4014cc0 con 0x55c35281fb00 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.568+0000 7f59ecc00640 1 -- 172.21.15.3:0/2289784087 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f59f0040040 con 0x55c35281fb00 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.569+0000 7f5a0d00c740 10 client.16503 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.569+0000 7f5a0d00c740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.569+0000 7f5a0d00c740 20 client.16503 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.569+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c3528389c0 0x55c35283ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.569+0000 7f5a0d00c740 4 client.16503 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.569+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c352790d70 con 0x55c3528389c0 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.569+0000 7f5a03400640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c3528389c0 0x55c35283ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.570+0000 7f5a03400640 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c3528389c0 0x55c35283ae60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbee97c56aa4bd8d server_cookie=3bdb63641147e940 in_seq=0 out_seq=0 2024-03-21T08:35:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.570+0000 7f59ecc00640 10 client.16503 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f59ecc00640 1 -- 172.21.15.3:0/2289784087 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c352790d70 con 0x55c3528389c0 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f59ecc00640 10 client.16503 handle_command_reply: tid=0 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 client.16503 shutdown 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281fb00 msgr2=0x55c352825e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281fb00 0x55c352825e80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59f0035000 tx=0x7f59f002bcc0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 shutdown_connections 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c3528389c0 0x55c35283ae60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c3528288d0 0x55c3528263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281fb00 0x55c352825e80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2289784087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281f1e0 0x55c352825940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 >> 172.21.15.3:0/2289784087 conn(0x55c352819b00 msgr2=0x55c35281b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f59ecc00640 0 client.16503 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 shutdown_connections 2024-03-21T08:35:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2289784087 wait complete. 2024-03-21T08:35:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.571+0000 7f5a0d00c740 20 client.16503 trim_cache size 0 max 16384 2024-03-21T08:35:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 Processor -- start 2024-03-21T08:35:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 -- start start 2024-03-21T08:35:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281f1e0 0x55c352828070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281fb00 0x55c352824210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3528288d0 0x55c352824750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a041b7610 con 0x55c35281f1e0 2024-03-21T08:35:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a041b50e0 con 0x55c3528288d0 2024-03-21T08:35:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.587+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a041b3180 con 0x55c35281fb00 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281f1e0 0x55c352828070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3528288d0 0x55c352824750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3528288d0 0x55c352824750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:48588/0 (socket says 172.21.15.3:48588) 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 -- 172.21.15.3:0/1128411911 learned_addr learned my addr 172.21.15.3:0/1128411911 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a02a00640 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281fb00 0x55c352824210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 -- 172.21.15.3:0/1128411911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281fb00 msgr2=0x55c352824210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281fb00 0x55c352824210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 -- 172.21.15.3:0/1128411911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281f1e0 msgr2=0x55c352828070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281f1e0 0x55c352828070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03e00640 1 -- 172.21.15.3:0/1128411911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35281e640 con 0x55c3528288d0 2024-03-21T08:35:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a02a00640 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281fb00 0x55c352824210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.588+0000 7f5a03400640 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281f1e0 0x55c352828070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.589+0000 7f5a03e00640 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3528288d0 0x55c352824750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0cc850 tx=0x7f59fc09af10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a525b330879a541c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.589+0000 7f59ef400640 1 -- 172.21.15.3:0/1128411911 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc020040 con 0x55c3528288d0 2024-03-21T08:35:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.589+0000 7f59ef400640 1 -- 172.21.15.3:0/1128411911 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59fc09f5e0 con 0x55c3528288d0 2024-03-21T08:35:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.589+0000 7f59ef400640 1 -- 172.21.15.3:0/1128411911 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc09f120 con 0x55c3528288d0 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 -- 172.21.15.3:0/1128411911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3528288d0 msgr2=0x55c352824750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3528288d0 0x55c352824750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0cc850 tx=0x7f59fc09af10 comp rx=0 tx=0).stop 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 -- 172.21.15.3:0/1128411911 shutdown_connections 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3528288d0 0x55c352824750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281fb00 0x55c352824210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/1128411911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c35281f1e0 0x55c352828070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 -- 172.21.15.3:0/1128411911 >> 172.21.15.3:0/1128411911 conn(0x55c352819b00 msgr2=0x55c352820c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 -- 172.21.15.3:0/1128411911 shutdown_connections 2024-03-21T08:35:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 -- 172.21.15.3:0/1128411911 wait complete. 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.590+0000 7f5a0d00c740 1 -- start start 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 1 Processor -- start 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c3528301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c35282b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a041b3780 con 0x55c3528288d0 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a041b4a80 con 0x55c35281fb00 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a041b3dc0 con 0x55c35281f1e0 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a0d00c740 10 client.0 _ll_register_callbacks cb 0x7f59f0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41420/0 (socket says 172.21.15.3:41420) 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a03e00640 1 -- 172.21.15.3:0/2198625845 learned_addr learned my addr 172.21.15.3:0/2198625845 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.591+0000 7f5a02a00640 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c35282b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a03400640 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c3528301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a03e00640 1 -- 172.21.15.3:0/2198625845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 msgr2=0x55c3528301b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a03e00640 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c3528301b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a03e00640 1 -- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 msgr2=0x55c35282b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a03e00640 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c35282b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a03e00640 1 -- 172.21.15.3:0/2198625845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c35282c190 con 0x55c3528288d0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a03e00640 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282bc50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f59fc005d00 tx=0x7f59fc0999a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f572cd3ab23851f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f59ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f59ecc00640 1 -- 172.21.15.3:0/2198625845 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc014030 con 0x55c3528288d0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f59ecc00640 1 -- 172.21.15.3:0/2198625845 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f59fc097b00 con 0x55c3528288d0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.592+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59fc09f5e0 con 0x55c3528288d0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f59ecc00640 1 -- 172.21.15.3:0/2198625845 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59fc020040 con 0x55c3528288d0 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 10 client.16509 fetch_fsmap learned FSMap version 27 2024-03-21T08:35:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c352893f80 con 0x55c3528288d0 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f59ecc00640 1 -- 172.21.15.3:0/2198625845 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59fc09f5e0 con 0x55c3528288d0 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f59ecc00640 1 -- 172.21.15.3:0/2198625845 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f59fc0b4280 con 0x55c3528288d0 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 10 client.16509 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 20 client.16509 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c352894650 0x55c352896a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 4 client.16509 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:35:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.593+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c35278b0c0 con 0x55c352894650 2024-03-21T08:35:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.594+0000 7f5a03400640 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c352894650 0x55c352896a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.594+0000 7f5a03400640 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c352894650 0x55c352896a70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91e4c4ebbce8116a server_cookie=2e8862113d9bbef1 in_seq=0 out_seq=0 2024-03-21T08:35:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.594+0000 7f59ecc00640 10 client.16509 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f59ecc00640 1 -- 172.21.15.3:0/2198625845 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c35278b0c0 con 0x55c352894650 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f59ecc00640 10 client.16509 handle_command_reply: tid=0 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 client.16509 shutdown 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 msgr2=0x55c35282bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282bc50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f59fc005d00 tx=0x7f59fc0999a0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 shutdown_connections 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c3528288d0 0x55c35282bc50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c35281fb00 0x55c35282b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55c352894650 0x55c352896a70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 --2- 172.21.15.3:0/2198625845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c35281f1e0 0x55c3528301b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 >> 172.21.15.3:0/2198625845 conn(0x55c352819b00 msgr2=0x55c352823de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.666+0000 7f59ecc00640 0 client.16509 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.667+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 shutdown_connections 2024-03-21T08:35:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.667+0000 7f5a0d00c740 1 -- 172.21.15.3:0/2198625845 wait complete. 2024-03-21T08:35:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.667+0000 7f5a0d00c740 20 client.16509 trim_cache size 0 max 16384 2024-03-21T08:35:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:35:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:35:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:35:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:35:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f59e80a3070 msgr2=0x7f59e80a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f59e80a3070 0x7f59e80a5530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a040ff6b0 tx=0x7f59f00058b0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04112490 msgr2=0x7f5a041a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04112490 0x7f5a041a2bd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d720 tx=0x7f59fc00dbe0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 shutdown_connections 2024-03-21T08:35:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a04112490 0x7f5a041a2bd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f59e80a3070 0x7f59e80a5530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a04074360 0x7f5a0419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 --2- 172.21.15.3:0/867184709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a04073a40 0x7f5a0419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 >> 172.21.15.3:0/867184709 conn(0x7f5a040fcf50 msgr2=0x7f5a040fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 shutdown_connections 2024-03-21T08:35:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:03.668+0000 7f5a08c00640 1 -- 172.21.15.3:0/867184709 wait complete. 2024-03-21T08:35:04.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:03 smithi003 ceph-mon[31675]: pgmap v99: 129 pgs: 129 active+clean; 47 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 768 KiB/s wr, 215 op/s 2024-03-21T08:35:04.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:03 smithi084 ceph-mon[35477]: pgmap v99: 129 pgs: 129 active+clean; 47 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 768 KiB/s wr, 215 op/s 2024-03-21T08:35:04.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:03 smithi171 ceph-mon[35388]: pgmap v99: 129 pgs: 129 active+clean; 47 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 768 KiB/s wr, 215 op/s 2024-03-21T08:35:05.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:04 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:35:05.279 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:04 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:35:05.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:04 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:35:06.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:05 smithi003 ceph-mon[31675]: pgmap v100: 129 pgs: 129 active+clean; 47 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 652 KiB/s wr, 173 op/s 2024-03-21T08:35:06.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:05 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:35:06.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:05 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:35:06.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:05 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:35:06.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:05 smithi084 ceph-mon[35477]: pgmap v100: 129 pgs: 129 active+clean; 47 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 652 KiB/s wr, 173 op/s 2024-03-21T08:35:06.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:05 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:35:06.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:05 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:35:06.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:05 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:35:06.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:05 smithi171 ceph-mon[35388]: pgmap v100: 129 pgs: 129 active+clean; 47 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 652 KiB/s wr, 173 op/s 2024-03-21T08:35:06.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:05 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:35:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:05 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:35:06.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:05 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:35:08.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:07 smithi003 ceph-mon[31675]: pgmap v101: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-03-21T08:35:08.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:07 smithi084 ceph-mon[35477]: pgmap v101: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-03-21T08:35:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:07 smithi171 ceph-mon[35388]: pgmap v101: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-03-21T08:35:10.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:09 smithi084 ceph-mon[35477]: pgmap v102: 129 pgs: 129 active+clean; 48 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 930 KiB/s wr, 192 op/s 2024-03-21T08:35:10.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:09 smithi171 ceph-mon[35388]: pgmap v102: 129 pgs: 129 active+clean; 48 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 930 KiB/s wr, 192 op/s 2024-03-21T08:35:10.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:09 smithi003 ceph-mon[31675]: pgmap v102: 129 pgs: 129 active+clean; 48 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 930 KiB/s wr, 192 op/s 2024-03-21T08:35:12.229 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:11 smithi003 ceph-mon[31675]: pgmap v103: 129 pgs: 129 active+clean; 48 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 930 KiB/s wr, 187 op/s 2024-03-21T08:35:12.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:11 smithi084 ceph-mon[35477]: pgmap v103: 129 pgs: 129 active+clean; 48 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 930 KiB/s wr, 187 op/s 2024-03-21T08:35:12.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:11 smithi171 ceph-mon[35388]: pgmap v103: 129 pgs: 129 active+clean; 48 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 930 KiB/s wr, 187 op/s 2024-03-21T08:35:14.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:13 smithi084 ceph-mon[35477]: pgmap v104: 129 pgs: 129 active+clean; 55 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-03-21T08:35:14.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:13 smithi171 ceph-mon[35388]: pgmap v104: 129 pgs: 129 active+clean; 55 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-03-21T08:35:14.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:13 smithi003 ceph-mon[31675]: pgmap v104: 129 pgs: 129 active+clean; 55 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-03-21T08:35:16.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:15 smithi084 ceph-mon[35477]: pgmap v105: 129 pgs: 129 active+clean; 55 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-03-21T08:35:16.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:15 smithi171 ceph-mon[35388]: pgmap v105: 129 pgs: 129 active+clean; 55 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-03-21T08:35:16.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:15 smithi003 ceph-mon[31675]: pgmap v105: 129 pgs: 129 active+clean; 55 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.841+0000 7ff6aac00640 1 -- 172.21.15.3:0/1945916226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4069850 msgr2=0x7ff6a410e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.841+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1945916226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4069850 0x7ff6a410e330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009ca0 tx=0x7ff69c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.841+0000 7ff6aac00640 1 -- 172.21.15.3:0/1945916226 shutdown_connections 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.841+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1945916226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4069850 0x7ff6a410e330 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.841+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1945916226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6a4068e90 0x7ff6a4069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.841+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1945916226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a41075b0 0x7ff6a4107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.841+0000 7ff6aac00640 1 -- 172.21.15.3:0/1945916226 >> 172.21.15.3:0/1945916226 conn(0x7ff6a4075f00 msgr2=0x7ff6a4076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.842+0000 7ff6aac00640 1 -- 172.21.15.3:0/1945916226 shutdown_connections 2024-03-21T08:35:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.842+0000 7ff6aac00640 1 -- 172.21.15.3:0/1945916226 wait complete. 2024-03-21T08:35:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 Processor -- start 2024-03-21T08:35:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 -- start start 2024-03-21T08:35:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4068e90 0x7ff6a41006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a4069850 0x7ff6a4100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6a41075b0 0x7ff6a4101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4111700 con 0x7ff6a4068e90 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4111580 con 0x7ff6a4069850 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6aac00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4111880 con 0x7ff6a41075b0 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6a9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4068e90 0x7ff6a41006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6a9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4068e90 0x7ff6a41006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56644/0 (socket says 172.21.15.3:56644) 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.843+0000 7ff6a9800640 1 -- 172.21.15.3:0/1383961152 learned_addr learned my addr 172.21.15.3:0/1383961152 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a4069850 0x7ff6a4100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6aa200640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6a41075b0 0x7ff6a4101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6a9800640 1 -- 172.21.15.3:0/1383961152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6a41075b0 msgr2=0x7ff6a4101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6a9800640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6a41075b0 0x7ff6a4101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6a9800640 1 -- 172.21.15.3:0/1383961152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a4069850 msgr2=0x7ff6a4100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6a9800640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a4069850 0x7ff6a4100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6a9800640 1 -- 172.21.15.3:0/1383961152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a41a60a0 con 0x7ff6a4068e90 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6aa200640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6a41075b0 0x7ff6a4101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.844+0000 7ff6a9800640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4068e90 0x7ff6a41006f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9f0 tx=0x7ff69800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.845+0000 7ff6a2000640 1 -- 172.21.15.3:0/1383961152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800cd80 con 0x7ff6a4068e90 2024-03-21T08:35:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.845+0000 7ff6a2000640 1 -- 172.21.15.3:0/1383961152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6980108e0 con 0x7ff6a4068e90 2024-03-21T08:35:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.845+0000 7ff6a2000640 1 -- 172.21.15.3:0/1383961152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698018920 con 0x7ff6a4068e90 2024-03-21T08:35:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.845+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a41a6390 con 0x7ff6a4068e90 2024-03-21T08:35:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.845+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a41a6890 con 0x7ff6a4068e90 2024-03-21T08:35:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 Processor -- start 2024-03-21T08:35:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6a2000640 1 -- 172.21.15.3:0/1383961152 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7ff698010a90 con 0x7ff6a4068e90 2024-03-21T08:35:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 -- start start 2024-03-21T08:35:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eacb00 0x5586d0eb5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586d0e6e3b0 con 0x5586d0eb58d0 2024-03-21T08:35:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586d0e10830 con 0x5586d0eac1e0 2024-03-21T08:35:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586d0db95d0 con 0x5586d0eacb00 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.849+0000 7ff6aa200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6aa200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:51868/0 (socket says 172.21.15.3:51868) 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6aa200640 1 -- 172.21.15.3:0/3166789090 learned_addr learned my addr 172.21.15.3:0/3166789090 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6a9800640 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eacb00 0x5586d0eb5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6aa200640 1 -- 172.21.15.3:0/3166789090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eacb00 msgr2=0x5586d0eb5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6aa200640 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eacb00 0x5586d0eb5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6aa200640 1 -- 172.21.15.3:0/3166789090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 msgr2=0x5586d0eb7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6aa200640 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6aa200640 1 -- 172.21.15.3:0/3166789090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586d0dd12c0 con 0x5586d0eac1e0 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.850+0000 7ff6a9800640 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eacb00 0x5586d0eb5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.851+0000 7ff6aa200640 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eac5c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff69c037040 tx=0x7ff69c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947a2bda6fc3e081 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.851+0000 7ff6a2000640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff68c0a3120 0x7ff68c0a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff6a2000640 1 -- 172.21.15.3:0/1383961152 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7ff6980cfec0 con 0x7ff6a4068e90 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff696a00640 1 -- 172.21.15.3:0/3166789090 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c039050 con 0x5586d0eac1e0 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff696a00640 1 -- 172.21.15.3:0/3166789090 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff69c045070 con 0x5586d0eac1e0 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff6a9800640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff68c0a3120 0x7ff68c0a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff696a00640 1 -- 172.21.15.3:0/3166789090 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c004040 con 0x5586d0eac1e0 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff6aee7c740 1 -- 172.21.15.3:0/3166789090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 msgr2=0x5586d0eac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eac5c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff69c037040 tx=0x7ff69c009800 comp rx=0 tx=0).stop 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.852+0000 7ff6a9800640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff68c0a3120 0x7ff68c0a55e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5586d0eabc90 tx=0x7ff69800cf20 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 -- 172.21.15.3:0/3166789090 shutdown_connections 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eacb00 0x5586d0eb5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/3166789090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eac5c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 -- 172.21.15.3:0/3166789090 >> 172.21.15.3:0/3166789090 conn(0x5586d0ea6b00 msgr2=0x5586d0ea6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 -- 172.21.15.3:0/3166789090 shutdown_connections 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 -- 172.21.15.3:0/3166789090 wait complete. 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.853+0000 7ff6aee7c740 1 -- start start 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 1 Processor -- start 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eac1e0 0x5586d0eb2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eacb00 0x5586d0eb2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586d0dcca20 con 0x5586d0eb58d0 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586d0e0c8d0 con 0x5586d0eacb00 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586d0eb84a0 con 0x5586d0eac1e0 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aee7c740 10 client.0 _ll_register_callbacks cb 0x5586d0e091c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eacb00 0x5586d0eb2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.854+0000 7ff6aa200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eacb00 0x5586d0eb2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:51870/0 (socket says 172.21.15.3:51870) 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56656/0 (socket says 172.21.15.3:56656) 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6a8e00640 1 -- 172.21.15.3:0/465846458 learned_addr learned my addr 172.21.15.3:0/465846458 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eac1e0 0x5586d0eb2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 -- 172.21.15.3:0/465846458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eac1e0 msgr2=0x5586d0eb2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eac1e0 0x5586d0eb2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 -- 172.21.15.3:0/465846458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eacb00 msgr2=0x5586d0eb2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eacb00 0x5586d0eb2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 -- 172.21.15.3:0/465846458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586d0b25210 con 0x5586d0eb58d0 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff6aa200640 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb33a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009430 tx=0x7ff69c003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f883fe836168ad90 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff68b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.855+0000 7ff68b400640 1 -- 172.21.15.3:0/465846458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c032700 con 0x5586d0eb58d0 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.856+0000 7ff68b400640 1 -- 172.21.15.3:0/465846458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7ff69c0328a0 con 0x5586d0eb58d0 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.856+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff69c045070 con 0x5586d0eb58d0 2024-03-21T08:35:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.856+0000 7ff68b400640 1 -- 172.21.15.3:0/465846458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c004140 con 0x5586d0eb58d0 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.856+0000 7ff6aee7c740 10 client.16521 fetch_fsmap learned FSMap version 27 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.856+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5586d0bf1620 con 0x5586d0eb58d0 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.856+0000 7ff68b400640 1 -- 172.21.15.3:0/465846458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff69c045070 con 0x5586d0eb58d0 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.856+0000 7ff68b400640 1 -- 172.21.15.3:0/465846458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7ff69c040a00 con 0x5586d0eb58d0 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.857+0000 7ff6aee7c740 10 client.16521 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.857+0000 7ff6aee7c740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.857+0000 7ff6aee7c740 20 client.16521 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.857+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0ec5690 0x5586d0ec7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.857+0000 7ff6aee7c740 4 client.16521 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:35:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.857+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586d0e1dd70 con 0x5586d0ec5690 2024-03-21T08:35:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.857+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0ec5690 0x5586d0ec7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.858+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0ec5690 0x5586d0ec7a80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5b16e97b10e857 server_cookie=5186489f47fbf6d4 in_seq=0 out_seq=0 2024-03-21T08:35:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.858+0000 7ff68b400640 10 client.16521 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff68b400640 1 -- 172.21.15.3:0/465846458 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5586d0e1dd70 con 0x5586d0ec5690 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff68b400640 10 client.16521 handle_command_reply: tid=0 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 client.16521 shutdown 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 msgr2=0x5586d0eb33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb33a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009430 tx=0x7ff69c003bb0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 shutdown_connections 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eb58d0 0x5586d0eb33a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eacb00 0x5586d0eb2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0ec5690 0x5586d0ec7a80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/465846458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eac1e0 0x5586d0eb2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 >> 172.21.15.3:0/465846458 conn(0x5586d0ea6b00 msgr2=0x5586d0ea8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.859+0000 7ff68b400640 0 client.16521 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.860+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 shutdown_connections 2024-03-21T08:35:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.860+0000 7ff6aee7c740 1 -- 172.21.15.3:0/465846458 wait complete. 2024-03-21T08:35:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.860+0000 7ff6aee7c740 20 client.16521 trim_cache size 0 max 16384 2024-03-21T08:35:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.877+0000 7ff6aee7c740 1 Processor -- start 2024-03-21T08:35:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.877+0000 7ff6aee7c740 1 -- start start 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.877+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eb31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0eafba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 0x5586d0eb00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41b3170 con 0x5586d0eacb00 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41b0c40 con 0x5586d0eac1e0 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41aece0 con 0x5586d0eb58d0 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0eafba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0eafba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56664/0 (socket says 172.21.15.3:56664) 2024-03-21T08:35:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 -- 172.21.15.3:0/456293910 learned_addr learned my addr 172.21.15.3:0/456293910 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6aa200640 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eb31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 -- 172.21.15.3:0/456293910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 msgr2=0x5586d0eb00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 0x5586d0eb00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 -- 172.21.15.3:0/456293910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 msgr2=0x5586d0eb31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eb31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.878+0000 7ff6a8e00640 1 -- 172.21.15.3:0/456293910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586d0ea1d10 con 0x5586d0eacb00 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.879+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0eafba0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff69000bc80 tx=0x7ff69000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950f1056cbb9a297 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.879+0000 7ff696000640 1 -- 172.21.15.3:0/456293910 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690020160 con 0x5586d0eacb00 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.879+0000 7ff696000640 1 -- 172.21.15.3:0/456293910 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff690020300 con 0x5586d0eacb00 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.880+0000 7ff696000640 1 -- 172.21.15.3:0/456293910 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690011ab0 con 0x5586d0eacb00 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 -- 172.21.15.3:0/456293910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 msgr2=0x5586d0eafba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0eafba0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff69000bc80 tx=0x7ff69000b390 comp rx=0 tx=0).stop 2024-03-21T08:35:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 -- 172.21.15.3:0/456293910 shutdown_connections 2024-03-21T08:35:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 0x5586d0eb00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0eafba0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/456293910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0eb31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 -- 172.21.15.3:0/456293910 >> 172.21.15.3:0/456293910 conn(0x5586d0ea6b00 msgr2=0x5586d0eadc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 -- 172.21.15.3:0/456293910 shutdown_connections 2024-03-21T08:35:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 -- 172.21.15.3:0/456293910 wait complete. 2024-03-21T08:35:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 -- start start 2024-03-21T08:35:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.881+0000 7ff6aee7c740 1 Processor -- start 2024-03-21T08:35:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0ec8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0ec8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aee7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 0x5586d0ec8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41af2e0 con 0x5586d0eacb00 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41b05e0 con 0x5586d0eac1e0 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aee7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41af920 con 0x5586d0eb58d0 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aee7c740 10 client.0 _ll_register_callbacks cb 0x7ff690004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0ec8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0ec8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56676/0 (socket says 172.21.15.3:56676) 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6a8e00640 1 -- 172.21.15.3:0/650058309 learned_addr learned my addr 172.21.15.3:0/650058309 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.882+0000 7ff6aa200640 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0ec8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff6a8e00640 1 -- 172.21.15.3:0/650058309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 msgr2=0x5586d0ec8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 0x5586d0ec8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff6a8e00640 1 -- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 msgr2=0x5586d0ec8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0ec8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff6a8e00640 1 -- 172.21.15.3:0/650058309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586d0f28620 con 0x5586d0eacb00 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff6a8e00640 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0ec8660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff690023000 tx=0x7ff6900201e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0741ebbed4edf20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff68b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.883+0000 7ff68b400640 1 -- 172.21.15.3:0/650058309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690003af0 con 0x5586d0eacb00 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff690020300 con 0x5586d0eacb00 2024-03-21T08:35:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff68b400640 1 -- 172.21.15.3:0/650058309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7ff690003c90 con 0x5586d0eacb00 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff68b400640 1 -- 172.21.15.3:0/650058309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff690003e90 con 0x5586d0eacb00 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff6aee7c740 10 client.16527 fetch_fsmap learned FSMap version 27 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5586d0f29930 con 0x5586d0eacb00 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff68b400640 1 -- 172.21.15.3:0/650058309 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff690020300 con 0x5586d0eacb00 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff68b400640 1 -- 172.21.15.3:0/650058309 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7ff690030070 con 0x5586d0eacb00 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff6aee7c740 10 client.16527 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff6aee7c740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.884+0000 7ff6aee7c740 20 client.16527 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.885+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0f29fb0 0x5586d0f2c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.885+0000 7ff6aee7c740 4 client.16527 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:35:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.885+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5586d0e180c0 con 0x5586d0f29fb0 2024-03-21T08:35:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.885+0000 7ff6aa200640 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0f29fb0 0x5586d0f2c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.885+0000 7ff6aa200640 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0f29fb0 0x5586d0f2c3d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77fb3a3e3daa96a server_cookie=cac4570d1553451 in_seq=0 out_seq=0 2024-03-21T08:35:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.885+0000 7ff68b400640 10 client.16527 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff68b400640 1 -- 172.21.15.3:0/650058309 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5586d0e180c0 con 0x5586d0f29fb0 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff68b400640 10 client.16527 handle_command_reply: tid=0 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 client.16527 shutdown 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 msgr2=0x5586d0ec8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0ec8660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff690023000 tx=0x7ff6900201e0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 shutdown_connections 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5586d0f29fb0 0x5586d0f2c3d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d0eb58d0 0x5586d0ec8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5586d0eacb00 0x5586d0ec8660 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff68b400640 0 client.16527 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 --2- 172.21.15.3:0/650058309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586d0eac1e0 0x5586d0ec8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.949+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 >> 172.21.15.3:0/650058309 conn(0x5586d0ea6b00 msgr2=0x5586d0eb3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.950+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 shutdown_connections 2024-03-21T08:35:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.950+0000 7ff6aee7c740 1 -- 172.21.15.3:0/650058309 wait complete. 2024-03-21T08:35:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.950+0000 7ff6aee7c740 20 client.16527 trim_cache size 0 max 16384 2024-03-21T08:35:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:35:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:35:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:35:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff68c0a3120 msgr2=0x7ff68c0a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff68c0a3120 0x7ff68c0a55e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5586d0eabc90 tx=0x7ff69800cf20 comp rx=0 tx=0).stop 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4068e90 msgr2=0x7ff6a41006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4068e90 0x7ff6a41006f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9f0 tx=0x7ff69800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 shutdown_connections 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff68c0a3120 0x7ff68c0a55e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6a41075b0 0x7ff6a4101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a4069850 0x7ff6a4100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 --2- 172.21.15.3:0/1383961152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a4068e90 0x7ff6a41006f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 >> 172.21.15.3:0/1383961152 conn(0x7ff6a4075f00 msgr2=0x7ff6a40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 shutdown_connections 2024-03-21T08:35:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:16.951+0000 7ff6aac00640 1 -- 172.21.15.3:0/1383961152 wait complete. 2024-03-21T08:35:18.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:17 smithi084 ceph-mon[35477]: pgmap v106: 129 pgs: 129 active+clean; 62 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-03-21T08:35:18.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:17 smithi171 ceph-mon[35388]: pgmap v106: 129 pgs: 129 active+clean; 62 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-03-21T08:35:18.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:17 smithi003 ceph-mon[31675]: pgmap v106: 129 pgs: 129 active+clean; 62 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-03-21T08:35:20.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:19 smithi084 ceph-mon[35477]: pgmap v107: 129 pgs: 129 active+clean; 62 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-21T08:35:20.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:19 smithi171 ceph-mon[35388]: pgmap v107: 129 pgs: 129 active+clean; 62 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-21T08:35:20.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:19 smithi003 ceph-mon[31675]: pgmap v107: 129 pgs: 129 active+clean; 62 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-21T08:35:22.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:21 smithi084 ceph-mon[35477]: pgmap v108: 129 pgs: 129 active+clean; 62 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-03-21T08:35:22.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:21 smithi171 ceph-mon[35388]: pgmap v108: 129 pgs: 129 active+clean; 62 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-03-21T08:35:22.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:21 smithi003 ceph-mon[31675]: pgmap v108: 129 pgs: 129 active+clean; 62 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-03-21T08:35:24.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:23 smithi084 ceph-mon[35477]: pgmap v109: 129 pgs: 129 active+clean; 63 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-03-21T08:35:24.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:23 smithi171 ceph-mon[35388]: pgmap v109: 129 pgs: 129 active+clean; 63 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-03-21T08:35:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:23 smithi003 ceph-mon[31675]: pgmap v109: 129 pgs: 129 active+clean; 63 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-03-21T08:35:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:25 smithi084 ceph-mon[35477]: pgmap v110: 129 pgs: 129 active+clean; 63 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1002 KiB/s wr, 144 op/s 2024-03-21T08:35:26.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:25 smithi171 ceph-mon[35388]: pgmap v110: 129 pgs: 129 active+clean; 63 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1002 KiB/s wr, 144 op/s 2024-03-21T08:35:26.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:25 smithi003 ceph-mon[31675]: pgmap v110: 129 pgs: 129 active+clean; 63 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1002 KiB/s wr, 144 op/s 2024-03-21T08:35:28.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:27 smithi084 ceph-mon[35477]: pgmap v111: 129 pgs: 129 active+clean; 66 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-21T08:35:28.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:27 smithi171 ceph-mon[35388]: pgmap v111: 129 pgs: 129 active+clean; 66 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-21T08:35:28.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:27 smithi003 ceph-mon[31675]: pgmap v111: 129 pgs: 129 active+clean; 66 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-21T08:35:30.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.102+0000 7fe948000640 1 -- 172.21.15.3:0/2245757094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 msgr2=0x7fe940073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.102+0000 7fe948000640 1 --2- 172.21.15.3:0/2245757094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe940073e50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe934009900 tx=0x7fe93402bcd0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.103+0000 7fe948000640 1 -- 172.21.15.3:0/2245757094 shutdown_connections 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.103+0000 7fe948000640 1 --2- 172.21.15.3:0/2245757094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe94010def0 0x7fe940110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.103+0000 7fe948000640 1 --2- 172.21.15.3:0/2245757094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe940074390 0x7fe94010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.103+0000 7fe948000640 1 --2- 172.21.15.3:0/2245757094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe940073e50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.103+0000 7fe948000640 1 -- 172.21.15.3:0/2245757094 >> 172.21.15.3:0/2245757094 conn(0x7fe9400fd120 msgr2=0x7fe9400ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.103+0000 7fe948000640 1 -- 172.21.15.3:0/2245757094 shutdown_connections 2024-03-21T08:35:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.103+0000 7fe948000640 1 -- 172.21.15.3:0/2245757094 wait complete. 2024-03-21T08:35:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 Processor -- start 2024-03-21T08:35:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 -- start start 2024-03-21T08:35:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe94019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe940074390 0x7fe94019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe94010def0 0x7fe94019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe940113b50 con 0x7fe940073a70 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401139d0 con 0x7fe940074390 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.104+0000 7fe948000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe940113cd0 con 0x7fe94010def0 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe94019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe94019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54350/0 (socket says 172.21.15.3:54350) 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 -- 172.21.15.3:0/584100486 learned_addr learned my addr 172.21.15.3:0/584100486 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe944c00640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe940074390 0x7fe94019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe946000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe94010def0 0x7fe94019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 -- 172.21.15.3:0/584100486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe94010def0 msgr2=0x7fe94019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe94010def0 0x7fe94019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 -- 172.21.15.3:0/584100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe940074390 msgr2=0x7fe94019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe940074390 0x7fe94019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 -- 172.21.15.3:0/584100486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94019f0a0 con 0x7fe940073a70 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe946000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe94010def0 0x7fe94019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.105+0000 7fe945600640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe94019a0b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe934000c00 tx=0x7fe93402d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.106+0000 7fe93e000640 1 -- 172.21.15.3:0/584100486 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934043020 con 0x7fe940073a70 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.106+0000 7fe93e000640 1 -- 172.21.15.3:0/584100486 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe93402dd50 con 0x7fe940073a70 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.106+0000 7fe93e000640 1 -- 172.21.15.3:0/584100486 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934003ab0 con 0x7fe940073a70 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.106+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94019f330 con 0x7fe940073a70 2024-03-21T08:35:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.106+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9401a6c20 con 0x7fe940073a70 2024-03-21T08:35:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.109+0000 7fe93e000640 1 -- 172.21.15.3:0/584100486 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fe934046020 con 0x7fe940073a70 2024-03-21T08:35:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 Processor -- start 2024-03-21T08:35:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 -- start start 2024-03-21T08:35:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae97643fd0 0x55ae976443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae9764fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae976063b0 con 0x55ae97643fd0 2024-03-21T08:35:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae975a8830 con 0x55ae976448f0 2024-03-21T08:35:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe94adf5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae975515d0 con 0x55ae9764d6c0 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.110+0000 7fe946000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae97643fd0 0x55ae976443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe945600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae97643fd0 0x55ae976443b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54362/0 (socket says 172.21.15.3:54362) 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 -- 172.21.15.3:0/2463250227 learned_addr learned my addr 172.21.15.3:0/2463250227 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe944c00640 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae9764fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 -- 172.21.15.3:0/2463250227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 msgr2=0x55ae9764fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae9764fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 -- 172.21.15.3:0/2463250227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 msgr2=0x55ae9764d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 -- 172.21.15.3:0/2463250227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae975692c0 con 0x55ae97643fd0 2024-03-21T08:35:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe945600640 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe93e000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9280a3100 0x7fe9280a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe946000640 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae97643fd0 0x55ae976443b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe938008cf0 tx=0x7fe93800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03cc78b0bb197bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.111+0000 7fe93e000640 1 -- 172.21.15.3:0/584100486 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fe9340f1b80 con 0x7fe940073a70 2024-03-21T08:35:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.112+0000 7fe945600640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9280a3100 0x7fe9280a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.112+0000 7fe932600640 1 -- 172.21.15.3:0/2463250227 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938018680 con 0x55ae97643fd0 2024-03-21T08:35:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.112+0000 7fe932600640 1 -- 172.21.15.3:0/2463250227 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe938018cd0 con 0x55ae97643fd0 2024-03-21T08:35:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.112+0000 7fe932600640 1 -- 172.21.15.3:0/2463250227 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93800fe30 con 0x55ae97643fd0 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.112+0000 7fe94adf5740 1 -- 172.21.15.3:0/2463250227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae97643fd0 msgr2=0x55ae976443b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.112+0000 7fe94adf5740 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae97643fd0 0x55ae976443b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe938008cf0 tx=0x7fe93800d710 comp rx=0 tx=0).stop 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.112+0000 7fe945600640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9280a3100 0x7fe9280a55c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ef070 tx=0x7fe9340f56d0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 -- 172.21.15.3:0/2463250227 shutdown_connections 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae9764fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 --2- 172.21.15.3:0/2463250227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae97643fd0 0x55ae976443b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 -- 172.21.15.3:0/2463250227 >> 172.21.15.3:0/2463250227 conn(0x55ae9763e8f0 msgr2=0x55ae9763ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 -- 172.21.15.3:0/2463250227 shutdown_connections 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 -- 172.21.15.3:0/2463250227 wait complete. 2024-03-21T08:35:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 -- start start 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.113+0000 7fe94adf5740 1 Processor -- start 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae97643fd0 0x55ae9764a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae9764b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe94adf5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae97564a20 con 0x55ae9764d6c0 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe94adf5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae975a48d0 con 0x55ae976448f0 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe94adf5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae976502f0 con 0x55ae97643fd0 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe94adf5740 10 client.0 _ll_register_callbacks cb 0x55ae975a11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe944c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe944c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:45678/0 (socket says 172.21.15.3:45678) 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe944c00640 1 -- 172.21.15.3:0/1902008675 learned_addr learned my addr 172.21.15.3:0/1902008675 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe946000640 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae9764b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.114+0000 7fe946000640 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae97643fd0 0x55ae9764a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe944c00640 1 -- 172.21.15.3:0/1902008675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae97643fd0 msgr2=0x55ae9764a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe944c00640 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae97643fd0 0x55ae9764a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe944c00640 1 -- 172.21.15.3:0/1902008675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 msgr2=0x55ae9764b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe944c00640 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae9764b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe944c00640 1 -- 172.21.15.3:0/1902008675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae972bd210 con 0x55ae976448f0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe944c00640 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764ac60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00ba50 tx=0x7fe92c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47438516a6a04bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe926a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe926a00640 1 -- 172.21.15.3:0/1902008675 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c01b640 con 0x55ae976448f0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe938018cd0 con 0x55ae976448f0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.115+0000 7fe926a00640 1 -- 172.21.15.3:0/1902008675 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fe92c011440 con 0x55ae976448f0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe926a00640 1 -- 172.21.15.3:0/1902008675 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c01b7e0 con 0x55ae976448f0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 10 client.36275 fetch_fsmap learned FSMap version 27 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae97389620 con 0x55ae976448f0 2024-03-21T08:35:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe926a00640 1 -- 172.21.15.3:0/1902008675 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe938018cd0 con 0x55ae976448f0 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe926a00640 1 -- 172.21.15.3:0/1902008675 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fe92c019020 con 0x55ae976448f0 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 10 client.36275 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 10 client.36275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 20 client.36275 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae9765d7a0 0x55ae9765fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 4 client.36275 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.116+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae975b5d70 con 0x55ae9765d7a0 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.117+0000 7fe946000640 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae9765d7a0 0x55ae9765fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.117+0000 7fe946000640 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae9765d7a0 0x55ae9765fbc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216a1f03768c2570 server_cookie=ba6502ad91945b09 in_seq=0 out_seq=0 2024-03-21T08:35:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.117+0000 7fe926a00640 10 client.36275 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe926a00640 1 -- 172.21.15.3:0/1902008675 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae975b5d70 con 0x55ae9765d7a0 2024-03-21T08:35:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe926a00640 10 client.36275 handle_command_reply: tid=0 2024-03-21T08:35:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 client.36275 shutdown 2024-03-21T08:35:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 msgr2=0x55ae9764ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764ac60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00ba50 tx=0x7fe92c00bf10 comp rx=0 tx=0).stop 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 shutdown_connections 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae9764b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae976448f0 0x55ae9764ac60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae9765d7a0 0x55ae9765fbc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1902008675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae97643fd0 0x55ae9764a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 >> 172.21.15.3:0/1902008675 conn(0x55ae9763e8f0 msgr2=0x55ae97640640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe926a00640 0 client.36275 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 shutdown_connections 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 1 -- 172.21.15.3:0/1902008675 wait complete. 2024-03-21T08:35:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.119+0000 7fe94adf5740 20 client.36275 trim_cache size 0 max 16384 2024-03-21T08:35:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.135+0000 7fe94adf5740 1 Processor -- start 2024-03-21T08:35:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe94adf5740 1 -- start start 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae9764ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae976448f0 0x55ae97658a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae97658f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe94adf5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401b3310 con 0x55ae9764d6c0 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe94adf5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401b0de0 con 0x55ae97643fd0 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe94adf5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401aee80 con 0x55ae976448f0 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe946000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae9764ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe946000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae9764ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:45694/0 (socket says 172.21.15.3:45694) 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.136+0000 7fe946000640 1 -- 172.21.15.3:0/241911314 learned_addr learned my addr 172.21.15.3:0/241911314 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe946000640 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae97658f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe944c00640 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae976448f0 0x55ae97658a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe946000640 1 -- 172.21.15.3:0/241911314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae976448f0 msgr2=0x55ae97658a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe946000640 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae976448f0 0x55ae97658a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe946000640 1 -- 172.21.15.3:0/241911314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 msgr2=0x55ae97658f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe946000640 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae97658f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe946000640 1 -- 172.21.15.3:0/241911314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae97644720 con 0x55ae97643fd0 2024-03-21T08:35:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.137+0000 7fe946000640 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae9764ce50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe938008c40 tx=0x7fe938008610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f6a2ad368b3188 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe931c00640 1 -- 172.21.15.3:0/241911314 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93800f620 con 0x55ae97643fd0 2024-03-21T08:35:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe931c00640 1 -- 172.21.15.3:0/241911314 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe93800f7c0 con 0x55ae97643fd0 2024-03-21T08:35:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe931c00640 1 -- 172.21.15.3:0/241911314 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380189a0 con 0x55ae97643fd0 2024-03-21T08:35:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe94adf5740 1 -- 172.21.15.3:0/241911314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 msgr2=0x55ae9764ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe94adf5740 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae9764ce50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe938008c40 tx=0x7fe938008610 comp rx=0 tx=0).stop 2024-03-21T08:35:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe94adf5740 1 -- 172.21.15.3:0/241911314 shutdown_connections 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe94adf5740 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae9764d6c0 0x55ae97658f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe94adf5740 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae976448f0 0x55ae97658a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe94adf5740 1 --2- 172.21.15.3:0/241911314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae9764ce50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.138+0000 7fe94adf5740 1 -- 172.21.15.3:0/241911314 >> 172.21.15.3:0/241911314 conn(0x55ae9763e8f0 msgr2=0x55ae97648ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.139+0000 7fe94adf5740 1 -- 172.21.15.3:0/241911314 shutdown_connections 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.139+0000 7fe94adf5740 1 -- 172.21.15.3:0/241911314 wait complete. 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.139+0000 7fe94adf5740 1 -- start start 2024-03-21T08:35:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.139+0000 7fe94adf5740 1 Processor -- start 2024-03-21T08:35:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae97650d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae976448f0 0x55ae97651250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe94adf5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae976b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe94adf5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401af480 con 0x55ae976448f0 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe94adf5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401b0780 con 0x55ae97643fd0 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe94adf5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401afac0 con 0x55ae9764d6c0 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe94adf5740 10 client.0 _ll_register_callbacks cb 0x7fe92c004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe944c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae976448f0 0x55ae97651250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe944c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae976448f0 0x55ae97651250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54388/0 (socket says 172.21.15.3:54388) 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe944c00640 1 -- 172.21.15.3:0/1721295402 learned_addr learned my addr 172.21.15.3:0/1721295402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.140+0000 7fe946000640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae97650d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe946000640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae976b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe944c00640 1 -- 172.21.15.3:0/1721295402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 msgr2=0x55ae976b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe944c00640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae976b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe944c00640 1 -- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 msgr2=0x55ae97650d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe944c00640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae97650d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe944c00640 1 -- 172.21.15.3:0/1721295402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae976b8590 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe946000640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae976b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe944c00640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae976448f0 0x55ae97651250 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x55ae97649f80 tx=0x7fe92c01b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef74c975fadfb4c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe926a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe926a00640 1 -- 172.21.15.3:0/1721295402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c02c020 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe926a00640 1 -- 172.21.15.3:0/1721295402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fe92c01bd90 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.141+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe93800f7c0 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe926a00640 1 -- 172.21.15.3:0/1721295402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c037050 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 10 client.16545 fetch_fsmap learned FSMap version 27 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae976b9740 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe926a00640 1 -- 172.21.15.3:0/1721295402 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe93800f7c0 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe926a00640 1 -- 172.21.15.3:0/1721295402 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fe92c019040 con 0x55ae976448f0 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 10 client.16545 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T08:35:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 20 client.16545 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae976b9ef0 0x55ae976bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 4 client.16545 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:35:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.142+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae975b00c0 con 0x55ae976b9ef0 2024-03-21T08:35:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.143+0000 7fe946000640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae976b9ef0 0x55ae976bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.143+0000 7fe946000640 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae976b9ef0 0x55ae976bc310 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50850ff75dde56c server_cookie=27284497ecbe4767 in_seq=0 out_seq=0 2024-03-21T08:35:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.143+0000 7fe926a00640 10 client.16545 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.217+0000 7fe926a00640 1 -- 172.21.15.3:0/1721295402 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae975b00c0 con 0x55ae976b9ef0 2024-03-21T08:35:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe926a00640 10 client.16545 handle_command_reply: tid=0 2024-03-21T08:35:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 client.16545 shutdown 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae976448f0 msgr2=0x55ae97651250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae976448f0 0x55ae97651250 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x55ae97649f80 tx=0x7fe92c01b750 comp rx=0 tx=0).stop 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 shutdown_connections 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55ae976b9ef0 0x55ae976bc310 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae9764d6c0 0x55ae976b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ae976448f0 0x55ae97651250 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 --2- 172.21.15.3:0/1721295402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae97643fd0 0x55ae97650d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe926a00640 0 client.16545 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 >> 172.21.15.3:0/1721295402 conn(0x55ae9763e8f0 msgr2=0x55ae9764a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 shutdown_connections 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 1 -- 172.21.15.3:0/1721295402 wait complete. 2024-03-21T08:35:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.218+0000 7fe94adf5740 20 client.16545 trim_cache size 0 max 16384 2024-03-21T08:35:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:35:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:35:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:35:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9280a3100 msgr2=0x7fe9280a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9280a3100 0x7fe9280a55c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe9340ef070 tx=0x7fe9340f56d0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 msgr2=0x7fe94019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe94019a0b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe934000c00 tx=0x7fe93402d890 comp rx=0 tx=0).stop 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 shutdown_connections 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9280a3100 0x7fe9280a55c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe94010def0 0x7fe94019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe940074390 0x7fe94019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 --2- 172.21.15.3:0/584100486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe940073a70 0x7fe94019a0b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 >> 172.21.15.3:0/584100486 conn(0x7fe9400fd120 msgr2=0x7fe9400fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.219+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 shutdown_connections 2024-03-21T08:35:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:30.220+0000 7fe948000640 1 -- 172.21.15.3:0/584100486 wait complete. 2024-03-21T08:35:30.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:30 smithi084 ceph-mon[35477]: pgmap v112: 129 pgs: 129 active+clean; 66 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 744 KiB/s wr, 160 op/s 2024-03-21T08:35:30.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:30 smithi171 ceph-mon[35388]: pgmap v112: 129 pgs: 129 active+clean; 66 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 744 KiB/s wr, 160 op/s 2024-03-21T08:35:30.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:30 smithi003 ceph-mon[31675]: pgmap v112: 129 pgs: 129 active+clean; 66 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 744 KiB/s wr, 160 op/s 2024-03-21T08:35:32.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:32 smithi084 ceph-mon[35477]: pgmap v113: 129 pgs: 129 active+clean; 66 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 741 KiB/s wr, 156 op/s 2024-03-21T08:35:32.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:32 smithi171 ceph-mon[35388]: pgmap v113: 129 pgs: 129 active+clean; 66 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 741 KiB/s wr, 156 op/s 2024-03-21T08:35:32.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:32 smithi003 ceph-mon[31675]: pgmap v113: 129 pgs: 129 active+clean; 66 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 741 KiB/s wr, 156 op/s 2024-03-21T08:35:34.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:34 smithi084 ceph-mon[35477]: pgmap v114: 129 pgs: 129 active+clean; 66 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-03-21T08:35:34.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:34 smithi171 ceph-mon[35388]: pgmap v114: 129 pgs: 129 active+clean; 66 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-03-21T08:35:34.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:34 smithi003 ceph-mon[31675]: pgmap v114: 129 pgs: 129 active+clean; 66 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-03-21T08:35:36.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:36 smithi084 ceph-mon[35477]: pgmap v115: 129 pgs: 129 active+clean; 66 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 699 KiB/s wr, 156 op/s 2024-03-21T08:35:36.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:36 smithi171 ceph-mon[35388]: pgmap v115: 129 pgs: 129 active+clean; 66 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 699 KiB/s wr, 156 op/s 2024-03-21T08:35:36.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:36 smithi003 ceph-mon[31675]: pgmap v115: 129 pgs: 129 active+clean; 66 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 699 KiB/s wr, 156 op/s 2024-03-21T08:35:38.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:38 smithi084 ceph-mon[35477]: pgmap v116: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-03-21T08:35:38.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:38 smithi171 ceph-mon[35388]: pgmap v116: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-03-21T08:35:38.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:38 smithi003 ceph-mon[31675]: pgmap v116: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-03-21T08:35:40.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:40 smithi084 ceph-mon[35477]: pgmap v117: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 829 KiB/s wr, 158 op/s 2024-03-21T08:35:40.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:40 smithi171 ceph-mon[35388]: pgmap v117: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 829 KiB/s wr, 158 op/s 2024-03-21T08:35:40.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:40 smithi003 ceph-mon[31675]: pgmap v117: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 829 KiB/s wr, 158 op/s 2024-03-21T08:35:42.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:42 smithi084 ceph-mon[35477]: pgmap v118: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 822 KiB/s wr, 154 op/s 2024-03-21T08:35:42.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:42 smithi171 ceph-mon[35388]: pgmap v118: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 822 KiB/s wr, 154 op/s 2024-03-21T08:35:42.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:42 smithi003 ceph-mon[31675]: pgmap v118: 129 pgs: 129 active+clean; 71 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 822 KiB/s wr, 154 op/s 2024-03-21T08:35:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.370+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2301806213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 msgr2=0x7f2d70107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.370+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/2301806213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 0x7f2d70107940 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009900 tx=0x7f2d6002bc30 comp rx=0 tx=0).stop 2024-03-21T08:35:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.371+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2301806213 shutdown_connections 2024-03-21T08:35:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.371+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/2301806213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d70101e60 0x7f2d7010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.371+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/2301806213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d701014a0 0x7f2d70101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.371+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/2301806213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 0x7f2d70107940 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.371+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2301806213 >> 172.21.15.3:0/2301806213 conn(0x7f2d700fcfb0 msgr2=0x7f2d700ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.371+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2301806213 shutdown_connections 2024-03-21T08:35:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.371+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2301806213 wait complete. 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.372+0000 7f2d6fe00640 1 Processor -- start 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.372+0000 7f2d6fe00640 1 -- start start 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.372+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d701014a0 0x7f2d70199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.372+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d70101e60 0x7f2d7019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 0x7f2d7019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d70111740 con 0x7f2d70107560 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701115c0 con 0x7f2d701014a0 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701118c0 con 0x7f2d70101e60 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d701014a0 0x7f2d70199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 0x7f2d7019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d701014a0 0x7f2d70199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57480/0 (socket says 172.21.15.3:57480) 2024-03-21T08:35:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d6ea00640 1 -- 172.21.15.3:0/4064619045 learned_addr learned my addr 172.21.15.3:0/4064619045 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.373+0000 7f2d66000640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d70101e60 0x7f2d7019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6f400640 1 -- 172.21.15.3:0/4064619045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d70101e60 msgr2=0x7f2d7019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6f400640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d70101e60 0x7f2d7019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6f400640 1 -- 172.21.15.3:0/4064619045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d701014a0 msgr2=0x7f2d70199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6f400640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d701014a0 0x7f2d70199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6f400640 1 -- 172.21.15.3:0/4064619045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7019eed0 con 0x7f2d70107560 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d701014a0 0x7f2d70199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6f400640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 0x7f2d7019e7a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009fd0 tx=0x7f2d6800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d74a00640 1 -- 172.21.15.3:0/4064619045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800ce20 con 0x7f2d70107560 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7019f1c0 con 0x7f2d70107560 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d74a00640 1 -- 172.21.15.3:0/4064619045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d68005020 con 0x7f2d70107560 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.374+0000 7f2d74a00640 1 -- 172.21.15.3:0/4064619045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68010690 con 0x7f2d70107560 2024-03-21T08:35:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.375+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d701a6a50 con 0x7f2d70107560 2024-03-21T08:35:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.378+0000 7f2d74a00640 1 -- 172.21.15.3:0/4064619045 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f2d680051d0 con 0x7f2d70107560 2024-03-21T08:35:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.378+0000 7f2d78a32740 1 Processor -- start 2024-03-21T08:35:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d78a32740 1 -- start start 2024-03-21T08:35:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad601e0 0x56196ad605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad69390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad698d0 0x56196ad6bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d78a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196ad223b0 con 0x56196ad601e0 2024-03-21T08:35:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d78a32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196acc4830 con 0x56196ad60b00 2024-03-21T08:35:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d78a32740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196ac6d5d0 con 0x56196ad698d0 2024-03-21T08:35:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d66000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad601e0 0x56196ad605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.379+0000 7f2d66000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad601e0 0x56196ad605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38166/0 (socket says 172.21.15.3:38166) 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d6ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad69390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d66000640 1 -- 172.21.15.3:0/1717214869 learned_addr learned my addr 172.21.15.3:0/1717214869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d6ea00640 1 -- 172.21.15.3:0/1717214869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad698d0 msgr2=0x56196ad6bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d66000640 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad698d0 0x56196ad6bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad698d0 0x56196ad6bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d6ea00640 1 -- 172.21.15.3:0/1717214869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad601e0 msgr2=0x56196ad605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad601e0 0x56196ad605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d6ea00640 1 -- 172.21.15.3:0/1717214869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56196ac852c0 con 0x56196ad60b00 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d66000640 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad698d0 0x56196ad6bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.380+0000 7f2d74a00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2d540a3070 0x7f2d540a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.381+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad69390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d600095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2090759855ef6aa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.381+0000 7f2d74a00640 1 -- 172.21.15.3:0/4064619045 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2d680d0340 con 0x7f2d70107560 2024-03-21T08:35:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.381+0000 7f2d66000640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2d540a3070 0x7f2d540a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.381+0000 7f2d5fe00640 1 -- 172.21.15.3:0/1717214869 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60045070 con 0x56196ad60b00 2024-03-21T08:35:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.381+0000 7f2d5fe00640 1 -- 172.21.15.3:0/1717214869 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d6002c780 con 0x56196ad60b00 2024-03-21T08:35:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.381+0000 7f2d5fe00640 1 -- 172.21.15.3:0/1717214869 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60003c50 con 0x56196ad60b00 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 -- 172.21.15.3:0/1717214869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 msgr2=0x56196ad69390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d66000640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2d540a3070 0x7f2d540a5530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005fd0 tx=0x7f2d5800b690 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad69390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d600095e0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 -- 172.21.15.3:0/1717214869 shutdown_connections 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad698d0 0x56196ad6bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad69390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1717214869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad601e0 0x56196ad605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 -- 172.21.15.3:0/1717214869 >> 172.21.15.3:0/1717214869 conn(0x56196ad5ab00 msgr2=0x56196ad5aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 -- 172.21.15.3:0/1717214869 shutdown_connections 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.382+0000 7f2d78a32740 1 -- 172.21.15.3:0/1717214869 wait complete. 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.383+0000 7f2d78a32740 1 -- start start 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.383+0000 7f2d78a32740 1 Processor -- start 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.383+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad66df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad67330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d78a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196ac80a20 con 0x56196ad698d0 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d78a32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196acc08d0 con 0x56196ad60b00 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d78a32740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196ad6c4a0 con 0x56196ad601e0 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d78a32740 10 client.0 _ll_register_callbacks cb 0x56196acbd1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d66000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad66df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d66000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad66df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:57496/0 (socket says 172.21.15.3:57496) 2024-03-21T08:35:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad67330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d66000640 1 -- 172.21.15.3:0/1793610229 learned_addr learned my addr 172.21.15.3:0/1793610229 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6f400640 1 -- 172.21.15.3:0/1793610229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 msgr2=0x56196ad668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6f400640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6f400640 1 -- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 msgr2=0x56196ad66df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6f400640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad66df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6f400640 1 -- 172.21.15.3:0/1793610229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56196a9d9210 con 0x56196ad698d0 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d66000640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad66df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.384+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d6f400640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad67330 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2d680cd5e0 tx=0x7f2d6809caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e701aaeea0f73388 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1793610229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680b5060 con 0x56196ad698d0 2024-03-21T08:35:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1793610229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2d68098920 con 0x56196ad698d0 2024-03-21T08:35:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d6002c780 con 0x56196ad698d0 2024-03-21T08:35:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1793610229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d680a4cc0 con 0x56196ad698d0 2024-03-21T08:35:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d78a32740 10 client.16557 fetch_fsmap learned FSMap version 28 2024-03-21T08:35:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.385+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56196aaa5620 con 0x56196ad698d0 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1793610229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d6002c780 con 0x56196ad698d0 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1793610229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2d68098000 con 0x56196ad698d0 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d78a32740 10 client.16557 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d78a32740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d78a32740 20 client.16557 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196ad79830 0x56196ad7bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d78a32740 4 client.16557 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56196acd1d70 con 0x56196ad79830 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.386+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196ad79830 0x56196ad7bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.387+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196ad79830 0x56196ad7bc20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32708d642ce3c1f1 server_cookie=c62e558fff30c766 in_seq=0 out_seq=0 2024-03-21T08:35:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.387+0000 7f2d5cc00640 10 client.16557 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1793610229 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56196acd1d70 con 0x56196ad79830 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d5cc00640 10 client.16557 handle_command_reply: tid=0 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 client.16557 shutdown 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 msgr2=0x56196ad67330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad67330 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2d680cd5e0 tx=0x7f2d6809caf0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 shutdown_connections 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad67330 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad66df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196ad79830 0x56196ad7bc20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1793610229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 >> 172.21.15.3:0/1793610229 conn(0x56196ad5ab00 msgr2=0x56196ad5c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.389+0000 7f2d5cc00640 0 client.16557 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.390+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 shutdown_connections 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.390+0000 7f2d78a32740 1 -- 172.21.15.3:0/1793610229 wait complete. 2024-03-21T08:35:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.390+0000 7f2d78a32740 20 client.16557 trim_cache size 0 max 16384 2024-03-21T08:35:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.405+0000 7f2d78a32740 1 Processor -- start 2024-03-21T08:35:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.405+0000 7f2d78a32740 1 -- start start 2024-03-21T08:35:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad74bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d78a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701b3070 con 0x56196ad698d0 2024-03-21T08:35:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d78a32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701b0b70 con 0x56196ad60b00 2024-03-21T08:35:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d78a32740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701aec10 con 0x56196ad601e0 2024-03-21T08:35:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d66000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad74bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad750f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38176/0 (socket says 172.21.15.3:38176) 2024-03-21T08:35:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d6f400640 1 -- 172.21.15.3:0/2476012451 learned_addr learned my addr 172.21.15.3:0/2476012451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.406+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d6f400640 1 -- 172.21.15.3:0/2476012451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 msgr2=0x56196ad556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d6f400640 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d6f400640 1 -- 172.21.15.3:0/2476012451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 msgr2=0x56196ad74bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d6f400640 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad74bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d6f400640 1 -- 172.21.15.3:0/2476012451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56196ad5f9b0 con 0x56196ad698d0 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d66000640 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad74bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d6f400640 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad750f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2d68098710 tx=0x7f2d68005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e12f13b0317bb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d5f400640 1 -- 172.21.15.3:0/2476012451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680a4830 con 0x56196ad698d0 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d5f400640 1 -- 172.21.15.3:0/2476012451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d68098000 con 0x56196ad698d0 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.407+0000 7f2d5f400640 1 -- 172.21.15.3:0/2476012451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680b5060 con 0x56196ad698d0 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 -- 172.21.15.3:0/2476012451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 msgr2=0x56196ad750f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad750f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2d68098710 tx=0x7f2d68005bf0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 -- 172.21.15.3:0/2476012451 shutdown_connections 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad750f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad74bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 --2- 172.21.15.3:0/2476012451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 -- 172.21.15.3:0/2476012451 >> 172.21.15.3:0/2476012451 conn(0x56196ad5ab00 msgr2=0x56196ad5aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 -- 172.21.15.3:0/2476012451 shutdown_connections 2024-03-21T08:35:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.408+0000 7f2d78a32740 1 -- 172.21.15.3:0/2476012451 wait complete. 2024-03-21T08:35:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 -- start start 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 Processor -- start 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad63200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad63740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad63c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701af210 con 0x56196ad698d0 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701b0510 con 0x56196ad60b00 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701af850 con 0x56196ad601e0 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.409+0000 7f2d78a32740 10 client.0 _ll_register_callbacks cb 0x7f2d68005810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad63c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad63c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38178/0 (socket says 172.21.15.3:38178) 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 -- 172.21.15.3:0/1041766607 learned_addr learned my addr 172.21.15.3:0/1041766607 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d66000640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad63740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad63200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 -- 172.21.15.3:0/1041766607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 msgr2=0x56196ad63200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad63200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 -- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 msgr2=0x56196ad63740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad63740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 -- 172.21.15.3:0/1041766607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56196add4250 con 0x56196ad698d0 2024-03-21T08:35:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad63200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:35:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d66000640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad63740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d6f400640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad63c80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2d6801b000 tx=0x7f2d680a4a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc05a613dfe8c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.410+0000 7f2d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1041766607 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680b9070 con 0x56196ad698d0 2024-03-21T08:35:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1041766607 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f2d6809b910 con 0x56196ad698d0 2024-03-21T08:35:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d68098000 con 0x56196ad698d0 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1041766607 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d68083050 con 0x56196ad698d0 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d78a32740 10 client.16563 fetch_fsmap learned FSMap version 28 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56196add5560 con 0x56196ad698d0 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1041766607 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d68098000 con 0x56196ad698d0 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1041766607 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2d680b5060 con 0x56196ad698d0 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.411+0000 7f2d78a32740 10 client.16563 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.412+0000 7f2d78a32740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.412+0000 7f2d78a32740 20 client.16563 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.412+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196add5a90 0x56196add7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.412+0000 7f2d78a32740 4 client.16563 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:35:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.412+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56196accc0c0 con 0x56196add5a90 2024-03-21T08:35:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.412+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196add5a90 0x56196add7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.412+0000 7f2d6ea00640 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196add5a90 0x56196add7eb0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b781ae96688671 server_cookie=208c95a159385c1 in_seq=0 out_seq=0 2024-03-21T08:35:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.413+0000 7f2d5cc00640 10 client.16563 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d5cc00640 1 -- 172.21.15.3:0/1041766607 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56196accc0c0 con 0x56196add5a90 2024-03-21T08:35:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d5cc00640 10 client.16563 handle_command_reply: tid=0 2024-03-21T08:35:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 client.16563 shutdown 2024-03-21T08:35:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 msgr2=0x56196ad63c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad63c80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2d6801b000 tx=0x7f2d680a4a60 comp rx=0 tx=0).stop 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 shutdown_connections 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56196ad698d0 0x56196ad63c80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56196ad60b00 0x56196ad63740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x56196add5a90 0x56196add7eb0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d78a32740 1 --2- 172.21.15.3:0/1041766607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56196ad601e0 0x56196ad63200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.469+0000 7f2d5cc00640 0 client.16563 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.470+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 >> 172.21.15.3:0/1041766607 conn(0x56196ad5ab00 msgr2=0x56196ad556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.470+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 shutdown_connections 2024-03-21T08:35:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.470+0000 7f2d78a32740 1 -- 172.21.15.3:0/1041766607 wait complete. 2024-03-21T08:35:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.470+0000 7f2d78a32740 20 client.16563 trim_cache size 0 max 16384 2024-03-21T08:35:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:35:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:35:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:35:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:35:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.471+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2d540a3070 msgr2=0x7f2d540a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.471+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2d540a3070 0x7f2d540a5530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005fd0 tx=0x7f2d5800b690 comp rx=0 tx=0).stop 2024-03-21T08:35:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.471+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 msgr2=0x7f2d7019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.471+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 0x7f2d7019e7a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009fd0 tx=0x7f2d6800ef10 comp rx=0 tx=0).stop 2024-03-21T08:35:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 shutdown_connections 2024-03-21T08:35:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70107560 0x7f2d7019e7a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2d540a3070 0x7f2d540a5530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d70101e60 0x7f2d7019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/4064619045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d701014a0 0x7f2d70199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 >> 172.21.15.3:0/4064619045 conn(0x7f2d700fcfb0 msgr2=0x7f2d7010d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 shutdown_connections 2024-03-21T08:35:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:43.472+0000 7f2d6fe00640 1 -- 172.21.15.3:0/4064619045 wait complete. 2024-03-21T08:35:44.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:44 smithi084 ceph-mon[35477]: pgmap v119: 129 pgs: 129 active+clean; 77 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-21T08:35:44.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:44 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:44.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:44 smithi171 ceph-mon[35388]: pgmap v119: 129 pgs: 129 active+clean; 77 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-21T08:35:44.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:44 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:44.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:44 smithi003 ceph-mon[31675]: pgmap v119: 129 pgs: 129 active+clean; 77 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-21T08:35:44.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:44 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:46.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:46 smithi084 ceph-mon[35477]: pgmap v120: 129 pgs: 129 active+clean; 77 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1006 KiB/s wr, 125 op/s 2024-03-21T08:35:46.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:46 smithi171 ceph-mon[35388]: pgmap v120: 129 pgs: 129 active+clean; 77 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1006 KiB/s wr, 125 op/s 2024-03-21T08:35:46.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:46 smithi003 ceph-mon[31675]: pgmap v120: 129 pgs: 129 active+clean; 77 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1006 KiB/s wr, 125 op/s 2024-03-21T08:35:48.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:48 smithi084 ceph-mon[35477]: pgmap v121: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-03-21T08:35:48.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:48 smithi171 ceph-mon[35388]: pgmap v121: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-03-21T08:35:48.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:48 smithi003 ceph-mon[31675]: pgmap v121: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-03-21T08:35:50.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:50 smithi084 ceph-mon[35477]: pgmap v122: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:35:50.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:50 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:50.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:50 smithi171 ceph-mon[35388]: pgmap v122: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:35:50.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:50 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:50.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:50 smithi003 ceph-mon[31675]: pgmap v122: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:35:50.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:50 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:52.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:52 smithi084 ceph-mon[35477]: pgmap v123: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:35:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:52 smithi171 ceph-mon[35388]: pgmap v123: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:35:52.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:52 smithi003 ceph-mon[31675]: pgmap v123: 129 pgs: 129 active+clean; 85 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:35:54.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:54 smithi084 ceph-mon[35477]: pgmap v124: 129 pgs: 129 active+clean; 92 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-03-21T08:35:54.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:54 smithi171 ceph-mon[35388]: pgmap v124: 129 pgs: 129 active+clean; 92 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-03-21T08:35:54.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:54 smithi003 ceph-mon[31675]: pgmap v124: 129 pgs: 129 active+clean; 92 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-03-21T08:35:55.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:55 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:55.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:55 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:55.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:55 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:35:56.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:56 smithi171 ceph-mon[35388]: pgmap v125: 129 pgs: 129 active+clean; 92 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-03-21T08:35:56.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:56 smithi003 ceph-mon[31675]: pgmap v125: 129 pgs: 129 active+clean; 92 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-03-21T08:35:56.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:56 smithi084 ceph-mon[35477]: pgmap v125: 129 pgs: 129 active+clean; 92 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-03-21T08:35:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 -- 172.21.15.3:0/1865618357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 msgr2=0x7f1098111ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 --2- 172.21.15.3:0/1865618357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 0x7f1098111ff0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1080009910 tx=0x7f108002bc20 comp rx=0 tx=0).stop 2024-03-21T08:35:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 -- 172.21.15.3:0/1865618357 shutdown_connections 2024-03-21T08:35:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 --2- 172.21.15.3:0/1865618357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098112530 0x7f1098114920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 --2- 172.21.15.3:0/1865618357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 0x7f1098111ff0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 --2- 172.21.15.3:0/1865618357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1098075bd0 0x7f1098075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 -- 172.21.15.3:0/1865618357 >> 172.21.15.3:0/1865618357 conn(0x7f10980ff320 msgr2=0x7f1098101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 -- 172.21.15.3:0/1865618357 shutdown_connections 2024-03-21T08:35:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.628+0000 7f109e200640 1 -- 172.21.15.3:0/1865618357 wait complete. 2024-03-21T08:35:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.629+0000 7f109e200640 1 Processor -- start 2024-03-21T08:35:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f109e200640 1 -- start start 2024-03-21T08:35:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f109e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098075bd0 0x7f109819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f109e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 0x7f109819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f109e200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1098112530 0x7f10981a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f109e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098117f40 con 0x7f10980764f0 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f109e200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098117dc0 con 0x7f1098075bd0 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f109e200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981180c0 con 0x7f1098112530 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f1097400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098075bd0 0x7f109819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f1096a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 0x7f109819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f1097400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098075bd0 0x7f109819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:47216/0 (socket says 172.21.15.3:47216) 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.630+0000 7f1097400640 1 -- 172.21.15.3:0/382037997 learned_addr learned my addr 172.21.15.3:0/382037997 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1097e00640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1098112530 0x7f10981a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1097400640 1 -- 172.21.15.3:0/382037997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1098112530 msgr2=0x7f10981a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1097400640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1098112530 0x7f10981a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1097400640 1 -- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 msgr2=0x7f109819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1097400640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 0x7f109819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1097400640 1 -- 172.21.15.3:0/382037997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10981a3550 con 0x7f1098075bd0 2024-03-21T08:35:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1096a00640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 0x7f109819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.631+0000 7f1097400640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098075bd0 0x7f109819e560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f108800ea80 tx=0x7f108800ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.632+0000 7f108fe00640 1 -- 172.21.15.3:0/382037997 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108800ce40 con 0x7f1098075bd0 2024-03-21T08:35:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.632+0000 7f108fe00640 1 -- 172.21.15.3:0/382037997 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1088005020 con 0x7f1098075bd0 2024-03-21T08:35:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.632+0000 7f108fe00640 1 -- 172.21.15.3:0/382037997 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880116d0 con 0x7f1098075bd0 2024-03-21T08:35:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.632+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10981a3840 con 0x7f1098075bd0 2024-03-21T08:35:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.633+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10981ab0d0 con 0x7f1098075bd0 2024-03-21T08:35:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 Processor -- start 2024-03-21T08:35:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 -- start start 2024-03-21T08:35:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f108fe00640 1 -- 172.21.15.3:0/382037997 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f10880051d0 con 0x7f1098075bd0 2024-03-21T08:35:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e0e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e19ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948dd03b0 con 0x55b948e0dfd0 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948d72830 con 0x55b948e0e8f0 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.637+0000 7f10a0e50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948d1b5d0 con 0x55b948e176c0 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e0e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e17180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:47222/0 (socket says 172.21.15.3:47222) 2024-03-21T08:35:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 -- 172.21.15.3:0/3606683041 learned_addr learned my addr 172.21.15.3:0/3606683041 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1096a00640 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e19ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 -- 172.21.15.3:0/3606683041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 msgr2=0x55b948e19ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e19ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 -- 172.21.15.3:0/3606683041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 msgr2=0x55b948e0e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e0e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097400640 1 -- 172.21.15.3:0/3606683041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b948d332c0 con 0x55b948e0e8f0 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.638+0000 7f1097e00640 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e0e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.639+0000 7f1097400640 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e17180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f10880cd000 tx=0x7f10880d8830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97a2b81a99412e3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.639+0000 7f108fe00640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f107c0a3250 0x7f107c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.639+0000 7f1087e00640 1 -- 172.21.15.3:0/3606683041 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108809bb30 con 0x55b948e0e8f0 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.639+0000 7f1087e00640 1 -- 172.21.15.3:0/3606683041 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f108809c460 con 0x55b948e0e8f0 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.639+0000 7f1087e00640 1 -- 172.21.15.3:0/3606683041 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880978d0 con 0x55b948e0e8f0 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.639+0000 7f108fe00640 1 -- 172.21.15.3:0/382037997 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f10880cfdf0 con 0x7f1098075bd0 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.639+0000 7f1097e00640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f107c0a3250 0x7f107c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 -- 172.21.15.3:0/3606683041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 msgr2=0x55b948e17180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e17180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f10880cd000 tx=0x7f10880d8830 comp rx=0 tx=0).stop 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 -- 172.21.15.3:0/3606683041 shutdown_connections 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e19ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e17180 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3606683041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e0e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 -- 172.21.15.3:0/3606683041 >> 172.21.15.3:0/3606683041 conn(0x55b948e088f0 msgr2=0x55b948e08cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f1097e00640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f107c0a3250 0x7f107c0a5710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f109806aa00 tx=0x7f1090005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 -- 172.21.15.3:0/3606683041 shutdown_connections 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.640+0000 7f10a0e50740 1 -- 172.21.15.3:0/3606683041 wait complete. 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 -- start start 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 Processor -- start 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e14be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e15120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948d2ea20 con 0x55b948e0dfd0 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948d6e8d0 con 0x55b948e0e8f0 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b948e1a2f0 con 0x55b948e176c0 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.641+0000 7f10a0e50740 10 client.0 _ll_register_callbacks cb 0x55b948d6b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1097400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1096a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e14be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1097400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54006/0 (socket says 172.21.15.3:54006) 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1096a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e14be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:47224/0 (socket says 172.21.15.3:47224) 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1097400640 1 -- 172.21.15.3:0/489080697 learned_addr learned my addr 172.21.15.3:0/489080697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1097e00640 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e15120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1096a00640 1 -- 172.21.15.3:0/489080697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 msgr2=0x55b948e15120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1096a00640 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e15120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1096a00640 1 -- 172.21.15.3:0/489080697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 msgr2=0x55b948e146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1096a00640 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1096a00640 1 -- 172.21.15.3:0/489080697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b948a87210 con 0x55b948e0e8f0 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.642+0000 7f1097400640 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f1096a00640 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e14be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1080005a50 tx=0x7f1080005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42649bf0ad3545 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f1084c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:35:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f1084c00640 1 -- 172.21.15.3:0/489080697 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080043070 con 0x55b948e0e8f0 2024-03-21T08:35:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f1084c00640 1 -- 172.21.15.3:0/489080697 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f1080004250 con 0x55b948e0e8f0 2024-03-21T08:35:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f108809c460 con 0x55b948e0e8f0 2024-03-21T08:35:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f1084c00640 1 -- 172.21.15.3:0/489080697 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108004f050 con 0x55b948e0e8f0 2024-03-21T08:35:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f10a0e50740 10 client.36311 fetch_fsmap learned FSMap version 30 2024-03-21T08:35:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.643+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b948b53620 con 0x55b948e0e8f0 2024-03-21T08:35:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.644+0000 7f1084c00640 1 -- 172.21.15.3:0/489080697 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f108809c460 con 0x55b948e0e8f0 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.644+0000 7f1084c00640 1 -- 172.21.15.3:0/489080697 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f108002ca50 con 0x55b948e0e8f0 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.644+0000 7f10a0e50740 10 client.36311 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.644+0000 7f10a0e50740 10 client.36311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.645+0000 7f10a0e50740 20 client.36311 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.645+0000 7f10a0e50740 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e27620 0x55b948e29a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.645+0000 7f10a0e50740 4 client.36311 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.645+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b948d7fd70 con 0x55b948e27620 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.645+0000 7f1097400640 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e27620 0x55b948e29a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.646+0000 7f1097400640 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e27620 0x55b948e29a30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b2646bb6f4a73b server_cookie=81f67e0533bf4d4c in_seq=0 out_seq=0 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.646+0000 7f1084c00640 10 client.36311 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f1084c00640 1 -- 172.21.15.3:0/489080697 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b948d7fd70 con 0x55b948e27620 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f1084c00640 10 client.36311 handle_command_reply: tid=0 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 client.36311 shutdown 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 msgr2=0x55b948e14be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e14be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1080005a50 tx=0x7f1080005cf0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 shutdown_connections 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e27620 0x55b948e29a30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e176c0 0x55b948e15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e0e8f0 0x55b948e14be0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 --2- 172.21.15.3:0/489080697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f1084c00640 0 client.36311 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 >> 172.21.15.3:0/489080697 conn(0x55b948e088f0 msgr2=0x55b948e0a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 shutdown_connections 2024-03-21T08:35:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 1 -- 172.21.15.3:0/489080697 wait complete. 2024-03-21T08:35:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.648+0000 7f10a0e50740 20 client.36311 trim_cache size 0 max 16384 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 Processor -- start 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 -- start start 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0dfd0 0x55b948e16dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0e8f0 0x55b948e07950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e22a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981b77c0 con 0x55b948e0e8f0 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981b5260 con 0x55b948e176c0 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f10a0e50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981b3300 con 0x55b948e0dfd0 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f1096a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0e8f0 0x55b948e07950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f1097e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e22a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.665+0000 7f1096a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0e8f0 0x55b948e07950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54022/0 (socket says 172.21.15.3:54022) 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1096a00640 1 -- 172.21.15.3:0/3751168010 learned_addr learned my addr 172.21.15.3:0/3751168010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1097400640 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0dfd0 0x55b948e16dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1096a00640 1 -- 172.21.15.3:0/3751168010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0dfd0 msgr2=0x55b948e16dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1096a00640 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0dfd0 0x55b948e16dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1096a00640 1 -- 172.21.15.3:0/3751168010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 msgr2=0x55b948e22a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1096a00640 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e22a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1096a00640 1 -- 172.21.15.3:0/3751168010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b948e13650 con 0x55b948e0e8f0 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1097e00640 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e22a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:35:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1096a00640 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0e8f0 0x55b948e07950 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1080005de0 tx=0x7f108002bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975f1eb96e3bbee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.666+0000 7f1087400640 1 -- 172.21.15.3:0/3751168010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080043070 con 0x55b948e0e8f0 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f1087400640 1 -- 172.21.15.3:0/3751168010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1080034850 con 0x55b948e0e8f0 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f1087400640 1 -- 172.21.15.3:0/3751168010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080039040 con 0x55b948e0e8f0 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f10a0e50740 1 -- 172.21.15.3:0/3751168010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0e8f0 msgr2=0x55b948e07950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0e8f0 0x55b948e07950 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1080005de0 tx=0x7f108002bc20 comp rx=0 tx=0).stop 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f10a0e50740 1 -- 172.21.15.3:0/3751168010 shutdown_connections 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e22a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0e8f0 0x55b948e07950 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f10a0e50740 1 --2- 172.21.15.3:0/3751168010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0dfd0 0x55b948e16dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.667+0000 7f10a0e50740 1 -- 172.21.15.3:0/3751168010 >> 172.21.15.3:0/3751168010 conn(0x55b948e088f0 msgr2=0x55b948e08cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.668+0000 7f10a0e50740 1 -- 172.21.15.3:0/3751168010 shutdown_connections 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.668+0000 7f10a0e50740 1 -- 172.21.15.3:0/3751168010 wait complete. 2024-03-21T08:35:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.668+0000 7f10a0e50740 1 -- start start 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.668+0000 7f10a0e50740 1 Processor -- start 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e1ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0e8f0 0x55b948e1f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f10a0e50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f10a0e50740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981b3900 con 0x55b948e0dfd0 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f10a0e50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981b4c00 con 0x55b948e176c0 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f10a0e50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981b3f40 con 0x55b948e0e8f0 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f10a0e50740 10 client.0 _ll_register_callbacks cb 0x7f1090005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e1ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e1ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54024/0 (socket says 172.21.15.3:54024) 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 -- 172.21.15.3:0/1153060544 learned_addr learned my addr 172.21.15.3:0/1153060544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097e00640 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1096a00640 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0e8f0 0x55b948e1f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 -- 172.21.15.3:0/1153060544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0e8f0 msgr2=0x55b948e1f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0e8f0 0x55b948e1f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 -- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 msgr2=0x55b948e114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.669+0000 7f1097400640 1 -- 172.21.15.3:0/1153060544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b948e11c10 con 0x55b948e0dfd0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f1097400640 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e1ebb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1088009270 tx=0x7f1088063050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0e3ffcdce0c9ca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f1084c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f1084c00640 1 -- 172.21.15.3:0/1153060544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108801f040 con 0x55b948e0dfd0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b948d7a0c0 con 0x55b948e0dfd0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f1084c00640 1 -- 172.21.15.3:0/1153060544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f10880d3510 con 0x55b948e0dfd0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f1084c00640 1 -- 172.21.15.3:0/1153060544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1088097ab0 con 0x55b948e0dfd0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f10a0e50740 10 client.16581 fetch_fsmap learned FSMap version 30 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b948e83980 con 0x55b948e0dfd0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.670+0000 7f1084c00640 1 -- 172.21.15.3:0/1153060544 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b948d7a0c0 con 0x55b948e0dfd0 2024-03-21T08:35:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f1084c00640 1 -- 172.21.15.3:0/1153060544 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f10880df070 con 0x55b948e0dfd0 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f10a0e50740 10 client.16581 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f10a0e50740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f10a0e50740 20 client.16581 populate_metadata read hostname 'smithi003' 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f10a0e50740 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e83c10 0x55b948e86000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f10a0e50740 4 client.16581 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b948cf85d0 con 0x55b948e83c10 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.671+0000 7f1096a00640 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e83c10 0x55b948e86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:35:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.674+0000 7f1096a00640 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e83c10 0x55b948e86000 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5ea9ddd04c8088 server_cookie=b0e5679e06589050 in_seq=0 out_seq=0 2024-03-21T08:35:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.674+0000 7f1084c00640 10 client.16581 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.801+0000 7f1084c00640 1 -- 172.21.15.3:0/1153060544 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b948cf85d0 con 0x55b948e83c10 2024-03-21T08:35:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.801+0000 7f1084c00640 10 client.16581 handle_command_reply: tid=0 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 client.16581 shutdown 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 msgr2=0x55b948e1ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e1ebb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1088009270 tx=0x7f1088063050 comp rx=0 tx=0).stop 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 shutdown_connections 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b948e176c0 0x55b948e114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55b948e83c10 0x55b948e86000 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b948e0e8f0 0x55b948e1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 --2- 172.21.15.3:0/1153060544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b948e0dfd0 0x55b948e1ebb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f1084c00640 0 client.16581 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 >> 172.21.15.3:0/1153060544 conn(0x55b948e088f0 msgr2=0x55b948e14ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 shutdown_connections 2024-03-21T08:35:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 1 -- 172.21.15.3:0/1153060544 wait complete. 2024-03-21T08:35:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f10a0e50740 20 client.16581 trim_cache size 0 max 16384 2024-03-21T08:35:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:35:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:35:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:35:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:35:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f107c0a3250 msgr2=0x7f107c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f107c0a3250 0x7f107c0a5710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f109806aa00 tx=0x7f1090005f90 comp rx=0 tx=0).stop 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098075bd0 msgr2=0x7f109819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098075bd0 0x7f109819e560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f108800ea80 tx=0x7f108800ef40 comp rx=0 tx=0).stop 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 shutdown_connections 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f107c0a3250 0x7f107c0a5710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1098112530 0x7f10981a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10980764f0 0x7f109819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 --2- 172.21.15.3:0/382037997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1098075bd0 0x7f109819e560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 >> 172.21.15.3:0/382037997 conn(0x7f10980ff320 msgr2=0x7f1098113d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 shutdown_connections 2024-03-21T08:35:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:35:56.802+0000 7f109e200640 1 -- 172.21.15.3:0/382037997 wait complete. 2024-03-21T08:35:58.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:35:58 smithi003 ceph-mon[31675]: pgmap v126: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-03-21T08:35:58.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:35:58 smithi084 ceph-mon[35477]: pgmap v126: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-03-21T08:35:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:35:58 smithi171 ceph-mon[35388]: pgmap v126: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-03-21T08:36:00.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:00 smithi003 ceph-mon[31675]: pgmap v127: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:36:00.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:00 smithi084 ceph-mon[35477]: pgmap v127: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:36:00.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:00 smithi171 ceph-mon[35388]: pgmap v127: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:36:02.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:02 smithi003 ceph-mon[31675]: pgmap v128: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:36:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:02 smithi084 ceph-mon[35477]: pgmap v128: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:36:02.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:02 smithi171 ceph-mon[35388]: pgmap v128: 129 pgs: 129 active+clean; 99 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-21T08:36:04.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:04 smithi003 ceph-mon[31675]: pgmap v129: 129 pgs: 129 active+clean; 102 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-03-21T08:36:04.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:04 smithi084 ceph-mon[35477]: pgmap v129: 129 pgs: 129 active+clean; 102 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-03-21T08:36:04.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:04 smithi171 ceph-mon[35388]: pgmap v129: 129 pgs: 129 active+clean; 102 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-03-21T08:36:06.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:06 smithi003 ceph-mon[31675]: pgmap v130: 129 pgs: 129 active+clean; 102 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-03-21T08:36:06.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:06 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:36:06.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:06 smithi084 ceph-mon[35477]: pgmap v130: 129 pgs: 129 active+clean; 102 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-03-21T08:36:06.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:06 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:36:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:06 smithi171 ceph-mon[35388]: pgmap v130: 129 pgs: 129 active+clean; 102 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-03-21T08:36:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:06 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:36:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:07 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:36:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:07 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:36:07.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:07 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:36:07.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:07 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:36:07.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:07 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:36:07.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:07 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:36:07.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:07 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:36:07.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:07 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:36:07.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:07 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:36:08.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:08 smithi003 ceph-mon[31675]: pgmap v131: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-03-21T08:36:08.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:08 smithi084 ceph-mon[35477]: pgmap v131: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-03-21T08:36:08.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:08 smithi171 ceph-mon[35388]: pgmap v131: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-03-21T08:36:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 -- 172.21.15.3:0/2675498201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40074360 msgr2=0x7f1a40112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 --2- 172.21.15.3:0/2675498201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40074360 0x7f1a40112120 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009860 tx=0x7f1a2802bc20 comp rx=0 tx=0).stop 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 -- 172.21.15.3:0/2675498201 shutdown_connections 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 --2- 172.21.15.3:0/2675498201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a40112660 0x7f1a40114a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 --2- 172.21.15.3:0/2675498201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40074360 0x7f1a40112120 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 --2- 172.21.15.3:0/2675498201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a40073a40 0x7f1a40073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 -- 172.21.15.3:0/2675498201 >> 172.21.15.3:0/2675498201 conn(0x7f1a400fd120 msgr2=0x7f1a400ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 -- 172.21.15.3:0/2675498201 shutdown_connections 2024-03-21T08:36:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.968+0000 7f1a45200640 1 -- 172.21.15.3:0/2675498201 wait complete. 2024-03-21T08:36:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.969+0000 7f1a45200640 1 Processor -- start 2024-03-21T08:36:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a45200640 1 -- start start 2024-03-21T08:36:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a45200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a40073a40 0x7f1a4019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a45200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40112660 0x7f1a4019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a45200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a401a2fd0 0x7f1a401a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a45200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40117f10 con 0x7f1a40112660 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a45200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40117d90 con 0x7f1a401a2fd0 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a45200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40118090 con 0x7f1a40073a40 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a3e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a401a2fd0 0x7f1a401a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40112660 0x7f1a4019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a3e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a401a2fd0 0x7f1a401a3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55148/0 (socket says 172.21.15.3:55148) 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.970+0000 7f1a3e800640 1 -- 172.21.15.3:0/1428516513 learned_addr learned my addr 172.21.15.3:0/1428516513 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a3de00640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a40073a40 0x7f1a4019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a3e800640 1 -- 172.21.15.3:0/1428516513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a40073a40 msgr2=0x7f1a4019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a3e800640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a40073a40 0x7f1a4019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a3e800640 1 -- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40112660 msgr2=0x7f1a4019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a3e800640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40112660 0x7f1a4019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a3e800640 1 -- 172.21.15.3:0/1428516513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a401a3ac0 con 0x7f1a401a2fd0 2024-03-21T08:36:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a3e800640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a401a2fd0 0x7f1a401a3480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a380058c0 tx=0x7f1a3800edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.971+0000 7f1a36a00640 1 -- 172.21.15.3:0/1428516513 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800cd80 con 0x7f1a401a2fd0 2024-03-21T08:36:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.972+0000 7f1a36a00640 1 -- 172.21.15.3:0/1428516513 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a380035f0 con 0x7f1a401a2fd0 2024-03-21T08:36:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.972+0000 7f1a36a00640 1 -- 172.21.15.3:0/1428516513 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a380044e0 con 0x7f1a401a2fd0 2024-03-21T08:36:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.972+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a401a3db0 con 0x7f1a401a2fd0 2024-03-21T08:36:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.972+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a401ab520 con 0x7f1a401a2fd0 2024-03-21T08:36:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.975+0000 7f1a47f86740 1 Processor -- start 2024-03-21T08:36:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a36a00640 1 -- 172.21.15.3:0/1428516513 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f1a380037a0 con 0x7f1a401a2fd0 2024-03-21T08:36:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a47f86740 1 -- start start 2024-03-21T08:36:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167f1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167f16c0 0x560c167f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a47f86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c167aa3b0 con 0x560c167f16c0 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a47f86740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1674c830 con 0x560c167e7fd0 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.976+0000 7f1a47f86740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c166f55d0 con 0x560c167e88f0 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167e83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55162/0 (socket says 172.21.15.3:55162) 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 -- 172.21.15.3:0/955851640 learned_addr learned my addr 172.21.15.3:0/955851640 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3de00640 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167f16c0 0x560c167f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a36a00640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f1a240a3100 0x7f1a240a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a36a00640 1 -- 172.21.15.3:0/1428516513 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f1a380cfe40 con 0x7f1a401a2fd0 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 -- 172.21.15.3:0/955851640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 msgr2=0x560c167f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167f1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 -- 172.21.15.3:0/955851640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167f16c0 msgr2=0x560c167f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167f16c0 0x560c167f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.977+0000 7f1a3d400640 1 -- 172.21.15.3:0/955851640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c1670d2c0 con 0x560c167e7fd0 2024-03-21T08:36:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.978+0000 7f1a3e800640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f1a240a3100 0x7f1a240a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.978+0000 7f1a3d400640 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167e83b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1a280057c0 tx=0x7f1a28005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd2557dd68f9302 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.978+0000 7f1a2f400640 1 -- 172.21.15.3:0/955851640 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28049070 con 0x560c167e7fd0 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.978+0000 7f1a3e800640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f1a240a3100 0x7f1a240a55c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009230 tx=0x7f1a38004730 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.978+0000 7f1a2f400640 1 -- 172.21.15.3:0/955851640 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a28034780 con 0x560c167e7fd0 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.978+0000 7f1a2f400640 1 -- 172.21.15.3:0/955851640 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28039040 con 0x560c167e7fd0 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 -- 172.21.15.3:0/955851640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 msgr2=0x560c167e83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167e83b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1a280057c0 tx=0x7f1a28005d40 comp rx=0 tx=0).stop 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 -- 172.21.15.3:0/955851640 shutdown_connections 2024-03-21T08:36:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167f16c0 0x560c167f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 --2- 172.21.15.3:0/955851640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167e83b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 -- 172.21.15.3:0/955851640 >> 172.21.15.3:0/955851640 conn(0x560c167e28f0 msgr2=0x560c167e2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 -- 172.21.15.3:0/955851640 shutdown_connections 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.979+0000 7f1a47f86740 1 -- 172.21.15.3:0/955851640 wait complete. 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.980+0000 7f1a47f86740 1 -- start start 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.980+0000 7f1a47f86740 1 Processor -- start 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.980+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.980+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.980+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 0x560c167ef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a47f86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c16708a20 con 0x560c167e88f0 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a47f86740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c167488d0 con 0x560c167e7fd0 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a47f86740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c167f4290 con 0x560c167f16c0 2024-03-21T08:36:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a47f86740 10 client.0 _ll_register_callbacks cb 0x560c167451c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40426/0 (socket says 172.21.15.3:40426) 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 -- 172.21.15.3:0/1839959031 learned_addr learned my addr 172.21.15.3:0/1839959031 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3d400640 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 -- 172.21.15.3:0/1839959031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 msgr2=0x560c167ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 0x560c167ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 -- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 msgr2=0x560c167ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.981+0000 7f1a3de00640 1 -- 172.21.15.3:0/1839959031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c16461210 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.982+0000 7f1a3de00640 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eecf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012cc0 tx=0x7f1a3000c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac461635a9ccd9e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.982+0000 7f1a23e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.982+0000 7f1a23e00640 1 -- 172.21.15.3:0/1839959031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3001b440 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.982+0000 7f1a23e00640 1 -- 172.21.15.3:0/1839959031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f1a3001a3e0 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.982+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a28034780 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a23e00640 1 -- 172.21.15.3:0/1839959031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a3001b5e0 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 10 client.16593 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560c1652d620 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a23e00640 1 -- 172.21.15.3:0/1839959031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a28034780 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a23e00640 1 -- 172.21.15.3:0/1839959031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1a30019070 con 0x560c167e88f0 2024-03-21T08:36:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 10 client.16593 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 20 client.16593 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c168017d0 0x560c16803c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 4 client.16593 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.983+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c16759d70 con 0x560c168017d0 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.984+0000 7f1a3d400640 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c168017d0 0x560c16803c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.986+0000 7f1a3d400640 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c168017d0 0x560c16803c10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f7c441310f64a0 server_cookie=ae8d94e1681f2dec in_seq=0 out_seq=0 2024-03-21T08:36:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.986+0000 7f1a23e00640 10 client.16593 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.987+0000 7f1a23e00640 1 -- 172.21.15.3:0/1839959031 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560c16759d70 con 0x560c168017d0 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a23e00640 10 client.16593 handle_command_reply: tid=0 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 client.16593 shutdown 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 msgr2=0x560c167eecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eecf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012cc0 tx=0x7f1a3000c700 comp rx=0 tx=0).stop 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 shutdown_connections 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c168017d0 0x560c16803c10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 0x560c167ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eecf0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 --2- 172.21.15.3:0/1839959031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 >> 172.21.15.3:0/1839959031 conn(0x560c167e28f0 msgr2=0x560c167e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a23e00640 0 client.16593 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.988+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 shutdown_connections 2024-03-21T08:36:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.989+0000 7f1a47f86740 1 -- 172.21.15.3:0/1839959031 wait complete. 2024-03-21T08:36:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:09.989+0000 7f1a47f86740 20 client.16593 trim_cache size 0 max 16384 2024-03-21T08:36:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.004+0000 7f1a47f86740 1 Processor -- start 2024-03-21T08:36:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.004+0000 7f1a47f86740 1 -- start start 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e7fd0 0x560c167f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167fc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167f16c0 0x560c167fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a47f86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401b7d50 con 0x560c167e7fd0 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a47f86740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401b57f0 con 0x560c167f16c0 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a47f86740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401b3890 con 0x560c167e88f0 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e7fd0 0x560c167f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e7fd0 0x560c167f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40430/0 (socket says 172.21.15.3:40430) 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a3d400640 1 -- 172.21.15.3:0/2508413732 learned_addr learned my addr 172.21.15.3:0/2508413732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.005+0000 7f1a3d400640 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167f16c0 0x560c167fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a3de00640 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167fc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a3d400640 1 -- 172.21.15.3:0/2508413732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 msgr2=0x560c167fc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a3d400640 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167fc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a3d400640 1 -- 172.21.15.3:0/2508413732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167f16c0 msgr2=0x560c167fcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a3d400640 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167f16c0 0x560c167fcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a3d400640 1 -- 172.21.15.3:0/2508413732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c167e7790 con 0x560c167e7fd0 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a3d400640 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e7fd0 0x560c167f0ee0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009860 tx=0x7f1a28035ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1a5665cdd4e4ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a2ea00640 1 -- 172.21.15.3:0/2508413732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28055070 con 0x560c167e7fd0 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a2ea00640 1 -- 172.21.15.3:0/2508413732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a28034780 con 0x560c167e7fd0 2024-03-21T08:36:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.006+0000 7f1a2ea00640 1 -- 172.21.15.3:0/2508413732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28039040 con 0x560c167e7fd0 2024-03-21T08:36:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 -- 172.21.15.3:0/2508413732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e7fd0 msgr2=0x560c167f0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e7fd0 0x560c167f0ee0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009860 tx=0x7f1a28035ea0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 -- 172.21.15.3:0/2508413732 shutdown_connections 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167f16c0 0x560c167fcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167e88f0 0x560c167fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 --2- 172.21.15.3:0/2508413732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e7fd0 0x560c167f0ee0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 -- 172.21.15.3:0/2508413732 >> 172.21.15.3:0/2508413732 conn(0x560c167e28f0 msgr2=0x560c167ed080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 -- 172.21.15.3:0/2508413732 shutdown_connections 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.007+0000 7f1a47f86740 1 -- 172.21.15.3:0/2508413732 wait complete. 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 -- start start 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 Processor -- start 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 0x560c167eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401b3e90 con 0x560c167e88f0 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401b5190 con 0x560c167e7fd0 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401b44d0 con 0x560c167f16c0 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.008+0000 7f1a47f86740 10 client.0 _ll_register_callbacks cb 0x7f1a30009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40436/0 (socket says 172.21.15.3:40436) 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 -- 172.21.15.3:0/4030973863 learned_addr learned my addr 172.21.15.3:0/4030973863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3d400640 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 -- 172.21.15.3:0/4030973863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 msgr2=0x560c167eaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 0x560c167eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 -- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 msgr2=0x560c167ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 -- 172.21.15.3:0/4030973863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c167eb570 con 0x560c167e88f0 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3d400640 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:36:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a3de00640 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eaa20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1a30006010 tx=0x7f1a3000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c84a4917eb96a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a23e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a23e00640 1 -- 172.21.15.3:0/4030973863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3002f060 con 0x560c167e88f0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.009+0000 7f1a23e00640 1 -- 172.21.15.3:0/4030973863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f1a30012110 con 0x560c167e88f0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a28034780 con 0x560c167e88f0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a23e00640 1 -- 172.21.15.3:0/4030973863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a300049e0 con 0x560c167e88f0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 10 client.16599 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560c1685cf00 con 0x560c167e88f0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a23e00640 1 -- 172.21.15.3:0/4030973863 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a28034780 con 0x560c167e88f0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a23e00640 1 -- 172.21.15.3:0/4030973863 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1a30019050 con 0x560c167e88f0 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 10 client.16599 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 20 client.16599 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c1685d5a0 0x560c1685f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 4 client.16599 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.010+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c167540c0 con 0x560c1685d5a0 2024-03-21T08:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.011+0000 7f1a3d400640 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c1685d5a0 0x560c1685f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.012+0000 7f1a3d400640 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c1685d5a0 0x560c1685f9c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce46b3b24e67ad2d server_cookie=5a009254cf0d2e7d in_seq=0 out_seq=0 2024-03-21T08:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.012+0000 7f1a23e00640 10 client.16599 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a23e00640 1 -- 172.21.15.3:0/4030973863 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c167540c0 con 0x560c1685d5a0 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a23e00640 10 client.16599 handle_command_reply: tid=0 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 client.16599 shutdown 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 msgr2=0x560c167eaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eaa20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1a30006010 tx=0x7f1a3000cc00 comp rx=0 tx=0).stop 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 shutdown_connections 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x560c1685d5a0 0x560c1685f9c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c167f16c0 0x560c167eaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c167e88f0 0x560c167eaa20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 --2- 172.21.15.3:0/4030973863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560c167e7fd0 0x560c167ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.068+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 >> 172.21.15.3:0/4030973863 conn(0x560c167e28f0 msgr2=0x560c167e3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.069+0000 7f1a23e00640 0 client.16599 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.069+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 shutdown_connections 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.069+0000 7f1a47f86740 1 -- 172.21.15.3:0/4030973863 wait complete. 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.069+0000 7f1a47f86740 20 client.16599 trim_cache size 0 max 16384 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:36:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:36:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f1a240a3100 msgr2=0x7f1a240a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f1a240a3100 0x7f1a240a55c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009230 tx=0x7f1a38004730 comp rx=0 tx=0).stop 2024-03-21T08:36:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a401a2fd0 msgr2=0x7f1a401a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a401a2fd0 0x7f1a401a3480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a380058c0 tx=0x7f1a3800edd0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 shutdown_connections 2024-03-21T08:36:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a401a2fd0 0x7f1a401a3480 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a40112660 0x7f1a4019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f1a240a3100 0x7f1a240a55c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 --2- 172.21.15.3:0/1428516513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a40073a40 0x7f1a4019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 >> 172.21.15.3:0/1428516513 conn(0x7f1a400fd120 msgr2=0x7f1a400fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 shutdown_connections 2024-03-21T08:36:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:10.070+0000 7f1a45200640 1 -- 172.21.15.3:0/1428516513 wait complete. 2024-03-21T08:36:10.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:10 smithi084 ceph-mon[35477]: pgmap v132: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-03-21T08:36:10.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:10 smithi171 ceph-mon[35388]: pgmap v132: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-03-21T08:36:10.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:10 smithi003 ceph-mon[31675]: pgmap v132: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-03-21T08:36:12.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:12 smithi003 ceph-mon[31675]: pgmap v133: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-03-21T08:36:12.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:12 smithi084 ceph-mon[35477]: pgmap v133: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-03-21T08:36:12.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:12 smithi171 ceph-mon[35388]: pgmap v133: 129 pgs: 129 active+clean; 109 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-03-21T08:36:14.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:14 smithi084 ceph-mon[35477]: pgmap v134: 129 pgs: 129 active+clean; 112 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-03-21T08:36:14.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:14 smithi171 ceph-mon[35388]: pgmap v134: 129 pgs: 129 active+clean; 112 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-03-21T08:36:14.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:14 smithi003 ceph-mon[31675]: pgmap v134: 129 pgs: 129 active+clean; 112 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-03-21T08:36:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:16 smithi084 ceph-mon[35477]: pgmap v135: 129 pgs: 129 active+clean; 112 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-21T08:36:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:16 smithi171 ceph-mon[35388]: pgmap v135: 129 pgs: 129 active+clean; 112 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-21T08:36:16.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:16 smithi003 ceph-mon[31675]: pgmap v135: 129 pgs: 129 active+clean; 112 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-21T08:36:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:18 smithi084 ceph-mon[35477]: pgmap v136: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-21T08:36:18.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:18 smithi171 ceph-mon[35388]: pgmap v136: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-21T08:36:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:18 smithi003 ceph-mon[31675]: pgmap v136: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-21T08:36:20.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:20 smithi084 ceph-mon[35477]: pgmap v137: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-21T08:36:20.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:20 smithi171 ceph-mon[35388]: pgmap v137: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-21T08:36:20.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:20 smithi003 ceph-mon[31675]: pgmap v137: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-21T08:36:22.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:22 smithi084 ceph-mon[35477]: pgmap v138: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-21T08:36:22.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:22 smithi171 ceph-mon[35388]: pgmap v138: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-21T08:36:22.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:22 smithi003 ceph-mon[31675]: pgmap v138: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-21T08:36:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4292433315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08103690 msgr2=0x7f4e08103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4292433315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08103690 0x7f4e08103b10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009900 tx=0x7f4df402bc50 comp rx=0 tx=0).stop 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4292433315 shutdown_connections 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4292433315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e08104050 0x7f4e08110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4292433315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08103690 0x7f4e08103b10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4292433315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 0x7f4e08109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4292433315 >> 172.21.15.3:0/4292433315 conn(0x7f4e080ff180 msgr2=0x7f4e081015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4292433315 shutdown_connections 2024-03-21T08:36:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.227+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4292433315 wait complete. 2024-03-21T08:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.229+0000 7f4e0ce00640 1 Processor -- start 2024-03-21T08:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.229+0000 7f4e0ce00640 1 -- start start 2024-03-21T08:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e0ce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e08103690 0x7f4e08199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e0ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08104050 0x7f4e0819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e0ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 0x7f4e0819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e0ce00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081139b0 con 0x7f4e081096a0 2024-03-21T08:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e0ce00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08113830 con 0x7f4e08104050 2024-03-21T08:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e0ce00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08113b30 con 0x7f4e08103690 2024-03-21T08:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 0x7f4e0819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e06a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08104050 0x7f4e0819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 0x7f4e0819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37024/0 (socket says 172.21.15.3:37024) 2024-03-21T08:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e07e00640 1 -- 172.21.15.3:0/4206594620 learned_addr learned my addr 172.21.15.3:0/4206594620 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.230+0000 7f4e07400640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e08103690 0x7f4e08199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.231+0000 7f4e06a00640 1 -- 172.21.15.3:0/4206594620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e08103690 msgr2=0x7f4e08199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.231+0000 7f4e06a00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e08103690 0x7f4e08199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.231+0000 7f4e06a00640 1 -- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 msgr2=0x7f4e0819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.231+0000 7f4e06a00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 0x7f4e0819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.231+0000 7f4e06a00640 1 -- 172.21.15.3:0/4206594620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0819ee60 con 0x7f4e08104050 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.231+0000 7f4e07e00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 0x7f4e0819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.232+0000 7f4e06a00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08104050 0x7f4e0819a3b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4df402c190 tx=0x7f4df40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.232+0000 7f4dffe00640 1 -- 172.21.15.3:0/4206594620 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4043020 con 0x7f4e08104050 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.232+0000 7f4dffe00640 1 -- 172.21.15.3:0/4206594620 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4df402cb30 con 0x7f4e08104050 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.232+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0819f0f0 con 0x7f4e08104050 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.232+0000 7f4dffe00640 1 -- 172.21.15.3:0/4206594620 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4003ae0 con 0x7f4e08104050 2024-03-21T08:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.232+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e081a69e0 con 0x7f4e08104050 2024-03-21T08:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.235+0000 7f4e11167740 1 Processor -- start 2024-03-21T08:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.235+0000 7f4e11167740 1 -- start start 2024-03-21T08:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e11167740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc00965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e11167740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 0x55afc009f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e11167740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc00a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e11167740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc00583b0 con 0x55afc00961e0 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e11167740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbfffa830 con 0x55afc009f8d0 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e11167740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbffa35d0 con 0x55afc0096b00 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc00965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc00965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37040/0 (socket says 172.21.15.3:37040) 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4e07e00640 1 -- 172.21.15.3:0/3570612366 learned_addr learned my addr 172.21.15.3:0/3570612366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.236+0000 7f4dffe00640 1 -- 172.21.15.3:0/4206594620 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f4df4046020 con 0x7f4e08104050 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4e07e00640 1 -- 172.21.15.3:0/3570612366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 msgr2=0x55afc009f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4e07e00640 1 --2- 172.21.15.3:0/3570612366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 0x55afc009f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4e07e00640 1 -- 172.21.15.3:0/3570612366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 msgr2=0x55afc00a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4e07e00640 1 --2- 172.21.15.3:0/3570612366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc00a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4e07e00640 1 -- 172.21.15.3:0/3570612366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afbffbb2c0 con 0x55afc00961e0 2024-03-21T08:36:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4e07e00640 1 --2- 172.21.15.3:0/3570612366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc00965c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4e000117d0 tx=0x7f4e00011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23dba8adc4d032bb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4dfcc00640 1 -- 172.21.15.3:0/3570612366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000fb40 con 0x55afc00961e0 2024-03-21T08:36:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4dfcc00640 1 -- 172.21.15.3:0/3570612366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e00016070 con 0x55afc00961e0 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.237+0000 7f4dfcc00640 1 -- 172.21.15.3:0/3570612366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00010630 con 0x55afc00961e0 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4dffe00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4dec0a30f0 0x7f4dec0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4dffe00640 1 -- 172.21.15.3:0/4206594620 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f4df40f1ac0 con 0x7f4e08104050 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e07400640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4dec0a30f0 0x7f4dec0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e11167740 1 -- 172.21.15.3:0/3570612366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 msgr2=0x55afc00965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e11167740 1 --2- 172.21.15.3:0/3570612366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc00965c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4e000117d0 tx=0x7f4e00011c90 comp rx=0 tx=0).stop 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e11167740 1 -- 172.21.15.3:0/3570612366 shutdown_connections 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e11167740 1 --2- 172.21.15.3:0/3570612366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc00a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e11167740 1 --2- 172.21.15.3:0/3570612366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 0x55afc009f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e11167740 1 --2- 172.21.15.3:0/3570612366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc00965c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.238+0000 7f4e11167740 1 -- 172.21.15.3:0/3570612366 >> 172.21.15.3:0/3570612366 conn(0x55afc0090b00 msgr2=0x55afc0090ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.239+0000 7f4e11167740 1 -- 172.21.15.3:0/3570612366 shutdown_connections 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.239+0000 7f4e11167740 1 -- 172.21.15.3:0/3570612366 wait complete. 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.239+0000 7f4e07400640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4dec0a30f0 0x7f4dec0a55b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4df80095f0 tx=0x7f4df80046c0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.239+0000 7f4e11167740 1 -- start start 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.239+0000 7f4e11167740 1 Processor -- start 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e11167740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc009c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e11167740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 0x55afc009ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e11167740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc009d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e11167740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbffb6a20 con 0x55afc00961e0 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e11167740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbfff68d0 con 0x55afc009f8d0 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e11167740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc00a2500 con 0x55afc0096b00 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e11167740 10 client.0 _ll_register_callbacks cb 0x55afbfff31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e07400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc009c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc009d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc009d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:54042/0 (socket says 172.21.15.3:54042) 2024-03-21T08:36:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e07e00640 1 -- 172.21.15.3:0/4248117674 learned_addr learned my addr 172.21.15.3:0/4248117674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.240+0000 7f4e06a00640 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 0x55afc009ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.241+0000 7f4e07e00640 1 -- 172.21.15.3:0/4248117674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 msgr2=0x55afc009ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.241+0000 7f4e07e00640 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 0x55afc009ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.241+0000 7f4e07e00640 1 -- 172.21.15.3:0/4248117674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 msgr2=0x55afc009c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.241+0000 7f4e07e00640 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc009c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.241+0000 7f4e07e00640 1 -- 172.21.15.3:0/4248117674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afbfd0f210 con 0x55afc009f8d0 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.241+0000 7f4e07e00640 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc009d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4e00012cb0 tx=0x7f4e0000f940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6829ff3399bdd92 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.241+0000 7f4df1600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.242+0000 7f4df1600640 1 -- 172.21.15.3:0/4248117674 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0002b060 con 0x55afc009f8d0 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.242+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e00016070 con 0x55afc009f8d0 2024-03-21T08:36:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.242+0000 7f4df1600640 1 -- 172.21.15.3:0/4248117674 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f4e0000fd40 con 0x55afc009f8d0 2024-03-21T08:36:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.242+0000 7f4df1600640 1 -- 172.21.15.3:0/4248117674 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e00032050 con 0x55afc009f8d0 2024-03-21T08:36:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.242+0000 7f4e11167740 10 client.36344 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.242+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55afbfddb620 con 0x55afc009f8d0 2024-03-21T08:36:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.242+0000 7f4df1600640 1 -- 172.21.15.3:0/4248117674 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e00016070 con 0x55afc009f8d0 2024-03-21T08:36:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4df1600640 1 -- 172.21.15.3:0/4248117674 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4e0000f040 con 0x55afc009f8d0 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4e11167740 10 client.36344 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4e11167740 10 client.36344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4e11167740 20 client.36344 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4e11167740 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc00af860 0x55afc00b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4e11167740 4 client.36344 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afc0007d70 con 0x55afc00af860 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.243+0000 7f4e07400640 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc00af860 0x55afc00b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.244+0000 7f4e07400640 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc00af860 0x55afc00b1c50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a54f0b853d7ab1 server_cookie=ed0113d40f78fc86 in_seq=0 out_seq=0 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.244+0000 7f4df1600640 10 client.36344 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4df1600640 1 -- 172.21.15.3:0/4248117674 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55afc0007d70 con 0x55afc00af860 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4df1600640 10 client.36344 handle_command_reply: tid=0 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 client.36344 shutdown 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 msgr2=0x55afc009d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc009d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4e00012cb0 tx=0x7f4e0000f940 comp rx=0 tx=0).stop 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 shutdown_connections 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc009d3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc00af860 0x55afc00b1c50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc0096b00 0x55afc009ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 --2- 172.21.15.3:0/4248117674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc00961e0 0x55afc009c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4df1600640 0 client.36344 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 >> 172.21.15.3:0/4248117674 conn(0x55afc0090b00 msgr2=0x55afc0092850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 shutdown_connections 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.245+0000 7f4e11167740 1 -- 172.21.15.3:0/4248117674 wait complete. 2024-03-21T08:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.246+0000 7f4e11167740 20 client.36344 trim_cache size 0 max 16384 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.262+0000 7f4e11167740 1 Processor -- start 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.262+0000 7f4e11167740 1 -- start start 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.262+0000 7f4e11167740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc009d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e11167740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc0096b00 0x55afc00aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e11167740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc009f8d0 0x55afc0099bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e11167740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081b3140 con 0x55afc009f8d0 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e11167740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081b0be0 con 0x55afc0096b00 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e11167740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081aec80 con 0x55afc00961e0 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc009f8d0 0x55afc0099bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e06a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc0096b00 0x55afc00aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc009f8d0 0x55afc0099bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37052/0 (socket says 172.21.15.3:37052) 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 -- 172.21.15.3:0/1493574832 learned_addr learned my addr 172.21.15.3:0/1493574832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07400640 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc009d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 -- 172.21.15.3:0/1493574832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 msgr2=0x55afc009d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc009d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 -- 172.21.15.3:0/1493574832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc0096b00 msgr2=0x55afc00aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc0096b00 0x55afc00aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.263+0000 7f4e07e00640 1 -- 172.21.15.3:0/1493574832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afc009ae90 con 0x55afc009f8d0 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.264+0000 7f4e07e00640 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc009f8d0 0x55afc0099bc0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4e000117d0 tx=0x7f4e00012ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33772cc35c5bb31a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.264+0000 7f4df3e00640 1 -- 172.21.15.3:0/1493574832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0002b040 con 0x55afc009f8d0 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.264+0000 7f4df3e00640 1 -- 172.21.15.3:0/1493574832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e00016070 con 0x55afc009f8d0 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.264+0000 7f4df3e00640 1 -- 172.21.15.3:0/1493574832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00004a00 con 0x55afc009f8d0 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 -- 172.21.15.3:0/1493574832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc009f8d0 msgr2=0x55afc0099bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc009f8d0 0x55afc0099bc0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4e000117d0 tx=0x7f4e00012ce0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 -- 172.21.15.3:0/1493574832 shutdown_connections 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc009f8d0 0x55afc0099bc0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc0096b00 0x55afc00aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 --2- 172.21.15.3:0/1493574832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc009d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 -- 172.21.15.3:0/1493574832 >> 172.21.15.3:0/1493574832 conn(0x55afc0090b00 msgr2=0x55afc00aa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 -- 172.21.15.3:0/1493574832 shutdown_connections 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 -- 172.21.15.3:0/1493574832 wait complete. 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.265+0000 7f4e11167740 1 -- start start 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 1 Processor -- start 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc00b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc0096b00 0x55afc00b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc00b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081af280 con 0x55afc0096b00 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081b0580 con 0x55afc009f8d0 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081af8c0 con 0x55afc00961e0 2024-03-21T08:36:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.266+0000 7f4e11167740 10 client.0 _ll_register_callbacks cb 0x7f4e00011650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc0096b00 0x55afc00b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc0096b00 0x55afc00b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37068/0 (socket says 172.21.15.3:37068) 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 -- 172.21.15.3:0/3912012531 learned_addr learned my addr 172.21.15.3:0/3912012531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e07e00640 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc00b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e07400640 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc00b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 -- 172.21.15.3:0/3912012531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 msgr2=0x55afc00b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc00b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 -- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 msgr2=0x55afc00b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc00b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 -- 172.21.15.3:0/3912012531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afc0109d90 con 0x55afc0096b00 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4e06a00640 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc0096b00 0x55afc00b26d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4df4006010 tx=0x7f4df40b9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd015e5315ad71d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4df1600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:36:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.267+0000 7f4df1600640 1 -- 172.21.15.3:0/3912012531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df40bdba0 con 0x55afc0096b00 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4df1600640 1 -- 172.21.15.3:0/3912012531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f4df4003c80 con 0x55afc0096b00 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e00016070 con 0x55afc0096b00 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4df1600640 1 -- 172.21.15.3:0/3912012531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df40c1cf0 con 0x55afc0096b00 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4e11167740 10 client.16617 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55afc010b060 con 0x55afc0096b00 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4df1600640 1 -- 172.21.15.3:0/3912012531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e00016070 con 0x55afc0096b00 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4df1600640 1 -- 172.21.15.3:0/3912012531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4df40d5020 con 0x55afc0096b00 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4e11167740 10 client.16617 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4e11167740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.268+0000 7f4e11167740 20 client.16617 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.269+0000 7f4e11167740 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc010b7f0 0x55afc010dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.269+0000 7f4e11167740 4 client.16617 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:36:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.269+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55afc00020c0 con 0x55afc010b7f0 2024-03-21T08:36:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.269+0000 7f4e07400640 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc010b7f0 0x55afc010dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.270+0000 7f4e07400640 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc010b7f0 0x55afc010dc10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a158a73302d319 server_cookie=d2cf5bd10befe624 in_seq=0 out_seq=0 2024-03-21T08:36:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.270+0000 7f4df1600640 10 client.16617 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4df1600640 1 -- 172.21.15.3:0/3912012531 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55afc00020c0 con 0x55afc010b7f0 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4df1600640 10 client.16617 handle_command_reply: tid=0 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 client.16617 shutdown 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc0096b00 msgr2=0x55afc00b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc0096b00 0x55afc00b26d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4df4006010 tx=0x7f4df40b9f70 comp rx=0 tx=0).stop 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 shutdown_connections 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55afc009f8d0 0x55afc00b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55afc0096b00 0x55afc00b26d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x55afc010b7f0 0x55afc010dc10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 --2- 172.21.15.3:0/3912012531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afc00961e0 0x55afc00b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4df1600640 0 client.16617 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 >> 172.21.15.3:0/3912012531 conn(0x55afc0090b00 msgr2=0x55afc00aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 shutdown_connections 2024-03-21T08:36:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 1 -- 172.21.15.3:0/3912012531 wait complete. 2024-03-21T08:36:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.346+0000 7f4e11167740 20 client.16617 trim_cache size 0 max 16384 2024-03-21T08:36:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:36:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:36:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:36:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:36:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4dec0a30f0 msgr2=0x7f4dec0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4dec0a30f0 0x7f4dec0a55b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4df80095f0 tx=0x7f4df80046c0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08104050 msgr2=0x7f4e0819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08104050 0x7f4e0819a3b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4df402c190 tx=0x7f4df40045e0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 shutdown_connections 2024-03-21T08:36:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e081096a0 0x7f4e0819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e08104050 0x7f4e0819a3b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f4dec0a30f0 0x7f4dec0a55b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 --2- 172.21.15.3:0/4206594620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e08103690 0x7f4e08199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 >> 172.21.15.3:0/4206594620 conn(0x7f4e080ff180 msgr2=0x7f4e080ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.347+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 shutdown_connections 2024-03-21T08:36:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:23.348+0000 7f4e0ce00640 1 -- 172.21.15.3:0/4206594620 wait complete. 2024-03-21T08:36:24.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:24 smithi084 ceph-mon[35477]: pgmap v139: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-03-21T08:36:24.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:24 smithi171 ceph-mon[35388]: pgmap v139: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-03-21T08:36:24.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:24 smithi003 ceph-mon[31675]: pgmap v139: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-03-21T08:36:26.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:26 smithi084 ceph-mon[35477]: pgmap v140: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-03-21T08:36:26.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:26 smithi171 ceph-mon[35388]: pgmap v140: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-03-21T08:36:26.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:26 smithi003 ceph-mon[31675]: pgmap v140: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-03-21T08:36:28.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:28 smithi084 ceph-mon[35477]: pgmap v141: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-03-21T08:36:28.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:28 smithi171 ceph-mon[35388]: pgmap v141: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-03-21T08:36:28.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:28 smithi003 ceph-mon[31675]: pgmap v141: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-03-21T08:36:30.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:30 smithi084 ceph-mon[35477]: pgmap v142: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-21T08:36:30.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:30 smithi171 ceph-mon[35388]: pgmap v142: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-21T08:36:30.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:30 smithi003 ceph-mon[31675]: pgmap v142: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-21T08:36:32.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:32 smithi084 ceph-mon[35477]: pgmap v143: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-21T08:36:32.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:32 smithi171 ceph-mon[35388]: pgmap v143: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-21T08:36:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:32 smithi003 ceph-mon[31675]: pgmap v143: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-21T08:36:34.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:34 smithi084 ceph-mon[35477]: pgmap v144: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-03-21T08:36:34.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:34 smithi171 ceph-mon[35388]: pgmap v144: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-03-21T08:36:34.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:34 smithi003 ceph-mon[31675]: pgmap v144: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-03-21T08:36:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 -- 172.21.15.3:0/3396207689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc103890 msgr2=0x7f3ddc103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 --2- 172.21.15.3:0/3396207689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc103890 0x7f3ddc103d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009850 tx=0x7f3dc802bb70 comp rx=0 tx=0).stop 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 -- 172.21.15.3:0/3396207689 shutdown_connections 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 --2- 172.21.15.3:0/3396207689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ddc104250 0x7f3ddc110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 --2- 172.21.15.3:0/3396207689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc103890 0x7f3ddc103d10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 --2- 172.21.15.3:0/3396207689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc1098a0 0x7f3ddc109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 -- 172.21.15.3:0/3396207689 >> 172.21.15.3:0/3396207689 conn(0x7f3ddc0ff320 msgr2=0x7f3ddc101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 -- 172.21.15.3:0/3396207689 shutdown_connections 2024-03-21T08:36:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.498+0000 7f3de4600640 1 -- 172.21.15.3:0/3396207689 wait complete. 2024-03-21T08:36:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.499+0000 7f3de4600640 1 Processor -- start 2024-03-21T08:36:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de4600640 1 -- start start 2024-03-21T08:36:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de4600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc103890 0x7f3ddc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de4600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ddc104250 0x7f3ddc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de4600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc1098a0 0x7f3ddc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de4600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc113b60 con 0x7f3ddc103890 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de4600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1139e0 con 0x7f3ddc1098a0 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de4600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc113ce0 con 0x7f3ddc104250 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc103890 0x7f3ddc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc103890 0x7f3ddc19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47394/0 (socket says 172.21.15.3:47394) 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.500+0000 7f3de1c00640 1 -- 172.21.15.3:0/1619233541 learned_addr learned my addr 172.21.15.3:0/1619233541 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de2600640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc1098a0 0x7f3ddc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de1200640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ddc104250 0x7f3ddc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de1c00640 1 -- 172.21.15.3:0/1619233541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ddc104250 msgr2=0x7f3ddc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de1c00640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ddc104250 0x7f3ddc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de1c00640 1 -- 172.21.15.3:0/1619233541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc1098a0 msgr2=0x7f3ddc19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de1c00640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc1098a0 0x7f3ddc19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de1c00640 1 -- 172.21.15.3:0/1619233541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc19f070 con 0x7f3ddc103890 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.501+0000 7f3de1c00640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc103890 0x7f3ddc19a080 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000e9c0 tx=0x7f3dd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.502+0000 7f3dda000640 1 -- 172.21.15.3:0/1619233541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000cde0 con 0x7f3ddc103890 2024-03-21T08:36:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.502+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc19f360 con 0x7f3ddc103890 2024-03-21T08:36:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.502+0000 7f3dda000640 1 -- 172.21.15.3:0/1619233541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dd00035f0 con 0x7f3ddc103890 2024-03-21T08:36:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.502+0000 7f3dda000640 1 -- 172.21.15.3:0/1619233541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0010690 con 0x7f3ddc103890 2024-03-21T08:36:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.502+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc1a6bf0 con 0x7f3ddc103890 2024-03-21T08:36:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.506+0000 7f3de7392740 1 Processor -- start 2024-03-21T08:36:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.506+0000 7f3de7392740 1 -- start start 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3dda000640 1 -- 172.21.15.3:0/1619233541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f3dd00037a0 con 0x7f3ddc103890 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de7392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895bfd0 0x56190895c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de7392740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895c8f0 0x561908965180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de7392740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x561908967ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de7392740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190891e3b0 con 0x56190895c8f0 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de7392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619088c0830 con 0x56190895bfd0 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de7392740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619088695d0 con 0x5619089656c0 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895c8f0 0x561908965180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de2600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895bfd0 0x56190895c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de2600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895bfd0 0x56190895c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:60874/0 (socket says 172.21.15.3:60874) 2024-03-21T08:36:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de2600640 1 -- 172.21.15.3:0/2635121032 learned_addr learned my addr 172.21.15.3:0/2635121032 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.507+0000 7f3de1c00640 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895c8f0 0x561908965180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47410/0 (socket says 172.21.15.3:47410) 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.508+0000 7f3de1200640 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x561908967ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.508+0000 7f3de2600640 1 -- 172.21.15.3:0/2635121032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 msgr2=0x561908967ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.508+0000 7f3de2600640 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x561908967ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.508+0000 7f3de2600640 1 -- 172.21.15.3:0/2635121032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895c8f0 msgr2=0x561908965180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.508+0000 7f3de2600640 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895c8f0 0x561908965180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.508+0000 7f3de2600640 1 -- 172.21.15.3:0/2635121032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619088812c0 con 0x56190895bfd0 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.508+0000 7f3de2600640 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895bfd0 0x56190895c3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4010370 tx=0x7f3dd4010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e943f5b7347ed08 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.509+0000 7f3dda000640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3dc40a3080 0x7f3dc40a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.509+0000 7f3dda000640 1 -- 172.21.15.3:0/1619233541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3dd00cff00 con 0x7f3ddc103890 2024-03-21T08:36:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.509+0000 7f3de1c00640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3dc40a3080 0x7f3dc40a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.509+0000 7f3dcea00640 1 -- 172.21.15.3:0/2635121032 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4016070 con 0x56190895bfd0 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3dcea00640 1 -- 172.21.15.3:0/2635121032 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dd4004d20 con 0x56190895bfd0 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de1c00640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3dc40a3080 0x7f3dc40a5540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00cd2e0 tx=0x7f3dd00d3a50 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3dcea00640 1 -- 172.21.15.3:0/2635121032 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4005390 con 0x56190895bfd0 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de7392740 1 -- 172.21.15.3:0/2635121032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895bfd0 msgr2=0x56190895c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de7392740 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895bfd0 0x56190895c3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4010370 tx=0x7f3dd4010830 comp rx=0 tx=0).stop 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de7392740 1 -- 172.21.15.3:0/2635121032 shutdown_connections 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de7392740 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x561908967ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de7392740 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895c8f0 0x561908965180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de7392740 1 --2- 172.21.15.3:0/2635121032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895bfd0 0x56190895c3b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.510+0000 7f3de7392740 1 -- 172.21.15.3:0/2635121032 >> 172.21.15.3:0/2635121032 conn(0x5619089568f0 msgr2=0x561908956cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.511+0000 7f3de7392740 1 -- 172.21.15.3:0/2635121032 shutdown_connections 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.511+0000 7f3de7392740 1 -- 172.21.15.3:0/2635121032 wait complete. 2024-03-21T08:36:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.511+0000 7f3de7392740 1 -- start start 2024-03-21T08:36:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.511+0000 7f3de7392740 1 Processor -- start 2024-03-21T08:36:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de7392740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x561908962730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de7392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de7392740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x5619089631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de7392740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190887ca20 con 0x56190895bfd0 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de7392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619088bc8d0 con 0x56190895c8f0 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de7392740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619089682f0 con 0x5619089656c0 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de7392740 10 client.0 _ll_register_callbacks cb 0x5619088b91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x561908962730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x561908962730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47420/0 (socket says 172.21.15.3:47420) 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de2600640 1 -- 172.21.15.3:0/778303676 learned_addr learned my addr 172.21.15.3:0/778303676 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.512+0000 7f3de1200640 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3de2600640 1 -- 172.21.15.3:0/778303676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 msgr2=0x5619089631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3de2600640 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x5619089631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3de2600640 1 -- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 msgr2=0x561908962c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3de2600640 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3de2600640 1 -- 172.21.15.3:0/778303676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619085d5210 con 0x56190895bfd0 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3de2600640 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x561908962730 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4010fe0 tx=0x7f3dd4004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b2c295f741104b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3dc3400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3dc3400640 1 -- 172.21.15.3:0/778303676 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4016070 con 0x56190895bfd0 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dd4004d20 con 0x56190895bfd0 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.513+0000 7f3dc3400640 1 -- 172.21.15.3:0/778303676 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3dd4003b70 con 0x56190895bfd0 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3dc3400640 1 -- 172.21.15.3:0/778303676 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd4003d70 con 0x56190895bfd0 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 10 client.16629 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619086a1620 con 0x56190895bfd0 2024-03-21T08:36:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3dc3400640 1 -- 172.21.15.3:0/778303676 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dd4004d20 con 0x56190895bfd0 2024-03-21T08:36:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3dc3400640 1 -- 172.21.15.3:0/778303676 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3dd400f040 con 0x56190895bfd0 2024-03-21T08:36:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 10 client.16629 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 20 client.16629 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561908975600 0x5619089779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 4 client.16629 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:36:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.514+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619088cdd70 con 0x561908975600 2024-03-21T08:36:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.515+0000 7f3de1200640 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561908975600 0x5619089779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.516+0000 7f3de1200640 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561908975600 0x5619089779f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f1ed84867c3a8b server_cookie=c8bca6628fafde5e in_seq=0 out_seq=0 2024-03-21T08:36:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.516+0000 7f3dc3400640 10 client.16629 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3dc3400640 1 -- 172.21.15.3:0/778303676 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5619088cdd70 con 0x561908975600 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3dc3400640 10 client.16629 handle_command_reply: tid=0 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 client.16629 shutdown 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 msgr2=0x561908962730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x561908962730 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4010fe0 tx=0x7f3dd4004c70 comp rx=0 tx=0).stop 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 shutdown_connections 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x561908975600 0x5619089779f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x5619089631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 --2- 172.21.15.3:0/778303676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x561908962730 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3dc3400640 0 client.16629 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 >> 172.21.15.3:0/778303676 conn(0x5619089568f0 msgr2=0x561908958640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 shutdown_connections 2024-03-21T08:36:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 1 -- 172.21.15.3:0/778303676 wait complete. 2024-03-21T08:36:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.520+0000 7f3de7392740 20 client.16629 trim_cache size 0 max 16384 2024-03-21T08:36:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 Processor -- start 2024-03-21T08:36:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 -- start start 2024-03-21T08:36:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56190895bfd0 0x561908964e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619089656c0 0x561908970a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1b3450 con 0x5619089656c0 2024-03-21T08:36:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1b0ef0 con 0x56190895c8f0 2024-03-21T08:36:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de7392740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1aef90 con 0x56190895bfd0 2024-03-21T08:36:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619089656c0 0x561908970a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.537+0000 7f3de1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619089656c0 0x561908970a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47430/0 (socket says 172.21.15.3:47430) 2024-03-21T08:36:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 -- 172.21.15.3:0/625098645 learned_addr learned my addr 172.21.15.3:0/625098645 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 -- 172.21.15.3:0/625098645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56190895bfd0 msgr2=0x561908964e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 --2- 172.21.15.3:0/625098645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56190895bfd0 0x561908964e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 -- 172.21.15.3:0/625098645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 msgr2=0x561908962e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 --2- 172.21.15.3:0/625098645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 -- 172.21.15.3:0/625098645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619089681d0 con 0x5619089656c0 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.538+0000 7f3de2600640 1 --2- 172.21.15.3:0/625098645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619089656c0 0x561908970a00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009770 tx=0x7f3dd4009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755a5e26aa4eab3f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3dce000640 1 -- 172.21.15.3:0/625098645 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4038070 con 0x5619089656c0 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3dce000640 1 -- 172.21.15.3:0/625098645 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dd4022070 con 0x5619089656c0 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3dce000640 1 -- 172.21.15.3:0/625098645 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4016030 con 0x5619089656c0 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3de7392740 1 -- 172.21.15.3:0/625098645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619089656c0 msgr2=0x561908970a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3de7392740 1 --2- 172.21.15.3:0/625098645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619089656c0 0x561908970a00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009770 tx=0x7f3dd4009600 comp rx=0 tx=0).stop 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3de7392740 1 -- 172.21.15.3:0/625098645 shutdown_connections 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3de7392740 1 --2- 172.21.15.3:0/625098645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619089656c0 0x561908970a00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3de7392740 1 --2- 172.21.15.3:0/625098645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x561908962e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3de7392740 1 --2- 172.21.15.3:0/625098645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56190895bfd0 0x561908964e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.539+0000 7f3de7392740 1 -- 172.21.15.3:0/625098645 >> 172.21.15.3:0/625098645 conn(0x5619089568f0 msgr2=0x5619089626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.540+0000 7f3de7392740 1 -- 172.21.15.3:0/625098645 shutdown_connections 2024-03-21T08:36:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.540+0000 7f3de7392740 1 -- 172.21.15.3:0/625098645 wait complete. 2024-03-21T08:36:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.540+0000 7f3de7392740 1 -- start start 2024-03-21T08:36:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.540+0000 7f3de7392740 1 Processor -- start 2024-03-21T08:36:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de7392740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x56190896a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de7392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x56190896aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de7392740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x56190896af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de7392740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1af590 con 0x56190895bfd0 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de7392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1b0890 con 0x56190895c8f0 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de7392740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1afbd0 con 0x5619089656c0 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de7392740 10 client.0 _ll_register_callbacks cb 0x7f3dd00d3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x56190896a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x56190896a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47444/0 (socket says 172.21.15.3:47444) 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de2600640 1 -- 172.21.15.3:0/3323645531 learned_addr learned my addr 172.21.15.3:0/3323645531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de2600640 1 -- 172.21.15.3:0/3323645531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 msgr2=0x56190896af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de2600640 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x56190896af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de2600640 1 -- 172.21.15.3:0/3323645531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 msgr2=0x56190896aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.541+0000 7f3de2600640 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x56190896aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.542+0000 7f3de2600640 1 -- 172.21.15.3:0/3323645531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619089d0b60 con 0x56190895bfd0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.542+0000 7f3de2600640 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x56190896a510 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4036040 tx=0x7f3dd4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d70876fb7d6ed3b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.542+0000 7f3dc3400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.542+0000 7f3dc3400640 1 -- 172.21.15.3:0/3323645531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4038070 con 0x56190895bfd0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.542+0000 7f3dc3400640 1 -- 172.21.15.3:0/3323645531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f3dd4004c50 con 0x56190895bfd0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.542+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dd4022070 con 0x56190895bfd0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.542+0000 7f3dc3400640 1 -- 172.21.15.3:0/3323645531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd4004120 con 0x56190895bfd0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 10 client.16635 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619089d1d70 con 0x56190895bfd0 2024-03-21T08:36:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3dc3400640 1 -- 172.21.15.3:0/3323645531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dd4022070 con 0x56190895bfd0 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3dc3400640 1 -- 172.21.15.3:0/3323645531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3dd4005450 con 0x56190895bfd0 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 10 client.16635 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 20 client.16635 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619089d2380 0x5619089d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 4 client.16635 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.543+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619088c80c0 con 0x5619089d2380 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.544+0000 7f3de1200640 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619089d2380 0x5619089d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.544+0000 7f3de1200640 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619089d2380 0x5619089d47a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf8b84fa6379d17 server_cookie=8360a14dec7b9b34 in_seq=0 out_seq=0 2024-03-21T08:36:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.544+0000 7f3dc3400640 10 client.16635 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.617+0000 7f3dc3400640 1 -- 172.21.15.3:0/3323645531 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619088c80c0 con 0x5619089d2380 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.617+0000 7f3dc3400640 10 client.16635 handle_command_reply: tid=0 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.617+0000 7f3de7392740 1 client.16635 shutdown 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.617+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 msgr2=0x56190896a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.617+0000 7f3de7392740 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x56190896a510 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4036040 tx=0x7f3dd4005c50 comp rx=0 tx=0).stop 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 shutdown_connections 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5619089d2380 0x5619089d47a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5619089656c0 0x56190896af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56190895c8f0 0x56190896aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 --2- 172.21.15.3:0/3323645531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56190895bfd0 0x56190896a510 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3dc3400640 0 client.16635 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 >> 172.21.15.3:0/3323645531 conn(0x5619089568f0 msgr2=0x561908955950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 shutdown_connections 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 1 -- 172.21.15.3:0/3323645531 wait complete. 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.618+0000 7f3de7392740 20 client.16635 trim_cache size 0 max 16384 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:36:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:36:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3dc40a3080 msgr2=0x7f3dc40a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3dc40a3080 0x7f3dc40a5540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00cd2e0 tx=0x7f3dd00d3a50 comp rx=0 tx=0).stop 2024-03-21T08:36:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc103890 msgr2=0x7f3ddc19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc103890 0x7f3ddc19a080 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000e9c0 tx=0x7f3dd000ee80 comp rx=0 tx=0).stop 2024-03-21T08:36:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 shutdown_connections 2024-03-21T08:36:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc1098a0 0x7f3ddc19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3dc40a3080 0x7f3dc40a5540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ddc104250 0x7f3ddc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 --2- 172.21.15.3:0/1619233541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ddc103890 0x7f3ddc19a080 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 >> 172.21.15.3:0/1619233541 conn(0x7f3ddc0ff320 msgr2=0x7f3ddc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 shutdown_connections 2024-03-21T08:36:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:36.619+0000 7f3de4600640 1 -- 172.21.15.3:0/1619233541 wait complete. 2024-03-21T08:36:36.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:36 smithi084 ceph-mon[35477]: pgmap v145: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-03-21T08:36:36.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:36 smithi171 ceph-mon[35388]: pgmap v145: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-03-21T08:36:36.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:36 smithi003 ceph-mon[31675]: pgmap v145: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-03-21T08:36:38.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:38 smithi084 ceph-mon[35477]: pgmap v146: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-03-21T08:36:38.909 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:38 smithi171 ceph-mon[35388]: pgmap v146: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-03-21T08:36:38.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:38 smithi003 ceph-mon[31675]: pgmap v146: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-03-21T08:36:40.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:40 smithi084 ceph-mon[35477]: pgmap v147: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-21T08:36:40.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:40 smithi171 ceph-mon[35388]: pgmap v147: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-21T08:36:40.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:40 smithi003 ceph-mon[31675]: pgmap v147: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-21T08:36:41.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:41 smithi084 ceph-mon[35477]: pgmap v148: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-21T08:36:41.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:41 smithi171 ceph-mon[35388]: pgmap v148: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-21T08:36:41.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:41 smithi003 ceph-mon[31675]: pgmap v148: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-21T08:36:44.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:43 smithi084 ceph-mon[35477]: pgmap v149: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-21T08:36:44.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:43 smithi171 ceph-mon[35388]: pgmap v149: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-21T08:36:44.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:43 smithi003 ceph-mon[31675]: pgmap v149: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-21T08:36:46.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:45 smithi084 ceph-mon[35477]: pgmap v150: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-03-21T08:36:46.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:45 smithi171 ceph-mon[35388]: pgmap v150: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-03-21T08:36:46.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:45 smithi003 ceph-mon[31675]: pgmap v150: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-03-21T08:36:48.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:47 smithi084 ceph-mon[35477]: pgmap v151: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-03-21T08:36:48.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:47 smithi171 ceph-mon[35388]: pgmap v151: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-03-21T08:36:48.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:47 smithi003 ceph-mon[31675]: pgmap v151: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-03-21T08:36:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 -- 172.21.15.3:0/1972530804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c10dab0 msgr2=0x7f997c10de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 --2- 172.21.15.3:0/1972530804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c10dab0 0x7f997c10de90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f996c009900 tx=0x7f996c02bc50 comp rx=0 tx=0).stop 2024-03-21T08:36:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 -- 172.21.15.3:0/1972530804 shutdown_connections 2024-03-21T08:36:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 --2- 172.21.15.3:0/1972530804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f997c108460 0x7f997c114970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 --2- 172.21.15.3:0/1972530804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c107aa0 0x7f997c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 --2- 172.21.15.3:0/1972530804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c10dab0 0x7f997c10de90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 -- 172.21.15.3:0/1972530804 >> 172.21.15.3:0/1972530804 conn(0x7f997c077220 msgr2=0x7f997c079640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 -- 172.21.15.3:0/1972530804 shutdown_connections 2024-03-21T08:36:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.774+0000 7f9982000640 1 -- 172.21.15.3:0/1972530804 wait complete. 2024-03-21T08:36:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.775+0000 7f9982000640 1 Processor -- start 2024-03-21T08:36:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.775+0000 7f9982000640 1 -- start start 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9982000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c107aa0 0x7f997c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9982000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f997c108460 0x7f997c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9982000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c10dab0 0x7f997c1a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9982000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c117d40 con 0x7f997c10dab0 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9982000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c117bc0 con 0x7f997c107aa0 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9982000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c117ec0 con 0x7f997c108460 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9981600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c10dab0 0x7f997c1a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9980c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c107aa0 0x7f997c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9981600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c10dab0 0x7f997c1a2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35412/0 (socket says 172.21.15.3:35412) 2024-03-21T08:36:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9980c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c107aa0 0x7f997c19e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:37506/0 (socket says 172.21.15.3:37506) 2024-03-21T08:36:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9981600640 1 -- 172.21.15.3:0/3021102939 learned_addr learned my addr 172.21.15.3:0/3021102939 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.776+0000 7f9973e00640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f997c108460 0x7f997c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9981600640 1 -- 172.21.15.3:0/3021102939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f997c108460 msgr2=0x7f997c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9981600640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f997c108460 0x7f997c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9981600640 1 -- 172.21.15.3:0/3021102939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c107aa0 msgr2=0x7f997c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9981600640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c107aa0 0x7f997c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9981600640 1 -- 172.21.15.3:0/3021102939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c1a32b0 con 0x7f997c10dab0 2024-03-21T08:36:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9973e00640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f997c108460 0x7f997c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:36:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9980c00640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c107aa0 0x7f997c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:36:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9981600640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c10dab0 0x7f997c1a2b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9974009fd0 tx=0x7f997400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f997a000640 1 -- 172.21.15.3:0/3021102939 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974017070 con 0x7f997c10dab0 2024-03-21T08:36:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c1a35a0 con 0x7f997c10dab0 2024-03-21T08:36:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f997a000640 1 -- 172.21.15.3:0/3021102939 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9974003e20 con 0x7f997c10dab0 2024-03-21T08:36:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f997a000640 1 -- 172.21.15.3:0/3021102939 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974010460 con 0x7f997c10dab0 2024-03-21T08:36:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.777+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c1aae30 con 0x7f997c10dab0 2024-03-21T08:36:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.781+0000 7f997a000640 1 -- 172.21.15.3:0/3021102939 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f9974003970 con 0x7f997c10dab0 2024-03-21T08:36:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.781+0000 7f998641e740 1 Processor -- start 2024-03-21T08:36:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.781+0000 7f998641e740 1 -- start start 2024-03-21T08:36:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.781+0000 7f998641e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.781+0000 7f998641e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da3b00 0x564516dac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.781+0000 7f998641e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516daecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.781+0000 7f998641e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564516d653b0 con 0x564516da31e0 2024-03-21T08:36:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f998641e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x564516d07830 con 0x564516dac8d0 2024-03-21T08:36:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f998641e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x564516cb05d0 con 0x564516da3b00 2024-03-21T08:36:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35424/0 (socket says 172.21.15.3:35424) 2024-03-21T08:36:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 -- 172.21.15.3:0/2784905105 learned_addr learned my addr 172.21.15.3:0/2784905105 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516daecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9980c00640 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da3b00 0x564516dac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 -- 172.21.15.3:0/2784905105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da3b00 msgr2=0x564516dac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da3b00 0x564516dac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 -- 172.21.15.3:0/2784905105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 msgr2=0x564516daecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.782+0000 7f9973e00640 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516daecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f9973e00640 1 -- 172.21.15.3:0/2784905105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564516cc82c0 con 0x564516da31e0 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f997a000640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99600a3070 0x7f99600a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f9973e00640 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da35c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99640137e0 tx=0x7f99640096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18896c886a4393c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f997a000640 1 -- 172.21.15.3:0/3021102939 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f99740cfa30 con 0x7f997c10dab0 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f9980c00640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99600a3070 0x7f99600a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f9971c00640 1 -- 172.21.15.3:0/2784905105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964013e20 con 0x564516da31e0 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f9971c00640 1 -- 172.21.15.3:0/2784905105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99640130d0 con 0x564516da31e0 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.783+0000 7f9971c00640 1 -- 172.21.15.3:0/2784905105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964011890 con 0x564516da31e0 2024-03-21T08:36:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 -- 172.21.15.3:0/2784905105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 msgr2=0x564516da35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da35c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99640137e0 tx=0x7f99640096e0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f9980c00640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99600a3070 0x7f99600a5530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f996c006010 tx=0x7f996c02c6e0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 -- 172.21.15.3:0/2784905105 shutdown_connections 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516daecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da3b00 0x564516dac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 --2- 172.21.15.3:0/2784905105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da35c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 -- 172.21.15.3:0/2784905105 >> 172.21.15.3:0/2784905105 conn(0x564516d9db00 msgr2=0x564516d9dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 -- 172.21.15.3:0/2784905105 shutdown_connections 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.784+0000 7f998641e740 1 -- 172.21.15.3:0/2784905105 wait complete. 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.785+0000 7f998641e740 1 -- start start 2024-03-21T08:36:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.785+0000 7f998641e740 1 Processor -- start 2024-03-21T08:36:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f998641e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516da99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f998641e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f998641e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516dac8d0 0x564516daa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f998641e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564516cc3a20 con 0x564516dac8d0 2024-03-21T08:36:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f998641e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x564516d038d0 con 0x564516da3b00 2024-03-21T08:36:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f998641e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x564516daf4a0 con 0x564516da31e0 2024-03-21T08:36:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f998641e740 10 client.0 _ll_register_callbacks cb 0x564516d001c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9981600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516dac8d0 0x564516daa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9973e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9981600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516dac8d0 0x564516daa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35432/0 (socket says 172.21.15.3:35432) 2024-03-21T08:36:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9981600640 1 -- 172.21.15.3:0/429203728 learned_addr learned my addr 172.21.15.3:0/429203728 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9980c00640 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516da99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9973e00640 1 -- 172.21.15.3:0/429203728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 msgr2=0x564516da99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9973e00640 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516da99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9973e00640 1 -- 172.21.15.3:0/429203728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516dac8d0 msgr2=0x564516daa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9973e00640 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516dac8d0 0x564516daa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9973e00640 1 -- 172.21.15.3:0/429203728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564516a1c210 con 0x564516da3b00 2024-03-21T08:36:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.786+0000 7f9981600640 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516dac8d0 0x564516daa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.787+0000 7f9973e00640 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da9f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f996401d000 tx=0x7f996400fe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5ed51f8e7cd24f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.787+0000 7f996a000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.787+0000 7f996a000640 1 -- 172.21.15.3:0/429203728 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964004240 con 0x564516da3b00 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.787+0000 7f996a000640 1 -- 172.21.15.3:0/429203728 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f99640043e0 con 0x564516da3b00 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.787+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99640130d0 con 0x564516da3b00 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f996a000640 1 -- 172.21.15.3:0/429203728 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9964011610 con 0x564516da3b00 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 10 client.36377 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564516ae8620 con 0x564516da3b00 2024-03-21T08:36:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f996a000640 1 -- 172.21.15.3:0/429203728 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99640130d0 con 0x564516da3b00 2024-03-21T08:36:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f996a000640 1 -- 172.21.15.3:0/429203728 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9964003780 con 0x564516da3b00 2024-03-21T08:36:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 10 client.36377 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 10 client.36377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 20 client.36377 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516dbc860 0x564516dbec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 4 client.36377 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:36:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.788+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564516d14d70 con 0x564516dbc860 2024-03-21T08:36:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.789+0000 7f9980c00640 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516dbc860 0x564516dbec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.790+0000 7f9980c00640 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516dbc860 0x564516dbec50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72663096dc690e3d server_cookie=5d772832fe051e36 in_seq=0 out_seq=0 2024-03-21T08:36:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.790+0000 7f996a000640 10 client.36377 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f996a000640 1 -- 172.21.15.3:0/429203728 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564516d14d70 con 0x564516dbc860 2024-03-21T08:36:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f996a000640 10 client.36377 handle_command_reply: tid=0 2024-03-21T08:36:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 client.36377 shutdown 2024-03-21T08:36:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 msgr2=0x564516da9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da9f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f996401d000 tx=0x7f996400fe30 comp rx=0 tx=0).stop 2024-03-21T08:36:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 shutdown_connections 2024-03-21T08:36:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516dac8d0 0x564516daa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da9f00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516dbc860 0x564516dbec50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 --2- 172.21.15.3:0/429203728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516da99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 >> 172.21.15.3:0/429203728 conn(0x564516d9db00 msgr2=0x564516d9f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f996a000640 0 client.36377 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.791+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 shutdown_connections 2024-03-21T08:36:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.792+0000 7f998641e740 1 -- 172.21.15.3:0/429203728 wait complete. 2024-03-21T08:36:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.792+0000 7f998641e740 20 client.36377 trim_cache size 0 max 16384 2024-03-21T08:36:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 Processor -- start 2024-03-21T08:36:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 -- start start 2024-03-21T08:36:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516dac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da3b00 0x564516d9c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516db7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1b75e0 con 0x564516da3b00 2024-03-21T08:36:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1b5080 con 0x564516dac8d0 2024-03-21T08:36:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f998641e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1b3120 con 0x564516da31e0 2024-03-21T08:36:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f9973e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da3b00 0x564516d9c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.810+0000 7f9981600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516db7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9981600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516db7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:37534/0 (socket says 172.21.15.3:37534) 2024-03-21T08:36:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9981600640 1 -- 172.21.15.3:0/3064768590 learned_addr learned my addr 172.21.15.3:0/3064768590 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9973e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da3b00 0x564516d9c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35442/0 (socket says 172.21.15.3:35442) 2024-03-21T08:36:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9980c00640 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516dac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9973e00640 1 -- 172.21.15.3:0/3064768590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 msgr2=0x564516dac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9973e00640 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516dac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9973e00640 1 -- 172.21.15.3:0/3064768590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 msgr2=0x564516db7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9973e00640 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516db7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9973e00640 1 -- 172.21.15.3:0/3064768590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564516da7f10 con 0x564516da3b00 2024-03-21T08:36:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9981600640 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516db7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:36:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.811+0000 7f9973e00640 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da3b00 0x564516d9c9f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9964003ba0 tx=0x7f9964005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bd224f133fbd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.812+0000 7f9971200640 1 -- 172.21.15.3:0/3064768590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964011470 con 0x564516da3b00 2024-03-21T08:36:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.812+0000 7f9971200640 1 -- 172.21.15.3:0/3064768590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9964009cf0 con 0x564516da3b00 2024-03-21T08:36:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.812+0000 7f9971200640 1 -- 172.21.15.3:0/3064768590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99640052c0 con 0x564516da3b00 2024-03-21T08:36:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 -- 172.21.15.3:0/3064768590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da3b00 msgr2=0x564516d9c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da3b00 0x564516d9c9f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9964003ba0 tx=0x7f9964005f90 comp rx=0 tx=0).stop 2024-03-21T08:36:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 -- 172.21.15.3:0/3064768590 shutdown_connections 2024-03-21T08:36:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516dac8d0 0x564516db7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da3b00 0x564516d9c9f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 --2- 172.21.15.3:0/3064768590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516da31e0 0x564516dac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 -- 172.21.15.3:0/3064768590 >> 172.21.15.3:0/3064768590 conn(0x564516d9db00 msgr2=0x564516d9c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 -- 172.21.15.3:0/3064768590 shutdown_connections 2024-03-21T08:36:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 -- 172.21.15.3:0/3064768590 wait complete. 2024-03-21T08:36:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.813+0000 7f998641e740 1 -- start start 2024-03-21T08:36:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 1 Processor -- start 2024-03-21T08:36:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516dac8d0 0x564516e16f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1b3720 con 0x564516da31e0 2024-03-21T08:36:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1b4a20 con 0x564516da3b00 2024-03-21T08:36:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1b3d60 con 0x564516dac8d0 2024-03-21T08:36:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f998641e740 10 client.0 _ll_register_callbacks cb 0x7f9964009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:36:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.814+0000 7f9980c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9973e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9980c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35450/0 (socket says 172.21.15.3:35450) 2024-03-21T08:36:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9980c00640 1 -- 172.21.15.3:0/3936131000 learned_addr learned my addr 172.21.15.3:0/3936131000 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9973e00640 1 -- 172.21.15.3:0/3936131000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516dac8d0 msgr2=0x564516e16f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9973e00640 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516dac8d0 0x564516e16f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9973e00640 1 -- 172.21.15.3:0/3936131000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 msgr2=0x564516da6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9973e00640 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9973e00640 1 -- 172.21.15.3:0/3936131000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564516e17570 con 0x564516da3b00 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f9973e00640 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da6d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x564516d9ea40 tx=0x7f99640114f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aac8b3bc121b96c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.815+0000 7f996a000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.819+0000 7f996a000640 1 -- 172.21.15.3:0/3936131000 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964018040 con 0x564516da3b00 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.819+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9964009cf0 con 0x564516da3b00 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.819+0000 7f996a000640 1 -- 172.21.15.3:0/3936131000 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f99640037f0 con 0x564516da3b00 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.819+0000 7f996a000640 1 -- 172.21.15.3:0/3936131000 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99640052c0 con 0x564516da3b00 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.819+0000 7f998641e740 10 client.36383 fetch_fsmap learned FSMap version 30 2024-03-21T08:36:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.819+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564516e18880 con 0x564516da3b00 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f996a000640 1 -- 172.21.15.3:0/3936131000 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9964009cf0 con 0x564516da3b00 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f996a000640 1 -- 172.21.15.3:0/3936131000 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9964033030 con 0x564516da3b00 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f998641e740 10 client.36383 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f998641e740 10 client.36383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f998641e740 20 client.36383 populate_metadata read hostname 'smithi003' 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f998641e740 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516e18d70 0x564516e1b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f998641e740 4 client.36383 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:36:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564516d0f0c0 con 0x564516e18d70 2024-03-21T08:36:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.820+0000 7f9980c00640 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516e18d70 0x564516e1b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:36:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.821+0000 7f9980c00640 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516e18d70 0x564516e1b190 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4184fa9bfce6f30 server_cookie=1c30e8e22d4f978b in_seq=0 out_seq=0 2024-03-21T08:36:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.821+0000 7f996a000640 10 client.36383 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f996a000640 1 -- 172.21.15.3:0/3936131000 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564516d0f0c0 con 0x564516e18d70 2024-03-21T08:36:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f996a000640 10 client.36383 handle_command_reply: tid=0 2024-03-21T08:36:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 client.36383 shutdown 2024-03-21T08:36:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 msgr2=0x564516da6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da6d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x564516d9ea40 tx=0x7f99640114f0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 shutdown_connections 2024-03-21T08:36:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x564516e18d70 0x564516e1b190 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564516dac8d0 0x564516e16f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564516da3b00 0x564516da6d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 --2- 172.21.15.3:0/3936131000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564516da31e0 0x564516da6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f996a000640 0 client.36383 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:36:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.878+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 >> 172.21.15.3:0/3936131000 conn(0x564516d9db00 msgr2=0x564516daa1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.879+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 shutdown_connections 2024-03-21T08:36:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.879+0000 7f998641e740 1 -- 172.21.15.3:0/3936131000 wait complete. 2024-03-21T08:36:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.879+0000 7f998641e740 20 client.36383 trim_cache size 0 max 16384 2024-03-21T08:36:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:36:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:36:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:36:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:36:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99600a3070 msgr2=0x7f99600a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99600a3070 0x7f99600a5530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f996c006010 tx=0x7f996c02c6e0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c10dab0 msgr2=0x7f997c1a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c10dab0 0x7f997c1a2b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9974009fd0 tx=0x7f997400b260 comp rx=0 tx=0).stop 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 shutdown_connections 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f997c10dab0 0x7f997c1a2b80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f99600a3070 0x7f99600a5530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f997c108460 0x7f997c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 --2- 172.21.15.3:0/3021102939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f997c107aa0 0x7f997c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 >> 172.21.15.3:0/3021102939 conn(0x7f997c077220 msgr2=0x7f997c077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:36:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 shutdown_connections 2024-03-21T08:36:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:36:49.880+0000 7f9982000640 1 -- 172.21.15.3:0/3021102939 wait complete. 2024-03-21T08:36:50.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:49 smithi084 ceph-mon[35477]: pgmap v152: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-03-21T08:36:50.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:49 smithi171 ceph-mon[35388]: pgmap v152: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-03-21T08:36:50.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:49 smithi003 ceph-mon[31675]: pgmap v152: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-03-21T08:36:52.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:51 smithi084 ceph-mon[35477]: pgmap v153: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-03-21T08:36:52.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:51 smithi171 ceph-mon[35388]: pgmap v153: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-03-21T08:36:52.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:51 smithi003 ceph-mon[31675]: pgmap v153: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-03-21T08:36:54.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:53 smithi084 ceph-mon[35477]: pgmap v154: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-03-21T08:36:54.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:53 smithi171 ceph-mon[35388]: pgmap v154: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-03-21T08:36:54.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:53 smithi003 ceph-mon[31675]: pgmap v154: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:36:54.971 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-03-21T08:36:54.972 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:36:54.972 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-03-21T08:36:54.972 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:36:54.972 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T08:36:54.972 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-03-21T08:36:54.978 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-03-21T08:36:55.025 INFO:tasks.workunit.client.0.smithi003.stderr:+ make -j4 2024-03-21T08:36:55.110 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize -cfi 2024-03-21T08:36:55.932 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: putting auxiliary files in '.'. 2024-03-21T08:36:55.932 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file './config.guess' 2024-03-21T08:36:55.977 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file './config.sub' 2024-03-21T08:36:56.045 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file './install-sh' 2024-03-21T08:36:56.106 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file './ltmain.sh' 2024-03-21T08:36:56.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:55 smithi084 ceph-mon[35477]: pgmap v155: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-03-21T08:36:56.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:55 smithi171 ceph-mon[35388]: pgmap v155: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-03-21T08:36:56.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:55 smithi003 ceph-mon[31675]: pgmap v155: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-03-21T08:36:56.215 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-03-21T08:36:56.215 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-03-21T08:36:56.247 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-03-21T08:36:56.276 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-03-21T08:36:56.309 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-03-21T08:36:56.343 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-03-21T08:36:56.373 INFO:tasks.workunit.client.0.smithi003.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-03-21T08:36:56.379 INFO:tasks.workunit.client.0.smithi003.stdout:cp include/install-sh . 2024-03-21T08:36:56.405 INFO:tasks.workunit.client.0.smithi003.stdout:aclocal -I m4 2024-03-21T08:36:58.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:57 smithi084 ceph-mon[35477]: pgmap v156: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-03-21T08:36:58.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:57 smithi171 ceph-mon[35388]: pgmap v156: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-03-21T08:36:58.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:57 smithi003 ceph-mon[31675]: pgmap v156: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-03-21T08:36:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:autoheader 2024-03-21T08:37:00.110 INFO:tasks.workunit.client.0.smithi003.stdout:autoconf 2024-03-21T08:37:00.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:36:59 smithi084 ceph-mon[35477]: pgmap v157: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-03-21T08:37:00.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:36:59 smithi171 ceph-mon[35388]: pgmap v157: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-03-21T08:37:00.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:36:59 smithi003 ceph-mon[31675]: pgmap v157: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-03-21T08:37:00.917 INFO:tasks.workunit.client.0.smithi003.stdout:./configure \ 2024-03-21T08:37:00.918 INFO:tasks.workunit.client.0.smithi003.stdout: --libexecdir=/usr/lib \ 2024-03-21T08:37:00.918 INFO:tasks.workunit.client.0.smithi003.stdout: --exec_prefix=/var/lib 2024-03-21T08:37:01.544 INFO:tasks.workunit.client.0.smithi003.stdout:checking build system type... x86_64-pc-linux-gnu 2024-03-21T08:37:01.545 INFO:tasks.workunit.client.0.smithi003.stdout:checking host system type... x86_64-pc-linux-gnu 2024-03-21T08:37:01.552 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to print strings... printf 2024-03-21T08:37:01.554 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... gcc 2024-03-21T08:37:01.982 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether the C compiler works... yes 2024-03-21T08:37:01.983 INFO:tasks.workunit.client.0.smithi003.stdout:checking for C compiler default output file name... a.out 2024-03-21T08:37:02.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:01 smithi084 ceph-mon[35477]: pgmap v158: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-03-21T08:37:02.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:01 smithi171 ceph-mon[35388]: pgmap v158: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-03-21T08:37:02.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:01 smithi003 ceph-mon[31675]: pgmap v158: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2024-03-21T08:37:02.198 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of executables... 2024-03-21T08:37:02.406 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are cross compiling... no 2024-03-21T08:37:02.599 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of object files... o 2024-03-21T08:37:02.736 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... yes 2024-03-21T08:37:02.856 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... yes 2024-03-21T08:37:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 -- 172.21.15.3:0/4117818585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 msgr2=0x7f60d010db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 --2- 172.21.15.3:0/4117818585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d010db00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009900 tx=0x7f60b802bc30 comp rx=0 tx=0).stop 2024-03-21T08:37:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 -- 172.21.15.3:0/4117818585 shutdown_connections 2024-03-21T08:37:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 --2- 172.21.15.3:0/4117818585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60d010e040 0x7f60d0110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 --2- 172.21.15.3:0/4117818585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d010db00 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 --2- 172.21.15.3:0/4117818585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0068f40 0x7f60d0069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 -- 172.21.15.3:0/4117818585 >> 172.21.15.3:0/4117818585 conn(0x7f60d0077690 msgr2=0x7f60d0077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 -- 172.21.15.3:0/4117818585 shutdown_connections 2024-03-21T08:37:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.025+0000 7f60d6400640 1 -- 172.21.15.3:0/4117818585 wait complete. 2024-03-21T08:37:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.026+0000 7f60d6400640 1 Processor -- start 2024-03-21T08:37:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.026+0000 7f60d6400640 1 -- start start 2024-03-21T08:37:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60d6400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0068f40 0x7f60d019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60d6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60d6400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60d010e040 0x7f60d019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60d6400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0113ad0 con 0x7f60d0069860 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60d6400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0113950 con 0x7f60d0068f40 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60d6400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0113c50 con 0x7f60d010e040 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44394/0 (socket says 172.21.15.3:44394) 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60c7e00640 1 -- 172.21.15.3:0/1223927528 learned_addr learned my addr 172.21.15.3:0/1223927528 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60cf400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0068f40 0x7f60d019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.027+0000 7f60cfe00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60d010e040 0x7f60d019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60c7e00640 1 -- 172.21.15.3:0/1223927528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60d010e040 msgr2=0x7f60d019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60c7e00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60d010e040 0x7f60d019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60c7e00640 1 -- 172.21.15.3:0/1223927528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0068f40 msgr2=0x7f60d019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60c7e00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0068f40 0x7f60d019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60c7e00640 1 -- 172.21.15.3:0/1223927528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d019f0a0 con 0x7f60d0069860 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60cf400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0068f40 0x7f60d019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60cfe00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60d010e040 0x7f60d019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60c7e00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d019a5f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f60b8005940 tx=0x7f60b802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60ccc00640 1 -- 172.21.15.3:0/1223927528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8043020 con 0x7f60d0069860 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60ccc00640 1 -- 172.21.15.3:0/1223927528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60b802cbe0 con 0x7f60d0069860 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d019f330 con 0x7f60d0069860 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.028+0000 7f60ccc00640 1 -- 172.21.15.3:0/1223927528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8003bf0 con 0x7f60d0069860 2024-03-21T08:37:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.029+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d01a6c20 con 0x7f60d0069860 2024-03-21T08:37:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.032+0000 7f60d921e740 1 Processor -- start 2024-03-21T08:37:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.032+0000 7f60ccc00640 1 -- 172.21.15.3:0/1223927528 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f60b8046020 con 0x7f60d0069860 2024-03-21T08:37:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.032+0000 7f60d921e740 1 -- start start 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.032+0000 7f60d921e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60d921e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c3c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60d921e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60d921e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b8bf53b0 con 0x5649b8c338f0 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60d921e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b8b97830 con 0x5649b8c32fd0 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60d921e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b8b405d0 con 0x5649b8c3c6c0 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c3c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c3c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44398/0 (socket says 172.21.15.3:44398) 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 -- 172.21.15.3:0/912837461 learned_addr learned my addr 172.21.15.3:0/912837461 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cfe00640 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60c7e00640 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 -- 172.21.15.3:0/912837461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 msgr2=0x5649b8c3eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 -- 172.21.15.3:0/912837461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 msgr2=0x5649b8c333b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c333b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.033+0000 7f60cf400640 1 -- 172.21.15.3:0/912837461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649b8b582c0 con 0x5649b8c338f0 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.034+0000 7f60ccc00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f60b40a3100 0x7f60b40a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.034+0000 7f60cf400640 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c3c180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f60c000bd40 tx=0x7f60c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b010a25867e4f8df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.034+0000 7f60ccc00640 1 -- 172.21.15.3:0/1223927528 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f60b80f1db0 con 0x7f60d0069860 2024-03-21T08:37:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.034+0000 7f60c4c00640 1 -- 172.21.15.3:0/912837461 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0015070 con 0x5649b8c338f0 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.034+0000 7f60c4c00640 1 -- 172.21.15.3:0/912837461 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60c000be30 con 0x5649b8c338f0 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.034+0000 7f60c4c00640 1 -- 172.21.15.3:0/912837461 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0011930 con 0x5649b8c338f0 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.034+0000 7f60cfe00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f60b40a3100 0x7f60b40a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 -- 172.21.15.3:0/912837461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 msgr2=0x5649b8c3c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60cfe00640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f60b40a3100 0x7f60b40a55c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005fd0 tx=0x7f60c8009680 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c3c180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f60c000bd40 tx=0x7f60c000f730 comp rx=0 tx=0).stop 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 -- 172.21.15.3:0/912837461 shutdown_connections 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c3c180 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 --2- 172.21.15.3:0/912837461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c333b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 -- 172.21.15.3:0/912837461 >> 172.21.15.3:0/912837461 conn(0x5649b8c2d8f0 msgr2=0x5649b8c2dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 -- 172.21.15.3:0/912837461 shutdown_connections 2024-03-21T08:37:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.035+0000 7f60d921e740 1 -- 172.21.15.3:0/912837461 wait complete. 2024-03-21T08:37:03.041 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-21T08:37:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.036+0000 7f60d921e740 1 -- start start 2024-03-21T08:37:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.036+0000 7f60d921e740 1 Processor -- start 2024-03-21T08:37:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60d921e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60d921e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60d921e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60d921e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b8b53a20 con 0x5649b8c338f0 2024-03-21T08:37:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60d921e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b8b938d0 con 0x5649b8c32fd0 2024-03-21T08:37:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60d921e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b8c3f2f0 con 0x5649b8c3c6c0 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60d921e740 10 client.0 _ll_register_callbacks cb 0x5649b8b901c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c39be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44410/0 (socket says 172.21.15.3:44410) 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60c7e00640 1 -- 172.21.15.3:0/669585843 learned_addr learned my addr 172.21.15.3:0/669585843 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.037+0000 7f60cf400640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60cfe00640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60c7e00640 1 -- 172.21.15.3:0/669585843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 msgr2=0x5649b8c3a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60c7e00640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60c7e00640 1 -- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 msgr2=0x5649b8c396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60c7e00640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60c7e00640 1 -- 172.21.15.3:0/669585843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649b88ac210 con 0x5649b8c338f0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60cfe00640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60c7e00640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c39be0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f60b80ba2f0 tx=0x7f60b8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dd1f7677087cc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60bd600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60bd600640 1 -- 172.21.15.3:0/669585843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80bd9d0 con 0x5649b8c338f0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60bd600640 1 -- 172.21.15.3:0/669585843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f60b80ba500 con 0x5649b8c338f0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.038+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60c000be30 con 0x5649b8c338f0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60bd600640 1 -- 172.21.15.3:0/669585843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b80bd7f0 con 0x5649b8c338f0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 10 client.16665 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5649b8978620 con 0x5649b8c338f0 2024-03-21T08:37:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60bd600640 1 -- 172.21.15.3:0/669585843 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60c000be30 con 0x5649b8c338f0 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60bd600640 1 -- 172.21.15.3:0/669585843 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f60b80be0f0 con 0x5649b8c338f0 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 10 client.16665 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 20 client.16665 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8c4c6c0 0x5649b8c4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 4 client.16665 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.039+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649b8ba4d70 con 0x5649b8c4c6c0 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.040+0000 7f60cf400640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8c4c6c0 0x5649b8c4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.040+0000 7f60cf400640 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8c4c6c0 0x5649b8c4eb00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05865b2b59a7f2c server_cookie=31800f1256bb88ec in_seq=0 out_seq=0 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.040+0000 7f60bd600640 10 client.16665 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60bd600640 1 -- 172.21.15.3:0/669585843 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5649b8ba4d70 con 0x5649b8c4c6c0 2024-03-21T08:37:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60bd600640 10 client.16665 handle_command_reply: tid=0 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 client.16665 shutdown 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 msgr2=0x5649b8c39be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c39be0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f60b80ba2f0 tx=0x7f60b8009620 comp rx=0 tx=0).stop 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 shutdown_connections 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8c4c6c0 0x5649b8c4eb00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c3a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c39be0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 --2- 172.21.15.3:0/669585843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60bd600640 0 client.16665 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 >> 172.21.15.3:0/669585843 conn(0x5649b8c2d8f0 msgr2=0x5649b8c2f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 shutdown_connections 2024-03-21T08:37:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 1 -- 172.21.15.3:0/669585843 wait complete. 2024-03-21T08:37:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.042+0000 7f60d921e740 20 client.16665 trim_cache size 0 max 16384 2024-03-21T08:37:03.055 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-03-21T08:37:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 Processor -- start 2024-03-21T08:37:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 -- start start 2024-03-21T08:37:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c3bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c37f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c47a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d01b3370 con 0x5649b8c338f0 2024-03-21T08:37:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d01b0e10 con 0x5649b8c32fd0 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60d921e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d01aeeb0 con 0x5649b8c3c6c0 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c37f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c37f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44420/0 (socket says 172.21.15.3:44420) 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60c7e00640 1 -- 172.21.15.3:0/2846681547 learned_addr learned my addr 172.21.15.3:0/2846681547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.060+0000 7f60cf400640 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c3bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60cfe00640 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c47a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60c7e00640 1 -- 172.21.15.3:0/2846681547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 msgr2=0x5649b8c47a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60c7e00640 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c47a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60c7e00640 1 -- 172.21.15.3:0/2846681547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 msgr2=0x5649b8c3bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60c7e00640 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c3bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60c7e00640 1 -- 172.21.15.3:0/2846681547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649b8c343f0 con 0x5649b8c338f0 2024-03-21T08:37:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60cf400640 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c3bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60c7e00640 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c37f70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f60b80ef6e0 tx=0x7f60b80c6450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39750823feff84a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.061+0000 7f60bfe00640 1 -- 172.21.15.3:0/2846681547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80a1070 con 0x5649b8c338f0 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60bfe00640 1 -- 172.21.15.3:0/2846681547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60b80be590 con 0x5649b8c338f0 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60bfe00640 1 -- 172.21.15.3:0/2846681547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80ba500 con 0x5649b8c338f0 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60d921e740 1 -- 172.21.15.3:0/2846681547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 msgr2=0x5649b8c37f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60d921e740 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c37f70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f60b80ef6e0 tx=0x7f60b80c6450 comp rx=0 tx=0).stop 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60d921e740 1 -- 172.21.15.3:0/2846681547 shutdown_connections 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60d921e740 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3c6c0 0x5649b8c47a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60d921e740 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c338f0 0x5649b8c37f70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60d921e740 1 --2- 172.21.15.3:0/2846681547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c3bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.062+0000 7f60d921e740 1 -- 172.21.15.3:0/2846681547 >> 172.21.15.3:0/2846681547 conn(0x5649b8c2d8f0 msgr2=0x5649b8c322b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.063+0000 7f60d921e740 1 -- 172.21.15.3:0/2846681547 shutdown_connections 2024-03-21T08:37:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.063+0000 7f60d921e740 1 -- 172.21.15.3:0/2846681547 wait complete. 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.063+0000 7f60d921e740 1 -- start start 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.063+0000 7f60d921e740 1 Processor -- start 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60d921e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c43b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60d921e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c3c6c0 0x5649b8c440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60d921e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3ffa0 0x5649b8c40420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60d921e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d01af4b0 con 0x5649b8c3c6c0 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60d921e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d01b07b0 con 0x5649b8c32fd0 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60d921e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d01afaf0 con 0x5649b8c3ffa0 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60d921e740 10 client.0 _ll_register_callbacks cb 0x7f60b80057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c3c6c0 0x5649b8c440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c3c6c0 0x5649b8c440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44426/0 (socket says 172.21.15.3:44426) 2024-03-21T08:37:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60c7e00640 1 -- 172.21.15.3:0/2043411010 learned_addr learned my addr 172.21.15.3:0/2043411010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.064+0000 7f60cf400640 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c43b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60cfe00640 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3ffa0 0x5649b8c40420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60c7e00640 1 -- 172.21.15.3:0/2043411010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3ffa0 msgr2=0x5649b8c40420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60c7e00640 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3ffa0 0x5649b8c40420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60c7e00640 1 -- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 msgr2=0x5649b8c43b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60c7e00640 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c43b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60c7e00640 1 -- 172.21.15.3:0/2043411010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649b8c40a90 con 0x5649b8c3c6c0 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60c7e00640 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c3c6c0 0x5649b8c440b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f60b8005ec0 tx=0x7f60b8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d08c5f061e8f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60bd600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:03.071 INFO:tasks.workunit.client.0.smithi003.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-03-21T08:37:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60bd600640 1 -- 172.21.15.3:0/2043411010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80a1070 con 0x5649b8c3c6c0 2024-03-21T08:37:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60bd600640 1 -- 172.21.15.3:0/2043411010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f60b80bd2e0 con 0x5649b8c3c6c0 2024-03-21T08:37:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.065+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60b80be590 con 0x5649b8c3c6c0 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60bd600640 1 -- 172.21.15.3:0/2043411010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b80bd9d0 con 0x5649b8c3c6c0 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 10 client.16671 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5649b8c40a90 con 0x5649b8c3c6c0 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60bd600640 1 -- 172.21.15.3:0/2043411010 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60b80be590 con 0x5649b8c3c6c0 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60bd600640 1 -- 172.21.15.3:0/2043411010 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f60b80a4070 con 0x5649b8c3c6c0 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 10 client.16671 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 20 client.16671 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8ca9030 0x5649b8cab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 4 client.16671 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:37:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.066+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649b8b9f0c0 con 0x5649b8ca9030 2024-03-21T08:37:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.067+0000 7f60cf400640 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8ca9030 0x5649b8cab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.068+0000 7f60cf400640 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8ca9030 0x5649b8cab450 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d50d854b82f92d server_cookie=18e95a659cb6aeb1 in_seq=0 out_seq=0 2024-03-21T08:37:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.068+0000 7f60bd600640 10 client.16671 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:03.074 INFO:tasks.workunit.client.0.smithi003.stdout:checking for egrep... /usr/bin/grep -E 2024-03-21T08:37:03.075 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fgrep... /usr/bin/grep -F 2024-03-21T08:37:03.086 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ld used by gcc... /usr/bin/ld 2024-03-21T08:37:03.092 INFO:tasks.workunit.client.0.smithi003.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-03-21T08:37:03.114 INFO:tasks.workunit.client.0.smithi003.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-03-21T08:37:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60bd600640 1 -- 172.21.15.3:0/2043411010 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649b8b9f0c0 con 0x5649b8ca9030 2024-03-21T08:37:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60bd600640 10 client.16671 handle_command_reply: tid=0 2024-03-21T08:37:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 client.16671 shutdown 2024-03-21T08:37:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c3c6c0 msgr2=0x5649b8c440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c3c6c0 0x5649b8c440b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f60b8005ec0 tx=0x7f60b8009620 comp rx=0 tx=0).stop 2024-03-21T08:37:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 shutdown_connections 2024-03-21T08:37:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5649b8ca9030 0x5649b8cab450 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5649b8c3ffa0 0x5649b8c40420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649b8c3c6c0 0x5649b8c440b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 --2- 172.21.15.3:0/2043411010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5649b8c32fd0 0x5649b8c43b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 >> 172.21.15.3:0/2043411010 conn(0x5649b8c2d8f0 msgr2=0x5649b8c37e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60bd600640 0 client.16671 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 shutdown_connections 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 1 -- 172.21.15.3:0/2043411010 wait complete. 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.172+0000 7f60d921e740 20 client.16671 trim_cache size 0 max 16384 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:37:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:37:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:37:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f60b40a3100 msgr2=0x7f60b40a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f60b40a3100 0x7f60b40a55c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005fd0 tx=0x7f60c8009680 comp rx=0 tx=0).stop 2024-03-21T08:37:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 msgr2=0x7f60d019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d019a5f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f60b8005940 tx=0x7f60b802c7e0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 shutdown_connections 2024-03-21T08:37:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f60b40a3100 0x7f60b40a55c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60d010e040 0x7f60d019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60d0069860 0x7f60d019a5f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 --2- 172.21.15.3:0/1223927528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60d0068f40 0x7f60d019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.173+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 >> 172.21.15.3:0/1223927528 conn(0x7f60d0077690 msgr2=0x7f60d010f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.174+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 shutdown_connections 2024-03-21T08:37:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:03.174+0000 7f60d6400640 1 -- 172.21.15.3:0/1223927528 wait complete. 2024-03-21T08:37:04.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:03 smithi084 ceph-mon[35477]: pgmap v159: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-03-21T08:37:04.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:03 smithi171 ceph-mon[35388]: pgmap v159: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-03-21T08:37:04.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:03 smithi003 ceph-mon[31675]: pgmap v159: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-03-21T08:37:04.259 INFO:tasks.workunit.client.0.smithi003.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-03-21T08:37:04.261 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether ln -s works... yes 2024-03-21T08:37:04.436 INFO:tasks.workunit.client.0.smithi003.stdout:checking the maximum length of command line arguments... 1572864 2024-03-21T08:37:04.447 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-03-21T08:37:04.448 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-03-21T08:37:04.449 INFO:tasks.workunit.client.0.smithi003.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-03-21T08:37:04.451 INFO:tasks.workunit.client.0.smithi003.stdout:checking for objdump... objdump 2024-03-21T08:37:04.453 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to recognize dependent libraries... pass_all 2024-03-21T08:37:04.454 INFO:tasks.workunit.client.0.smithi003.stdout:checking for dlltool... no 2024-03-21T08:37:04.455 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-03-21T08:37:04.457 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ar... ar 2024-03-21T08:37:04.798 INFO:tasks.workunit.client.0.smithi003.stdout:checking for archiver @FILE support... @ 2024-03-21T08:37:04.802 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strip... strip 2024-03-21T08:37:04.806 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ranlib... ranlib 2024-03-21T08:37:04.810 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gawk... gawk 2024-03-21T08:37:05.384 INFO:tasks.workunit.client.0.smithi003.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-03-21T08:37:05.394 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sysroot... no 2024-03-21T08:37:05.452 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a working dd... /usr/bin/dd 2024-03-21T08:37:05.501 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-03-21T08:37:05.629 INFO:tasks.workunit.client.0.smithi003.stdout:checking for mt... no 2024-03-21T08:37:05.705 INFO:tasks.workunit.client.0.smithi003.stdout:checking if : is a manifest tool... no 2024-03-21T08:37:05.998 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to run the C preprocessor... gcc -E 2024-03-21T08:37:06.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:05 smithi171 ceph-mon[35388]: pgmap v160: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 699 KiB/s wr, 75 op/s 2024-03-21T08:37:06.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:05 smithi003 ceph-mon[31675]: pgmap v160: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 699 KiB/s wr, 75 op/s 2024-03-21T08:37:06.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:05 smithi084 ceph-mon[35477]: pgmap v160: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 699 KiB/s wr, 75 op/s 2024-03-21T08:37:06.809 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... yes 2024-03-21T08:37:06.974 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/types.h... yes 2024-03-21T08:37:07.173 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/stat.h... yes 2024-03-21T08:37:07.383 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdlib.h... yes 2024-03-21T08:37:07.555 INFO:tasks.workunit.client.0.smithi003.stdout:checking for string.h... yes 2024-03-21T08:37:07.719 INFO:tasks.workunit.client.0.smithi003.stdout:checking for memory.h... yes 2024-03-21T08:37:07.874 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strings.h... yes 2024-03-21T08:37:08.047 INFO:tasks.workunit.client.0.smithi003.stdout:checking for inttypes.h... yes 2024-03-21T08:37:08.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:07 smithi003 ceph-mon[31675]: pgmap v161: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 871 KiB/s wr, 122 op/s 2024-03-21T08:37:08.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:07 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:37:08.177 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:07 smithi084 ceph-mon[35477]: pgmap v161: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 871 KiB/s wr, 122 op/s 2024-03-21T08:37:08.177 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:07 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:37:08.237 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... yes 2024-03-21T08:37:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:07 smithi171 ceph-mon[35388]: pgmap v161: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 871 KiB/s wr, 122 op/s 2024-03-21T08:37:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:07 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:37:08.454 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... yes 2024-03-21T08:37:08.650 INFO:tasks.workunit.client.0.smithi003.stdout:checking for dlfcn.h... yes 2024-03-21T08:37:08.680 INFO:tasks.workunit.client.0.smithi003.stdout:checking for objdir... .libs 2024-03-21T08:37:09.148 INFO:tasks.workunit.client.0.smithi003.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-03-21T08:37:09.150 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-03-21T08:37:09.344 INFO:tasks.workunit.client.0.smithi003.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-03-21T08:37:09.495 INFO:tasks.workunit.client.0.smithi003.stdout:checking if gcc static flag -static works... no 2024-03-21T08:37:09.795 INFO:tasks.workunit.client.0.smithi003.stdout:checking if gcc supports -c -o file.o... yes 2024-03-21T08:37:09.796 INFO:tasks.workunit.client.0.smithi003.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-03-21T08:37:09.827 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-03-21T08:37:10.106 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether -lc should be explicitly linked in... no 2024-03-21T08:37:10.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:09 smithi003 ceph-mon[31675]: pgmap v162: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 416 KiB/s wr, 75 op/s 2024-03-21T08:37:10.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:09 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:37:10.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:09 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:37:10.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:09 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:37:10.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:09 smithi084 ceph-mon[35477]: pgmap v162: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 416 KiB/s wr, 75 op/s 2024-03-21T08:37:10.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:09 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:37:10.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:09 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:37:10.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:09 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:37:10.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:09 smithi171 ceph-mon[35388]: pgmap v162: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 416 KiB/s wr, 75 op/s 2024-03-21T08:37:10.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:09 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:37:10.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:09 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:37:10.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:09 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:37:10.445 INFO:tasks.workunit.client.0.smithi003.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-03-21T08:37:10.445 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to hardcode library paths into programs... immediate 2024-03-21T08:37:10.450 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether stripping libraries is possible... yes 2024-03-21T08:37:10.451 INFO:tasks.workunit.client.0.smithi003.stdout:checking if libtool supports shared libraries... yes 2024-03-21T08:37:10.452 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether to build shared libraries... yes 2024-03-21T08:37:10.452 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether to build static libraries... yes 2024-03-21T08:37:10.477 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... (cached) gcc 2024-03-21T08:37:10.638 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-21T08:37:10.639 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-21T08:37:10.641 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-21T08:37:10.644 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gmake... /usr/bin/gmake 2024-03-21T08:37:10.644 INFO:tasks.workunit.client.0.smithi003.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-03-21T08:37:10.645 INFO:tasks.workunit.client.0.smithi003.stdout:checking for tar... /bin/tar 2024-03-21T08:37:10.647 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gzip... /bin/gzip 2024-03-21T08:37:10.717 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc -MM is supported... yes 2024-03-21T08:37:10.721 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sort... /bin/sort 2024-03-21T08:37:10.723 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether ln -s works... yes 2024-03-21T08:37:10.727 INFO:tasks.workunit.client.0.smithi003.stdout:checking for rpm... /bin/rpm 2024-03-21T08:37:10.742 INFO:tasks.workunit.client.0.smithi003.stdout:checking for rpmbuild... no 2024-03-21T08:37:10.744 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... (cached) yes 2024-03-21T08:37:11.141 INFO:tasks.workunit.client.0.smithi003.stdout:checking assert.h usability... yes 2024-03-21T08:37:11.213 INFO:tasks.workunit.client.0.smithi003.stdout:checking assert.h presence... yes 2024-03-21T08:37:11.214 INFO:tasks.workunit.client.0.smithi003.stdout:checking for assert.h... yes 2024-03-21T08:37:11.425 INFO:tasks.workunit.client.0.smithi003.stdout:checking bstring.h usability... no 2024-03-21T08:37:11.557 INFO:tasks.workunit.client.0.smithi003.stdout:checking bstring.h presence... no 2024-03-21T08:37:11.559 INFO:tasks.workunit.client.0.smithi003.stdout:checking for bstring.h... no 2024-03-21T08:37:11.733 INFO:tasks.workunit.client.0.smithi003.stdout:checking libgen.h usability... yes 2024-03-21T08:37:11.806 INFO:tasks.workunit.client.0.smithi003.stdout:checking libgen.h presence... yes 2024-03-21T08:37:11.807 INFO:tasks.workunit.client.0.smithi003.stdout:checking for libgen.h... yes 2024-03-21T08:37:12.033 INFO:tasks.workunit.client.0.smithi003.stdout:checking dirent.h usability... yes 2024-03-21T08:37:12.109 INFO:tasks.workunit.client.0.smithi003.stdout:checking dirent.h presence... yes 2024-03-21T08:37:12.110 INFO:tasks.workunit.client.0.smithi003.stdout:checking for dirent.h... yes 2024-03-21T08:37:12.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:11 smithi003 ceph-mon[31675]: pgmap v163: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 416 KiB/s wr, 75 op/s 2024-03-21T08:37:12.354 INFO:tasks.workunit.client.0.smithi003.stdout:checking errno.h usability... yes 2024-03-21T08:37:12.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:11 smithi084 ceph-mon[35477]: pgmap v163: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 416 KiB/s wr, 75 op/s 2024-03-21T08:37:12.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:11 smithi171 ceph-mon[35388]: pgmap v163: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 416 KiB/s wr, 75 op/s 2024-03-21T08:37:12.426 INFO:tasks.workunit.client.0.smithi003.stdout:checking errno.h presence... yes 2024-03-21T08:37:12.427 INFO:tasks.workunit.client.0.smithi003.stdout:checking for errno.h... yes 2024-03-21T08:37:12.675 INFO:tasks.workunit.client.0.smithi003.stdout:checking malloc.h usability... yes 2024-03-21T08:37:12.758 INFO:tasks.workunit.client.0.smithi003.stdout:checking malloc.h presence... yes 2024-03-21T08:37:12.759 INFO:tasks.workunit.client.0.smithi003.stdout:checking for malloc.h... yes 2024-03-21T08:37:12.999 INFO:tasks.workunit.client.0.smithi003.stdout:checking uuid.h usability... no 2024-03-21T08:37:13.156 INFO:tasks.workunit.client.0.smithi003.stdout:checking uuid.h presence... no 2024-03-21T08:37:13.158 INFO:tasks.workunit.client.0.smithi003.stdout:checking for uuid.h... no 2024-03-21T08:37:13.334 INFO:tasks.workunit.client.0.smithi003.stdout:checking uuid/uuid.h usability... yes 2024-03-21T08:37:13.412 INFO:tasks.workunit.client.0.smithi003.stdout:checking uuid/uuid.h presence... yes 2024-03-21T08:37:13.415 INFO:tasks.workunit.client.0.smithi003.stdout:checking for uuid/uuid.h... yes 2024-03-21T08:37:13.826 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/uuid.h usability... no 2024-03-21T08:37:13.970 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/uuid.h presence... no 2024-03-21T08:37:13.972 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/uuid.h... no 2024-03-21T08:37:14.144 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/file.h usability... yes 2024-03-21T08:37:14.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:13 smithi003 ceph-mon[31675]: pgmap v164: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 609 KiB/s wr, 108 op/s 2024-03-21T08:37:14.229 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/file.h presence... yes 2024-03-21T08:37:14.231 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/file.h... yes 2024-03-21T08:37:14.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:13 smithi084 ceph-mon[35477]: pgmap v164: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 609 KiB/s wr, 108 op/s 2024-03-21T08:37:14.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:13 smithi171 ceph-mon[35388]: pgmap v164: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 609 KiB/s wr, 108 op/s 2024-03-21T08:37:14.462 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/fcntl.h usability... yes 2024-03-21T08:37:14.520 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/fcntl.h presence... yes 2024-03-21T08:37:14.521 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/fcntl.h... yes 2024-03-21T08:37:14.775 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/syssgi.h usability... no 2024-03-21T08:37:14.907 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/syssgi.h presence... no 2024-03-21T08:37:14.909 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/syssgi.h... no 2024-03-21T08:37:15.128 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/param.h usability... yes 2024-03-21T08:37:15.219 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/param.h presence... yes 2024-03-21T08:37:15.221 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/param.h... yes 2024-03-21T08:37:15.310 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/stat.h... (cached) yes 2024-03-21T08:37:15.559 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/statvfs.h usability... yes 2024-03-21T08:37:15.643 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/statvfs.h presence... yes 2024-03-21T08:37:15.645 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/statvfs.h... yes 2024-03-21T08:37:15.873 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h usability... yes 2024-03-21T08:37:15.952 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h presence... yes 2024-03-21T08:37:15.954 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/time.h... yes 2024-03-21T08:37:16.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:15 smithi003 ceph-mon[31675]: pgmap v165: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 365 KiB/s wr, 79 op/s 2024-03-21T08:37:16.200 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/ioctl.h usability... yes 2024-03-21T08:37:16.281 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/ioctl.h presence... yes 2024-03-21T08:37:16.282 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/ioctl.h... yes 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.326+0000 7f58a4c00640 1 -- 172.21.15.3:0/224391131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a00743f0 msgr2=0x7f58a010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.326+0000 7f58a4c00640 1 --2- 172.21.15.3:0/224391131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a00743f0 0x7f58a010d950 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f588c009860 tx=0x7f588c02bc10 comp rx=0 tx=0).stop 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.327+0000 7f58a4c00640 1 -- 172.21.15.3:0/224391131 shutdown_connections 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.327+0000 7f58a4c00640 1 --2- 172.21.15.3:0/224391131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 0x7f58a01102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.327+0000 7f58a4c00640 1 --2- 172.21.15.3:0/224391131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a00743f0 0x7f58a010d950 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.327+0000 7f58a4c00640 1 --2- 172.21.15.3:0/224391131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58a0073ad0 0x7f58a0073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.327+0000 7f58a4c00640 1 -- 172.21.15.3:0/224391131 >> 172.21.15.3:0/224391131 conn(0x7f58a00fd120 msgr2=0x7f58a00ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.327+0000 7f58a4c00640 1 -- 172.21.15.3:0/224391131 shutdown_connections 2024-03-21T08:37:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.327+0000 7f58a4c00640 1 -- 172.21.15.3:0/224391131 wait complete. 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 Processor -- start 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 -- start start 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a0073ad0 0x7f58a019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58a00743f0 0x7f58a019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 0x7f58a019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0113ae0 con 0x7f58a010de90 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0113960 con 0x7f58a0073ad0 2024-03-21T08:37:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.328+0000 7f58a4c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0113c60 con 0x7f58a00743f0 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 0x7f58a019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 0x7f58a019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38756/0 (socket says 172.21.15.3:38756) 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 -- 172.21.15.3:0/2750732616 learned_addr learned my addr 172.21.15.3:0/2750732616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589de00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a0073ad0 0x7f58a019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589d400640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58a00743f0 0x7f58a019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 -- 172.21.15.3:0/2750732616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58a00743f0 msgr2=0x7f58a019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58a00743f0 0x7f58a019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 -- 172.21.15.3:0/2750732616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a0073ad0 msgr2=0x7f58a019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a0073ad0 0x7f58a019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589e800640 1 -- 172.21.15.3:0/2750732616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a019f090 con 0x7f58a010de90 2024-03-21T08:37:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.329+0000 7f589d400640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58a00743f0 0x7f58a019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:37:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.330+0000 7f589e800640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 0x7f58a019e960 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f589800efc0 tx=0x7f589800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.330+0000 7f5896a00640 1 -- 172.21.15.3:0/2750732616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898003970 con 0x7f58a010de90 2024-03-21T08:37:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.330+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a019f380 con 0x7f58a010de90 2024-03-21T08:37:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.330+0000 7f5896a00640 1 -- 172.21.15.3:0/2750732616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5898004020 con 0x7f58a010de90 2024-03-21T08:37:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.330+0000 7f5896a00640 1 -- 172.21.15.3:0/2750732616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800b940 con 0x7f58a010de90 2024-03-21T08:37:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.330+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a01a6c10 con 0x7f58a010de90 2024-03-21T08:37:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.335+0000 7f58a7a1a740 1 Processor -- start 2024-03-21T08:37:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.335+0000 7f5896a00640 1 -- 172.21.15.3:0/2750732616 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f589801d020 con 0x7f58a010de90 2024-03-21T08:37:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.335+0000 7f58a7a1a740 1 -- start start 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.335+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e43fd0 0x556ae0e443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e4d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0e063b0 con 0x556ae0e43fd0 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0da8830 con 0x556ae0e4d6c0 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0d515d0 con 0x556ae0e448f0 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f589d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e43fd0 0x556ae0e443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f589d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e43fd0 0x556ae0e443b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38760/0 (socket says 172.21.15.3:38760) 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f589d400640 1 -- 172.21.15.3:0/3889075364 learned_addr learned my addr 172.21.15.3:0/3889075364 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f589de00640 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.336+0000 7f589d400640 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.337+0000 7f589d400640 1 -- 172.21.15.3:0/3889075364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 msgr2=0x556ae0e4d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.337+0000 7f589d400640 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e4d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.337+0000 7f589d400640 1 -- 172.21.15.3:0/3889075364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e4d6c0 msgr2=0x556ae0e4fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.337+0000 7f589d400640 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.337+0000 7f589d400640 1 -- 172.21.15.3:0/3889075364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae0d692c0 con 0x556ae0e43fd0 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.337+0000 7f589d400640 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e43fd0 0x556ae0e443b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f588c005b60 tx=0x7f588c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595cae9dc226b5ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.337+0000 7f5896a00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f58840a3080 0x7f58840a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.338+0000 7f5896a00640 1 -- 172.21.15.3:0/2750732616 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f58980cfaa0 con 0x7f58a010de90 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.338+0000 7f588b000640 1 -- 172.21.15.3:0/3889075364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c048070 con 0x556ae0e43fd0 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.338+0000 7f588b000640 1 -- 172.21.15.3:0/3889075364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f588c0346c0 con 0x556ae0e43fd0 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.338+0000 7f588b000640 1 -- 172.21.15.3:0/3889075364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c039040 con 0x556ae0e43fd0 2024-03-21T08:37:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.338+0000 7f589de00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f58840a3080 0x7f58840a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3889075364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e43fd0 msgr2=0x556ae0e443b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f589de00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f58840a3080 0x7f58840a5540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x556ae0e43a80 tx=0x7f589000b390 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e43fd0 0x556ae0e443b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f588c005b60 tx=0x7f588c009510 comp rx=0 tx=0).stop 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3889075364 shutdown_connections 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e4d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3889075364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e43fd0 0x556ae0e443b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3889075364 >> 172.21.15.3:0/3889075364 conn(0x556ae0e3e8f0 msgr2=0x556ae0e3ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3889075364 shutdown_connections 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3889075364 wait complete. 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.339+0000 7f58a7a1a740 1 -- start start 2024-03-21T08:37:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 1 Processor -- start 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e4a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e448f0 0x556ae0e4ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0d64a20 con 0x556ae0e448f0 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0da48d0 con 0x556ae0e43fd0 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae0e50290 con 0x556ae0e4d6c0 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f58a7a1a740 10 client.0 _ll_register_callbacks cb 0x556ae0da11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f589d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e448f0 0x556ae0e4ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f589d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e448f0 0x556ae0e4ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38764/0 (socket says 172.21.15.3:38764) 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f589d400640 1 -- 172.21.15.3:0/2934109004 learned_addr learned my addr 172.21.15.3:0/2934109004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.340+0000 7f589de00640 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e4a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f589e800640 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f589d400640 1 -- 172.21.15.3:0/2934109004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e4d6c0 msgr2=0x556ae0e4b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f589d400640 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f589d400640 1 -- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 msgr2=0x556ae0e4a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f589d400640 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e4a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f589d400640 1 -- 172.21.15.3:0/2934109004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae0abd210 con 0x556ae0e448f0 2024-03-21T08:37:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f589d400640 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e448f0 0x556ae0e4ad20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f588c046040 tx=0x7f588c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a1b2349ef117c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.341+0000 7f587f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.342+0000 7f587f400640 1 -- 172.21.15.3:0/2934109004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c048040 con 0x556ae0e448f0 2024-03-21T08:37:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.342+0000 7f587f400640 1 -- 172.21.15.3:0/2934109004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f588c034c30 con 0x556ae0e448f0 2024-03-21T08:37:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.342+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f588c0346c0 con 0x556ae0e448f0 2024-03-21T08:37:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.342+0000 7f587f400640 1 -- 172.21.15.3:0/2934109004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f588c039040 con 0x556ae0e448f0 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.342+0000 7f58a7a1a740 10 client.16683 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.342+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556ae0b89620 con 0x556ae0e448f0 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.342+0000 7f587f400640 1 -- 172.21.15.3:0/2934109004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f588c0346c0 con 0x556ae0e448f0 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f587f400640 1 -- 172.21.15.3:0/2934109004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f588c04b070 con 0x556ae0e448f0 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f58a7a1a740 10 client.16683 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f58a7a1a740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f58a7a1a740 20 client.16683 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0e5d5e0 0x556ae0e5f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f58a7a1a740 4 client.16683 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ae0db5d70 con 0x556ae0e5d5e0 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.343+0000 7f589de00640 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0e5d5e0 0x556ae0e5f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.344+0000 7f589de00640 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0e5d5e0 0x556ae0e5f9d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a2db4303490d59 server_cookie=48a2c1f0484b307a in_seq=0 out_seq=0 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.344+0000 7f587f400640 10 client.16683 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f587f400640 1 -- 172.21.15.3:0/2934109004 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556ae0db5d70 con 0x556ae0e5d5e0 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f587f400640 10 client.16683 handle_command_reply: tid=0 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 client.16683 shutdown 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e448f0 msgr2=0x556ae0e4ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e448f0 0x556ae0e4ad20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f588c046040 tx=0x7f588c005f90 comp rx=0 tx=0).stop 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 shutdown_connections 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0e5d5e0 0x556ae0e5f9d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e4d6c0 0x556ae0e4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e448f0 0x556ae0e4ad20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/2934109004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e4a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f587f400640 0 client.16683 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 >> 172.21.15.3:0/2934109004 conn(0x556ae0e3e8f0 msgr2=0x556ae0e40640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 shutdown_connections 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.345+0000 7f58a7a1a740 1 -- 172.21.15.3:0/2934109004 wait complete. 2024-03-21T08:37:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.346+0000 7f58a7a1a740 20 client.16683 trim_cache size 0 max 16384 2024-03-21T08:37:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 Processor -- start 2024-03-21T08:37:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 -- start start 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e43fd0 0x556ae0e4cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e448f0 0x556ae0e58bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01b32a0 con 0x556ae0e4d6c0 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01b0d40 con 0x556ae0e448f0 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01aede0 con 0x556ae0e43fd0 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.362+0000 7f589e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e47a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38766/0 (socket says 172.21.15.3:38766) 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 -- 172.21.15.3:0/3793991175 learned_addr learned my addr 172.21.15.3:0/3793991175 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589d400640 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e448f0 0x556ae0e58bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589de00640 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e43fd0 0x556ae0e4cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 -- 172.21.15.3:0/3793991175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e43fd0 msgr2=0x556ae0e4cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e43fd0 0x556ae0e4cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 -- 172.21.15.3:0/3793991175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e448f0 msgr2=0x556ae0e58bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e448f0 0x556ae0e58bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 -- 172.21.15.3:0/3793991175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae0e393a0 con 0x556ae0e4d6c0 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589de00640 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e43fd0 0x556ae0e4cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:37:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.363+0000 7f589e800640 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e47a60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5898009830 tx=0x7f5898004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced894cc1ec2cba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.364+0000 7f588a600640 1 -- 172.21.15.3:0/3793991175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589809ba70 con 0x556ae0e4d6c0 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.364+0000 7f588a600640 1 -- 172.21.15.3:0/3793991175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f589809fbc0 con 0x556ae0e4d6c0 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.364+0000 7f588a600640 1 -- 172.21.15.3:0/3793991175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58980a4250 con 0x556ae0e4d6c0 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.364+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3793991175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 msgr2=0x556ae0e47a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.364+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e47a60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5898009830 tx=0x7f5898004290 comp rx=0 tx=0).stop 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3793991175 shutdown_connections 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e47a60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e448f0 0x556ae0e58bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/3793991175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e43fd0 0x556ae0e4cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3793991175 >> 172.21.15.3:0/3793991175 conn(0x556ae0e3e8f0 msgr2=0x556ae0e584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3793991175 shutdown_connections 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 -- 172.21.15.3:0/3793991175 wait complete. 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 -- start start 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.365+0000 7f58a7a1a740 1 Processor -- start 2024-03-21T08:37:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e62470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f58a7a1a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e60d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01af3e0 con 0x556ae0e4d6c0 2024-03-21T08:37:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01b06e0 con 0x556ae0e43fd0 2024-03-21T08:37:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f58a7a1a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01afa20 con 0x556ae0e448f0 2024-03-21T08:37:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f58a7a1a740 10 client.0 _ll_register_callbacks cb 0x7f588c046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f589de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e62470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f589de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e62470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:55646/0 (socket says 172.21.15.3:55646) 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f589de00640 1 -- 172.21.15.3:0/1014896805 learned_addr learned my addr 172.21.15.3:0/1014896805 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f589e800640 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e60d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.366+0000 7f589d400640 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.367+0000 7f589de00640 1 -- 172.21.15.3:0/1014896805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 msgr2=0x556ae0e60810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.367+0000 7f589de00640 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e60810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.367+0000 7f589de00640 1 -- 172.21.15.3:0/1014896805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 msgr2=0x556ae0e60d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.367+0000 7f589de00640 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e60d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.367+0000 7f589de00640 1 -- 172.21.15.3:0/1014896805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae0e61290 con 0x556ae0e43fd0 2024-03-21T08:37:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.367+0000 7f589de00640 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e62470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556ae0e502f0 tx=0x7f58900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfefe0003af9d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.367+0000 7f587f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.368+0000 7f587f400640 1 -- 172.21.15.3:0/1014896805 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589001ea10 con 0x556ae0e43fd0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.368+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f589809fbc0 con 0x556ae0e43fd0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.368+0000 7f587f400640 1 -- 172.21.15.3:0/1014896805 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f589001ebb0 con 0x556ae0e43fd0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.368+0000 7f587f400640 1 -- 172.21.15.3:0/1014896805 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5890038050 con 0x556ae0e43fd0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.368+0000 7f58a7a1a740 10 client.36419 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.368+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556ae0eb9840 con 0x556ae0e43fd0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.368+0000 7f587f400640 1 -- 172.21.15.3:0/1014896805 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f589809fbc0 con 0x556ae0e43fd0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f587f400640 1 -- 172.21.15.3:0/1014896805 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5890011560 con 0x556ae0e43fd0 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f58a7a1a740 10 client.36419 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f58a7a1a740 10 client.36419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f58a7a1a740 20 client.36419 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0eb9f40 0x556ae0ebc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f58a7a1a740 4 client.36419 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:37:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ae0db00c0 con 0x556ae0eb9f40 2024-03-21T08:37:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.369+0000 7f589d400640 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0eb9f40 0x556ae0ebc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.370+0000 7f589d400640 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0eb9f40 0x556ae0ebc360 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ae96c82fa39d2f server_cookie=7254a5ea5d623cc in_seq=0 out_seq=0 2024-03-21T08:37:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.370+0000 7f587f400640 10 client.36419 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:16.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:15 smithi084 ceph-mon[35477]: pgmap v165: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 365 KiB/s wr, 79 op/s 2024-03-21T08:37:16.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:15 smithi171 ceph-mon[35388]: pgmap v165: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 365 KiB/s wr, 79 op/s 2024-03-21T08:37:16.466 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/wait.h usability... yes 2024-03-21T08:37:16.526 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/wait.h presence... yes 2024-03-21T08:37:16.527 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/wait.h... yes 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.613+0000 7f587f400640 1 -- 172.21.15.3:0/1014896805 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ae0db00c0 con 0x556ae0eb9f40 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.613+0000 7f587f400640 10 client.36419 handle_command_reply: tid=0 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.613+0000 7f58a7a1a740 1 client.36419 shutdown 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.613+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 msgr2=0x556ae0e62470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.613+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e62470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556ae0e502f0 tx=0x7f58900046c0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 shutdown_connections 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ae0e4d6c0 0x556ae0e60d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x556ae0eb9f40 0x556ae0ebc360 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ae0e448f0 0x556ae0e60810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 --2- 172.21.15.3:0/1014896805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ae0e43fd0 0x556ae0e62470 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 >> 172.21.15.3:0/1014896805 conn(0x556ae0e3e8f0 msgr2=0x556ae0e4b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f587f400640 0 client.36419 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 shutdown_connections 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 1 -- 172.21.15.3:0/1014896805 wait complete. 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.614+0000 7f58a7a1a740 20 client.36419 trim_cache size 0 max 16384 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:37:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:37:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f58840a3080 msgr2=0x7f58840a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f58840a3080 0x7f58840a5540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x556ae0e43a80 tx=0x7f589000b390 comp rx=0 tx=0).stop 2024-03-21T08:37:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 msgr2=0x7f58a019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 0x7f58a019e960 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f589800efc0 tx=0x7f589800c800 comp rx=0 tx=0).stop 2024-03-21T08:37:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 shutdown_connections 2024-03-21T08:37:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58a010de90 0x7f58a019e960 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f58840a3080 0x7f58840a5540 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58a00743f0 0x7f58a019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 --2- 172.21.15.3:0/2750732616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a0073ad0 0x7f58a019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 >> 172.21.15.3:0/2750732616 conn(0x7f58a00fd120 msgr2=0x7f58a00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 shutdown_connections 2024-03-21T08:37:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:16.615+0000 7f58a4c00640 1 -- 172.21.15.3:0/2750732616 wait complete. 2024-03-21T08:37:16.628 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/types.h... (cached) yes 2024-03-21T08:37:16.674 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strings.h... (cached) yes 2024-03-21T08:37:16.851 INFO:tasks.workunit.client.0.smithi003.stdout:checking err.h usability... yes 2024-03-21T08:37:16.933 INFO:tasks.workunit.client.0.smithi003.stdout:checking err.h presence... yes 2024-03-21T08:37:16.935 INFO:tasks.workunit.client.0.smithi003.stdout:checking for err.h... yes 2024-03-21T08:37:17.153 INFO:tasks.workunit.client.0.smithi003.stdout:checking linux/falloc.h usability... yes 2024-03-21T08:37:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:checking linux/falloc.h presence... yes 2024-03-21T08:37:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:checking for linux/falloc.h... yes 2024-03-21T08:37:17.467 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-03-21T08:37:17.579 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-03-21T08:37:17.581 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/fs/xfs_fsops.h... no 2024-03-21T08:37:17.788 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/fs/xfs_itable.h usability... no 2024-03-21T08:37:17.927 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/fs/xfs_itable.h presence... no 2024-03-21T08:37:17.928 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/fs/xfs_itable.h... no 2024-03-21T08:37:18.133 INFO:tasks.workunit.client.0.smithi003.stdout:checking xfs/platform_defs.h usability... no 2024-03-21T08:37:18.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:17 smithi003 ceph-mon[31675]: pgmap v166: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 529 KiB/s wr, 113 op/s 2024-03-21T08:37:18.267 INFO:tasks.workunit.client.0.smithi003.stdout:checking xfs/platform_defs.h presence... no 2024-03-21T08:37:18.269 INFO:tasks.workunit.client.0.smithi003.stdout:checking for xfs/platform_defs.h... no 2024-03-21T08:37:18.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:17 smithi084 ceph-mon[35477]: pgmap v166: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 529 KiB/s wr, 113 op/s 2024-03-21T08:37:18.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:17 smithi171 ceph-mon[35388]: pgmap v166: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 529 KiB/s wr, 113 op/s 2024-03-21T08:37:18.466 INFO:tasks.workunit.client.0.smithi003.stdout:checking btrfs/ioctl.h usability... no 2024-03-21T08:37:18.586 INFO:tasks.workunit.client.0.smithi003.stdout:checking btrfs/ioctl.h presence... no 2024-03-21T08:37:18.588 INFO:tasks.workunit.client.0.smithi003.stdout:checking for btrfs/ioctl.h... no 2024-03-21T08:37:18.761 INFO:tasks.workunit.client.0.smithi003.stdout:checking cifs/ioctl.h usability... no 2024-03-21T08:37:18.889 INFO:tasks.workunit.client.0.smithi003.stdout:checking cifs/ioctl.h presence... no 2024-03-21T08:37:18.891 INFO:tasks.workunit.client.0.smithi003.stdout:checking for cifs/ioctl.h... no 2024-03-21T08:37:19.075 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/mman.h usability... yes 2024-03-21T08:37:19.156 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/mman.h presence... yes 2024-03-21T08:37:19.158 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/mman.h... yes 2024-03-21T08:37:19.354 INFO:tasks.workunit.client.0.smithi003.stdout:checking for xfs/xfs_log_format.h... no 2024-03-21T08:37:19.617 INFO:tasks.workunit.client.0.smithi003.stdout:checking for uuid_compare... no 2024-03-21T08:37:19.852 INFO:tasks.workunit.client.0.smithi003.stdout:checking for uuid_compare in -luuid... yes 2024-03-21T08:37:20.056 INFO:tasks.workunit.client.0.smithi003.stdout:checking for xfs/xfs.h... yes 2024-03-21T08:37:20.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:19 smithi003 ceph-mon[31675]: pgmap v167: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-03-21T08:37:20.227 INFO:tasks.workunit.client.0.smithi003.stdout:checking for xfs/libxfs.h... no 2024-03-21T08:37:20.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:19 smithi084 ceph-mon[35477]: pgmap v167: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-03-21T08:37:20.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:19 smithi171 ceph-mon[35388]: pgmap v167: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-03-21T08:37:20.396 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether xlog_assign_lsn is declared... no 2024-03-21T08:37:20.549 INFO:tasks.workunit.client.0.smithi003.stdout:checking for xfs/xqm.h... yes 2024-03-21T08:37:20.767 INFO:tasks.workunit.client.0.smithi003.stdout:checking xfsctl from xfs/xfs.h... ok 2024-03-21T08:37:20.923 INFO:tasks.workunit.client.0.smithi003.stdout:checking xfs/handle.h usability... yes 2024-03-21T08:37:21.005 INFO:tasks.workunit.client.0.smithi003.stdout:checking xfs/handle.h presence... yes 2024-03-21T08:37:21.008 INFO:tasks.workunit.client.0.smithi003.stdout:checking for xfs/handle.h... yes 2024-03-21T08:37:21.257 INFO:tasks.workunit.client.0.smithi003.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-03-21T08:37:21.464 INFO:tasks.workunit.client.0.smithi003.stdout:checking attr/attributes.h usability... yes 2024-03-21T08:37:21.531 INFO:tasks.workunit.client.0.smithi003.stdout:checking attr/attributes.h presence... yes 2024-03-21T08:37:21.532 INFO:tasks.workunit.client.0.smithi003.stdout:checking for attr/attributes.h... yes 2024-03-21T08:37:21.782 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/attributes.h usability... no 2024-03-21T08:37:21.916 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/attributes.h presence... no 2024-03-21T08:37:21.918 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/attributes.h... no 2024-03-21T08:37:22.160 INFO:tasks.workunit.client.0.smithi003.stdout:checking for attr_list in -lattr... yes 2024-03-21T08:37:22.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:21 smithi003 ceph-mon[31675]: pgmap v168: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-03-21T08:37:22.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:21 smithi084 ceph-mon[35477]: pgmap v168: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-03-21T08:37:22.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:21 smithi171 ceph-mon[35388]: pgmap v168: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 356 KiB/s wr, 66 op/s 2024-03-21T08:37:22.395 INFO:tasks.workunit.client.0.smithi003.stdout:checking for attr_set in -lattr... yes 2024-03-21T08:37:22.528 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/acl.h usability... yes 2024-03-21T08:37:22.596 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/acl.h presence... yes 2024-03-21T08:37:22.598 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/acl.h... yes 2024-03-21T08:37:22.730 INFO:tasks.workunit.client.0.smithi003.stdout:checking acl/libacl.h usability... yes 2024-03-21T08:37:22.814 INFO:tasks.workunit.client.0.smithi003.stdout:checking acl/libacl.h presence... yes 2024-03-21T08:37:22.814 INFO:tasks.workunit.client.0.smithi003.stdout:checking for acl/libacl.h... yes 2024-03-21T08:37:23.064 INFO:tasks.workunit.client.0.smithi003.stdout:checking for acl_init in -lacl... yes 2024-03-21T08:37:23.240 INFO:tasks.workunit.client.0.smithi003.stdout:checking gdbm-ndbm.h usability... no 2024-03-21T08:37:23.373 INFO:tasks.workunit.client.0.smithi003.stdout:checking gdbm-ndbm.h presence... no 2024-03-21T08:37:23.375 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gdbm-ndbm.h... no 2024-03-21T08:37:23.538 INFO:tasks.workunit.client.0.smithi003.stdout:checking gdbm/ndbm.h usability... no 2024-03-21T08:37:23.661 INFO:tasks.workunit.client.0.smithi003.stdout:checking gdbm/ndbm.h presence... no 2024-03-21T08:37:23.664 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gdbm/ndbm.h... no 2024-03-21T08:37:23.812 INFO:tasks.workunit.client.0.smithi003.stdout:checking ndbm.h usability... no 2024-03-21T08:37:23.947 INFO:tasks.workunit.client.0.smithi003.stdout:checking ndbm.h presence... no 2024-03-21T08:37:23.948 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ndbm.h... no 2024-03-21T08:37:24.081 INFO:tasks.workunit.client.0.smithi003.stdout:checking libaio.h usability... yes 2024-03-21T08:37:24.160 INFO:tasks.workunit.client.0.smithi003.stdout:checking libaio.h presence... yes 2024-03-21T08:37:24.161 INFO:tasks.workunit.client.0.smithi003.stdout:checking for libaio.h... yes 2024-03-21T08:37:24.372 INFO:tasks.workunit.client.0.smithi003.stdout:checking for dm_make_handle in -ldm... no 2024-03-21T08:37:24.373 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-03-21T08:37:24.373 INFO:tasks.workunit.client.0.smithi003.stdout:WARNING: could not find a valid DMAPI base library. 2024-03-21T08:37:24.373 INFO:tasks.workunit.client.0.smithi003.stdout:If you want DMAPI support please install the data migration 2024-03-21T08:37:24.373 INFO:tasks.workunit.client.0.smithi003.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-03-21T08:37:24.373 INFO:tasks.workunit.client.0.smithi003.stdout:from the dmapi source. 2024-03-21T08:37:24.373 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-03-21T08:37:24.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:23 smithi084 ceph-mon[35477]: pgmap v169: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 513 KiB/s wr, 97 op/s 2024-03-21T08:37:24.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:23 smithi171 ceph-mon[35388]: pgmap v169: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 513 KiB/s wr, 97 op/s 2024-03-21T08:37:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:23 smithi003 ceph-mon[31675]: pgmap v169: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 513 KiB/s wr, 97 op/s 2024-03-21T08:37:24.499 INFO:tasks.workunit.client.0.smithi003.stdout:checking linux/fiemap.h usability... yes 2024-03-21T08:37:24.549 INFO:tasks.workunit.client.0.smithi003.stdout:checking linux/fiemap.h presence... yes 2024-03-21T08:37:24.550 INFO:tasks.workunit.client.0.smithi003.stdout:checking for linux/fiemap.h... yes 2024-03-21T08:37:24.728 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fallocate... yes 2024-03-21T08:37:24.972 INFO:tasks.workunit.client.0.smithi003.stdout:checking for open_by_handle_at... yes 2024-03-21T08:37:25.148 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/prctl.h usability... yes 2024-03-21T08:37:25.225 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/prctl.h presence... yes 2024-03-21T08:37:25.227 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/prctl.h... yes 2024-03-21T08:37:25.382 INFO:tasks.workunit.client.0.smithi003.stdout:checking linux/fs.h usability... yes 2024-03-21T08:37:25.461 INFO:tasks.workunit.client.0.smithi003.stdout:checking linux/fs.h presence... yes 2024-03-21T08:37:25.466 INFO:tasks.workunit.client.0.smithi003.stdout:checking for linux/fs.h... yes 2024-03-21T08:37:25.605 INFO:tasks.workunit.client.0.smithi003.stdout:checking btrfsutil.h usability... no 2024-03-21T08:37:25.735 INFO:tasks.workunit.client.0.smithi003.stdout:checking btrfsutil.h presence... no 2024-03-21T08:37:25.737 INFO:tasks.workunit.client.0.smithi003.stdout:checking for btrfsutil.h... no 2024-03-21T08:37:25.971 INFO:tasks.workunit.client.0.smithi003.stdout:checking for copy_file_range... yes 2024-03-21T08:37:26.240 INFO:tasks.workunit.client.0.smithi003.stdout:checking for renameat2... yes 2024-03-21T08:37:26.313 INFO:tasks.workunit.client.0.smithi003.stdout:configure: creating ./config.status 2024-03-21T08:37:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:25 smithi084 ceph-mon[35477]: pgmap v170: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 320 KiB/s wr, 64 op/s 2024-03-21T08:37:26.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:25 smithi171 ceph-mon[35388]: pgmap v170: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 320 KiB/s wr, 64 op/s 2024-03-21T08:37:26.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:25 smithi003 ceph-mon[31675]: pgmap v170: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 320 KiB/s wr, 64 op/s 2024-03-21T08:37:27.790 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating include/builddefs 2024-03-21T08:37:27.915 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating include/config.h 2024-03-21T08:37:28.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:28 smithi084 ceph-mon[35477]: pgmap v171: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 487 KiB/s wr, 97 op/s 2024-03-21T08:37:28.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:28 smithi171 ceph-mon[35388]: pgmap v171: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 487 KiB/s wr, 97 op/s 2024-03-21T08:37:28.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:28 smithi003 ceph-mon[31675]: pgmap v171: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 487 KiB/s wr, 97 op/s 2024-03-21T08:37:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 -- 172.21.15.3:0/4011173231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 msgr2=0x7f5af0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 --2- 172.21.15.3:0/4011173231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0101920 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009900 tx=0x7f5adc02bc40 comp rx=0 tx=0).stop 2024-03-21T08:37:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 -- 172.21.15.3:0/4011173231 shutdown_connections 2024-03-21T08:37:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 --2- 172.21.15.3:0/4011173231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5af0101e60 0x7f5af010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 --2- 172.21.15.3:0/4011173231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0101920 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 --2- 172.21.15.3:0/4011173231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af0107560 0x7f5af0107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 -- 172.21.15.3:0/4011173231 >> 172.21.15.3:0/4011173231 conn(0x7f5af00fcfb0 msgr2=0x7f5af00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 -- 172.21.15.3:0/4011173231 shutdown_connections 2024-03-21T08:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.764+0000 7f5af4a00640 1 -- 172.21.15.3:0/4011173231 wait complete. 2024-03-21T08:37:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.765+0000 7f5af4a00640 1 Processor -- start 2024-03-21T08:37:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5af4a00640 1 -- start start 2024-03-21T08:37:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5af4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5af4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af0101e60 0x7f5af019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5af4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5af0107560 0x7f5af019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5af4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af0111740 con 0x7f5af01014a0 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5af4a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01115c0 con 0x7f5af0107560 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5af4a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01118c0 con 0x7f5af0101e60 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5aef400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5aef400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60506/0 (socket says 172.21.15.3:60506) 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5aef400640 1 -- 172.21.15.3:0/3269723332 learned_addr learned my addr 172.21.15.3:0/3269723332 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5aeea00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af0101e60 0x7f5af019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.766+0000 7f5aefe00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5af0107560 0x7f5af019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5aef400640 1 -- 172.21.15.3:0/3269723332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af0101e60 msgr2=0x7f5af019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5aef400640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af0101e60 0x7f5af019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5aef400640 1 -- 172.21.15.3:0/3269723332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5af0107560 msgr2=0x7f5af019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5aef400640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5af0107560 0x7f5af019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5aef400640 1 -- 172.21.15.3:0/3269723332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af019eed0 con 0x7f5af01014a0 2024-03-21T08:37:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5aef400640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0199ee0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d770 tx=0x7f5ae000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5ae7e00640 1 -- 172.21.15.3:0/3269723332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0017070 con 0x7f5af01014a0 2024-03-21T08:37:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af019f1c0 con 0x7f5af01014a0 2024-03-21T08:37:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5ae7e00640 1 -- 172.21.15.3:0/3269723332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ae00054d0 con 0x7f5af01014a0 2024-03-21T08:37:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.767+0000 7f5ae7e00640 1 -- 172.21.15.3:0/3269723332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0004cf0 con 0x7f5af01014a0 2024-03-21T08:37:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.768+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af01a6a50 con 0x7f5af01014a0 2024-03-21T08:37:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 Processor -- start 2024-03-21T08:37:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5ae7e00640 1 -- 172.21.15.3:0/3269723332 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f5ae0004e90 con 0x7f5af01014a0 2024-03-21T08:37:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 -- start start 2024-03-21T08:37:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a4b00 0x562c104ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104ad8d0 0x562c104afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c104663b0 con 0x562c104a41e0 2024-03-21T08:37:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c10408830 con 0x562c104a4b00 2024-03-21T08:37:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.771+0000 7f5af8de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c103b15d0 con 0x562c104ad8d0 2024-03-21T08:37:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60512/0 (socket says 172.21.15.3:60512) 2024-03-21T08:37:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 -- 172.21.15.3:0/142712449 learned_addr learned my addr 172.21.15.3:0/142712449 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aef400640 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a4b00 0x562c104ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aeea00640 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104ad8d0 0x562c104afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 -- 172.21.15.3:0/142712449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104ad8d0 msgr2=0x562c104afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104ad8d0 0x562c104afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 -- 172.21.15.3:0/142712449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a4b00 msgr2=0x562c104ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a4b00 0x562c104ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 -- 172.21.15.3:0/142712449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c103c92c0 con 0x562c104a41e0 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.772+0000 7f5aefe00640 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104a45c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009fa0 tx=0x7f5ae800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e063fa34093c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.773+0000 7f5ae7e00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5ad40a3070 0x7f5ad40a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.773+0000 7f5adbe00640 1 -- 172.21.15.3:0/142712449 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8015070 con 0x562c104a41e0 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.773+0000 7f5ae7e00640 1 -- 172.21.15.3:0/3269723332 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5ae00cf6d0 con 0x7f5af01014a0 2024-03-21T08:37:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.773+0000 7f5adbe00640 1 -- 172.21.15.3:0/142712449 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ae801ab10 con 0x562c104a41e0 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.773+0000 7f5adbe00640 1 -- 172.21.15.3:0/142712449 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8011ce0 con 0x562c104a41e0 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.773+0000 7f5aef400640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5ad40a3070 0x7f5ad40a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 -- 172.21.15.3:0/142712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 msgr2=0x562c104a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104a45c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009fa0 tx=0x7f5ae800f7f0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5aef400640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5ad40a3070 0x7f5ad40a5530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x562c104a3c90 tx=0x7f5ae00d3040 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 -- 172.21.15.3:0/142712449 shutdown_connections 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104ad8d0 0x562c104afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a4b00 0x562c104ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 --2- 172.21.15.3:0/142712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104a45c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 -- 172.21.15.3:0/142712449 >> 172.21.15.3:0/142712449 conn(0x562c1049eb00 msgr2=0x562c1049eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 -- 172.21.15.3:0/142712449 shutdown_connections 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 -- 172.21.15.3:0/142712449 wait complete. 2024-03-21T08:37:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 -- start start 2024-03-21T08:37:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.774+0000 7f5af8de3740 1 Processor -- start 2024-03-21T08:37:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104aa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 0x562c104aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104ad8d0 0x562c104ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5af8de3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c103c4a20 con 0x562c104a41e0 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5af8de3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c104048d0 con 0x562c104ad8d0 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5af8de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c104b0500 con 0x562c104a4b00 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5af8de3740 10 client.0 _ll_register_callbacks cb 0x562c104011c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60522/0 (socket says 172.21.15.3:60522) 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5aefe00640 1 -- 172.21.15.3:0/2592956258 learned_addr learned my addr 172.21.15.3:0/2592956258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.775+0000 7f5aefe00640 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104ad8d0 0x562c104ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5aeea00640 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 0x562c104aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5aefe00640 1 -- 172.21.15.3:0/2592956258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 msgr2=0x562c104aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5aefe00640 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 0x562c104aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5aefe00640 1 -- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104ad8d0 msgr2=0x562c104ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5aefe00640 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104ad8d0 0x562c104ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5aefe00640 1 -- 172.21.15.3:0/2592956258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c1011d210 con 0x562c104a41e0 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5aefe00640 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104aa970 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8011900 tx=0x7f5ae8005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cd0ead23cbd26e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5ad8c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5ad8c00640 1 -- 172.21.15.3:0/2592956258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8015050 con 0x562c104a41e0 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ae801ab10 con 0x562c104a41e0 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.776+0000 7f5ad8c00640 1 -- 172.21.15.3:0/2592956258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5ae8004a40 con 0x562c104a41e0 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5ad8c00640 1 -- 172.21.15.3:0/2592956258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ae8004c40 con 0x562c104a41e0 2024-03-21T08:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 10 client.16701 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562c101e9620 con 0x562c104a41e0 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5ad8c00640 1 -- 172.21.15.3:0/2592956258 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ae801ab10 con 0x562c104a41e0 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5ad8c00640 1 -- 172.21.15.3:0/2592956258 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5ae8027020 con 0x562c104a41e0 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 10 client.16701 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 20 client.16701 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104bd820 0x562c104bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 4 client.16701 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.777+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c10415d70 con 0x562c104bd820 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.778+0000 7f5aeea00640 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104bd820 0x562c104bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.779+0000 7f5aeea00640 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104bd820 0x562c104bfc10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0875baceb2d297 server_cookie=2be31743648cf454 in_seq=0 out_seq=0 2024-03-21T08:37:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.779+0000 7f5ad8c00640 10 client.16701 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5ad8c00640 1 -- 172.21.15.3:0/2592956258 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562c10415d70 con 0x562c104bd820 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5ad8c00640 10 client.16701 handle_command_reply: tid=0 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 client.16701 shutdown 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 msgr2=0x562c104aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104aa970 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8011900 tx=0x7f5ae8005c30 comp rx=0 tx=0).stop 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 shutdown_connections 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104ad8d0 0x562c104ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104bd820 0x562c104bfc10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 0x562c104aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 --2- 172.21.15.3:0/2592956258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a41e0 0x562c104aa970 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5ad8c00640 0 client.16701 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.780+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 >> 172.21.15.3:0/2592956258 conn(0x562c1049eb00 msgr2=0x562c104a0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.781+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 shutdown_connections 2024-03-21T08:37:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.781+0000 7f5af8de3740 1 -- 172.21.15.3:0/2592956258 wait complete. 2024-03-21T08:37:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.781+0000 7f5af8de3740 20 client.16701 trim_cache size 0 max 16384 2024-03-21T08:37:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 Processor -- start 2024-03-21T08:37:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 -- start start 2024-03-21T08:37:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c1049eb00 0x562c104ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a41e0 0x562c104a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a4b00 0x562c104a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01b31e0 con 0x562c104a4b00 2024-03-21T08:37:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01b0c80 con 0x562c1049eb00 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.796+0000 7f5af8de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01aed20 con 0x562c104a41e0 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a4b00 0x562c104a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a4b00 0x562c104a8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60538/0 (socket says 172.21.15.3:60538) 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 -- 172.21.15.3:0/1591921942 learned_addr learned my addr 172.21.15.3:0/1591921942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c1049eb00 0x562c104ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aeea00640 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a41e0 0x562c104a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 -- 172.21.15.3:0/1591921942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a41e0 msgr2=0x562c104a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a41e0 0x562c104a7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 -- 172.21.15.3:0/1591921942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c1049eb00 msgr2=0x562c104ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c1049eb00 0x562c104ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 -- 172.21.15.3:0/1591921942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1049e2f0 con 0x562c104a4b00 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5aefe00640 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a4b00 0x562c104a8170 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009b90 tx=0x7f5ae8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318e701236865155 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.797+0000 7f5adb400640 1 -- 172.21.15.3:0/1591921942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8015030 con 0x562c104a4b00 2024-03-21T08:37:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5adb400640 1 -- 172.21.15.3:0/1591921942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ae801e030 con 0x562c104a4b00 2024-03-21T08:37:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5adb400640 1 -- 172.21.15.3:0/1591921942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8004de0 con 0x562c104a4b00 2024-03-21T08:37:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 -- 172.21.15.3:0/1591921942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a4b00 msgr2=0x562c104a8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a4b00 0x562c104a8170 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009b90 tx=0x7f5ae8009c80 comp rx=0 tx=0).stop 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 -- 172.21.15.3:0/1591921942 shutdown_connections 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c104a4b00 0x562c104a8170 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a41e0 0x562c104a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 --2- 172.21.15.3:0/1591921942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c1049eb00 0x562c104ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 -- 172.21.15.3:0/1591921942 >> 172.21.15.3:0/1591921942 conn(0x562c104a38b0 msgr2=0x562c104ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 -- 172.21.15.3:0/1591921942 shutdown_connections 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.798+0000 7f5af8de3740 1 -- 172.21.15.3:0/1591921942 wait complete. 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 -- start start 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 Processor -- start 2024-03-21T08:37:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c1049eb00 0x562c104c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a41e0 0x562c104c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 0x562c10511d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01af320 con 0x562c1049eb00 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01b0620 con 0x562c104a41e0 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01af960 con 0x562c104a4b00 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.799+0000 7f5af8de3740 10 client.0 _ll_register_callbacks cb 0x7f5ae8009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c1049eb00 0x562c104c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c1049eb00 0x562c104c0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60546/0 (socket says 172.21.15.3:60546) 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 -- 172.21.15.3:0/3350446366 learned_addr learned my addr 172.21.15.3:0/3350446366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aeea00640 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a41e0 0x562c104c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 -- 172.21.15.3:0/3350446366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 msgr2=0x562c10511d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 0x562c10511d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 -- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a41e0 msgr2=0x562c104c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a41e0 0x562c104c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.800+0000 7f5aefe00640 1 -- 172.21.15.3:0/3350446366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c105123d0 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5aefe00640 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c1049eb00 0x562c104c0960 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ae801d040 tx=0x7f5ae8005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42038cbb660ba95d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5ad8c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5ad8c00640 1 -- 172.21.15.3:0/3350446366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8004fe0 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5ad8c00640 1 -- 172.21.15.3:0/3350446366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f5ae8005180 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ae801e030 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5ad8c00640 1 -- 172.21.15.3:0/3350446366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ae8005bd0 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5af8de3740 10 client.16707 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562c10519790 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.801+0000 7f5ad8c00640 1 -- 172.21.15.3:0/3350446366 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ae801e030 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5ad8c00640 1 -- 172.21.15.3:0/3350446366 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5ae8027040 con 0x562c1049eb00 2024-03-21T08:37:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5af8de3740 10 client.16707 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5af8de3740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5af8de3740 20 client.16707 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5af8de3740 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104ad8d0 0x562c1051bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5af8de3740 4 client.16707 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:37:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c104100c0 con 0x562c104ad8d0 2024-03-21T08:37:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.802+0000 7f5aeea00640 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104ad8d0 0x562c1051bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.803+0000 7f5aeea00640 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104ad8d0 0x562c1051bd20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828ec1ad4a61d478 server_cookie=3a418a7160cad85d in_seq=0 out_seq=0 2024-03-21T08:37:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:29.803+0000 7f5ad8c00640 10 client.16707 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:30.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:30 smithi084 ceph-mon[35477]: pgmap v172: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 324 KiB/s wr, 63 op/s 2024-03-21T08:37:30.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:30 smithi171 ceph-mon[35388]: pgmap v172: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 324 KiB/s wr, 63 op/s 2024-03-21T08:37:30.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:30 smithi003 ceph-mon[31675]: pgmap v172: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 324 KiB/s wr, 63 op/s 2024-03-21T08:37:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.017+0000 7f5ad8c00640 1 -- 172.21.15.3:0/3350446366 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c104100c0 con 0x562c104ad8d0 2024-03-21T08:37:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.017+0000 7f5ad8c00640 10 client.16707 handle_command_reply: tid=0 2024-03-21T08:37:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 client.16707 shutdown 2024-03-21T08:37:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c1049eb00 msgr2=0x562c104c0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c1049eb00 0x562c104c0960 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ae801d040 tx=0x7f5ae8005650 comp rx=0 tx=0).stop 2024-03-21T08:37:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 shutdown_connections 2024-03-21T08:37:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x562c104ad8d0 0x562c1051bd20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c104a4b00 0x562c10511d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562c104a41e0 0x562c104c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 --2- 172.21.15.3:0/3350446366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562c1049eb00 0x562c104c0960 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 >> 172.21.15.3:0/3350446366 conn(0x562c104a38b0 msgr2=0x562c1049db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.018+0000 7f5ad8c00640 0 client.16707 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.019+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 shutdown_connections 2024-03-21T08:37:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.019+0000 7f5af8de3740 1 -- 172.21.15.3:0/3350446366 wait complete. 2024-03-21T08:37:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.019+0000 7f5af8de3740 20 client.16707 trim_cache size 0 max 16384 2024-03-21T08:37:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:37:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:37:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:37:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:37:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.020+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5ad40a3070 msgr2=0x7f5ad40a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.020+0000 7f5af4a00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5ad40a3070 0x7f5ad40a5530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x562c104a3c90 tx=0x7f5ae00d3040 comp rx=0 tx=0).stop 2024-03-21T08:37:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.020+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 msgr2=0x7f5af0199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.020+0000 7f5af4a00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0199ee0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d770 tx=0x7f5ae000dc30 comp rx=0 tx=0).stop 2024-03-21T08:37:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 shutdown_connections 2024-03-21T08:37:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5af0107560 0x7f5af019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f5ad40a3070 0x7f5ad40a5530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af0101e60 0x7f5af019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 --2- 172.21.15.3:0/3269723332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5af01014a0 0x7f5af0199ee0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 >> 172.21.15.3:0/3269723332 conn(0x7f5af00fcfb0 msgr2=0x7f5af010d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 shutdown_connections 2024-03-21T08:37:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:31.021+0000 7f5af4a00640 1 -- 172.21.15.3:0/3269723332 wait complete. 2024-03-21T08:37:31.024 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing libtool commands 2024-03-21T08:37:31.692 INFO:tasks.workunit.client.0.smithi003.stdout:Building lib 2024-03-21T08:37:31.692 INFO:tasks.workunit.client.0.smithi003.stdout:Building tests 2024-03-21T08:37:31.693 INFO:tasks.workunit.client.0.smithi003.stdout:Building include 2024-03-21T08:37:31.743 INFO:tasks.workunit.client.0.smithi003.stdout: [LTDEP] 2024-03-21T08:37:32.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:32 smithi084 ceph-mon[35477]: pgmap v173: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 324 KiB/s wr, 63 op/s 2024-03-21T08:37:32.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:32 smithi171 ceph-mon[35388]: pgmap v173: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 324 KiB/s wr, 63 op/s 2024-03-21T08:37:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:32 smithi003 ceph-mon[31675]: pgmap v173: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 324 KiB/s wr, 63 op/s 2024-03-21T08:37:33.544 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] dataascii.lo 2024-03-21T08:37:33.546 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] databin.lo 2024-03-21T08:37:33.546 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] datapid.lo 2024-03-21T08:37:33.549 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] file_lock.lo 2024-03-21T08:37:33.944 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] forker.lo 2024-03-21T08:37:34.012 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] pattern.lo 2024-03-21T08:37:34.080 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] open_flags.lo 2024-03-21T08:37:34.127 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] random_range.lo 2024-03-21T08:37:34.368 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] string_to_tokens.lo 2024-03-21T08:37:34.402 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] str_to_bytes.lo 2024-03-21T08:37:34.583 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] tlibio.lo 2024-03-21T08:37:34.696 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] write_log.lo 2024-03-21T08:37:34.707 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] random.lo 2024-03-21T08:37:34.878 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:34 smithi084 ceph-mon[35477]: pgmap v174: 129 pgs: 129 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 415 KiB/s wr, 82 op/s 2024-03-21T08:37:34.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:34 smithi171 ceph-mon[35388]: pgmap v174: 129 pgs: 129 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 415 KiB/s wr, 82 op/s 2024-03-21T08:37:34.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:34 smithi003 ceph-mon[31675]: pgmap v174: 129 pgs: 129 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 415 KiB/s wr, 82 op/s 2024-03-21T08:37:35.392 INFO:tasks.workunit.client.0.smithi003.stdout: [LD] 2024-03-21T08:37:36.235 INFO:tasks.workunit.client.0.smithi003.stdout:Building src 2024-03-21T08:37:36.235 INFO:tasks.workunit.client.0.smithi003.stdout:Building ltp 2024-03-21T08:37:36.236 INFO:tasks.workunit.client.0.smithi003.stdout:Building m4 2024-03-21T08:37:36.237 INFO:tasks.workunit.client.0.smithi003.stdout:Building common 2024-03-21T08:37:36.312 INFO:tasks.workunit.client.0.smithi003.stdout: [DEP] 2024-03-21T08:37:36.391 INFO:tasks.workunit.client.0.smithi003.stdout: [DEP] 2024-03-21T08:37:36.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:36 smithi084 ceph-mon[35477]: pgmap v175: 129 pgs: 129 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-03-21T08:37:36.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:36 smithi171 ceph-mon[35388]: pgmap v175: 129 pgs: 129 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-03-21T08:37:36.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:36 smithi003 ceph-mon[31675]: pgmap v175: 129 pgs: 129 active+clean; 158 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-03-21T08:37:37.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:37 smithi084 ceph-mon[35477]: pgmap v176: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 378 KiB/s wr, 73 op/s 2024-03-21T08:37:37.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:37 smithi171 ceph-mon[35388]: pgmap v176: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 378 KiB/s wr, 73 op/s 2024-03-21T08:37:37.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:37 smithi003 ceph-mon[31675]: pgmap v176: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 378 KiB/s wr, 73 op/s 2024-03-21T08:37:40.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:39 smithi084 ceph-mon[35477]: pgmap v177: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 40 op/s 2024-03-21T08:37:40.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:39 smithi171 ceph-mon[35388]: pgmap v177: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 40 op/s 2024-03-21T08:37:40.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:39 smithi003 ceph-mon[31675]: pgmap v177: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 40 op/s 2024-03-21T08:37:41.411 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] doio 2024-03-21T08:37:41.411 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] fsstress 2024-03-21T08:37:41.413 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] fsx 2024-03-21T08:37:42.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:41 smithi084 ceph-mon[35477]: pgmap v178: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 40 op/s 2024-03-21T08:37:42.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:41 smithi171 ceph-mon[35388]: pgmap v178: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 40 op/s 2024-03-21T08:37:42.165 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c: In function 'attr_list_path': 2024-03-21T08:37:42.165 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-03-21T08:37:42.165 INFO:tasks.workunit.client.0.smithi003.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-03-21T08:37:42.165 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~ 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr:In file included from ../src/global.h:25, 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr: from fsstress.c:12: 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~ 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c: In function 'attr_remove_path': 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~ 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr:In file included from ../src/global.h:25, 2024-03-21T08:37:42.166 INFO:tasks.workunit.client.0.smithi003.stderr: from fsstress.c:12: 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~ 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c: In function 'attr_set_path': 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~ 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr:In file included from ../src/global.h:25, 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr: from fsstress.c:12: 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-03-21T08:37:42.167 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-03-21T08:37:42.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:41 smithi003 ceph-mon[31675]: pgmap v178: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 40 op/s 2024-03-21T08:37:42.180 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c: In function 'allocsp_f': 2024-03-21T08:37:42.181 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-03-21T08:37:42.181 INFO:tasks.workunit.client.0.smithi003.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-03-21T08:37:42.181 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~ 2024-03-21T08:37:42.181 INFO:tasks.workunit.client.0.smithi003.stderr: | XFS_IOC_RESVSP64 2024-03-21T08:37:42.181 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-03-21T08:37:42.205 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c: In function 'freesp_f': 2024-03-21T08:37:42.205 INFO:tasks.workunit.client.0.smithi003.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-03-21T08:37:42.206 INFO:tasks.workunit.client.0.smithi003.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-03-21T08:37:42.206 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~ 2024-03-21T08:37:42.206 INFO:tasks.workunit.client.0.smithi003.stderr: | XFS_IOC_RESVSP64 2024-03-21T08:37:42.258 INFO:tasks.workunit.client.0.smithi003.stderr:gmake[3]: *** [Makefile:46: fsstress] Error 1 2024-03-21T08:37:42.258 INFO:tasks.workunit.client.0.smithi003.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr:doio.c: In function 'parse_cmdline': 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr: 3113 | fprintf(stderr, 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~ 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr: 3115 | Prog, TagName, tok); 2024-03-21T08:37:42.682 INFO:tasks.workunit.client.0.smithi003.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-03-21T08:37:42.695 INFO:tasks.workunit.client.0.smithi003.stderr:fsx.c: In function 'main': 2024-03-21T08:37:42.695 INFO:tasks.workunit.client.0.smithi003.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-03-21T08:37:42.695 INFO:tasks.workunit.client.0.smithi003.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-03-21T08:37:42.695 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-21T08:37:42.696 INFO:tasks.workunit.client.0.smithi003.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-03-21T08:37:42.696 INFO:tasks.workunit.client.0.smithi003.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-03-21T08:37:42.696 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-21T08:37:42.907 INFO:tasks.workunit.client.0.smithi003.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-03-21T08:37:42.907 INFO:tasks.workunit.client.0.smithi003.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-03-21T08:37:44.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:43 smithi084 ceph-mon[35477]: pgmap v179: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 250 KiB/s wr, 47 op/s 2024-03-21T08:37:44.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:43 smithi171 ceph-mon[35388]: pgmap v179: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 250 KiB/s wr, 47 op/s 2024-03-21T08:37:44.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:43 smithi003 ceph-mon[31675]: pgmap v179: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 250 KiB/s wr, 47 op/s 2024-03-21T08:37:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.189+0000 7fb683e00640 1 -- 172.21.15.3:0/1784884962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c103870 msgr2=0x7fb67c103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.189+0000 7fb683e00640 1 --2- 172.21.15.3:0/1784884962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c103870 0x7fb67c103cf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb668009930 tx=0x7fb66802bc30 comp rx=0 tx=0).stop 2024-03-21T08:37:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.189+0000 7fb683e00640 1 -- 172.21.15.3:0/1784884962 shutdown_connections 2024-03-21T08:37:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.189+0000 7fb683e00640 1 --2- 172.21.15.3:0/1784884962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb67c104230 0x7fb67c110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.189+0000 7fb683e00640 1 --2- 172.21.15.3:0/1784884962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c103870 0x7fb67c103cf0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.189+0000 7fb683e00640 1 --2- 172.21.15.3:0/1784884962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb67c109880 0x7fb67c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.189+0000 7fb683e00640 1 -- 172.21.15.3:0/1784884962 >> 172.21.15.3:0/1784884962 conn(0x7fb67c0ff320 msgr2=0x7fb67c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.190+0000 7fb683e00640 1 -- 172.21.15.3:0/1784884962 shutdown_connections 2024-03-21T08:37:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.190+0000 7fb683e00640 1 -- 172.21.15.3:0/1784884962 wait complete. 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 Processor -- start 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 -- start start 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb67c103870 0x7fb67c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c104230 0x7fb67c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb67c109880 0x7fb67c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c113b40 con 0x7fb67c104230 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1139c0 con 0x7fb67c109880 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.191+0000 7fb683e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c113cc0 con 0x7fb67c103870 2024-03-21T08:37:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c104230 0x7fb67c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c104230 0x7fb67c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49862/0 (socket says 172.21.15.3:49862) 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 -- 172.21.15.3:0/2752792704 learned_addr learned my addr 172.21.15.3:0/2752792704 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb681e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb67c109880 0x7fb67c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb681400640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb67c103870 0x7fb67c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 -- 172.21.15.3:0/2752792704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb67c103870 msgr2=0x7fb67c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb67c103870 0x7fb67c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 -- 172.21.15.3:0/2752792704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb67c109880 msgr2=0x7fb67c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb67c109880 0x7fb67c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.192+0000 7fb680a00640 1 -- 172.21.15.3:0/2752792704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c19f080 con 0x7fb67c104230 2024-03-21T08:37:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.193+0000 7fb680a00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c104230 0x7fb67c19a620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb668009230 tx=0x7fb66802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.193+0000 7fb67a000640 1 -- 172.21.15.3:0/2752792704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668043020 con 0x7fb67c104230 2024-03-21T08:37:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.193+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c19f310 con 0x7fb67c104230 2024-03-21T08:37:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.193+0000 7fb67a000640 1 -- 172.21.15.3:0/2752792704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb66802caf0 con 0x7fb67c104230 2024-03-21T08:37:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.193+0000 7fb67a000640 1 -- 172.21.15.3:0/2752792704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668003ae0 con 0x7fb67c104230 2024-03-21T08:37:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.193+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c1a6c00 con 0x7fb67c104230 2024-03-21T08:37:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.197+0000 7fb67a000640 1 -- 172.21.15.3:0/2752792704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fb668046020 con 0x7fb67c104230 2024-03-21T08:37:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.197+0000 7fb686bc7740 1 Processor -- start 2024-03-21T08:37:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.197+0000 7fb686bc7740 1 -- start start 2024-03-21T08:37:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc9223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc92b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc92dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb686bc7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc8e43b0 con 0x5623cc92b6c0 2024-03-21T08:37:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb686bc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc886830 con 0x5623cc9228f0 2024-03-21T08:37:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb686bc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc82f5d0 con 0x5623cc921fd0 2024-03-21T08:37:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb680a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc92dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb680a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc92dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49864/0 (socket says 172.21.15.3:49864) 2024-03-21T08:37:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb680a00640 1 -- 172.21.15.3:0/1671524861 learned_addr learned my addr 172.21.15.3:0/1671524861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.198+0000 7fb681400640 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc92b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb681e00640 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc9223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb67a000640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb6640a3080 0x7fb6640a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb680a00640 1 -- 172.21.15.3:0/1671524861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 msgr2=0x5623cc9223b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb680a00640 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc9223b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb680a00640 1 -- 172.21.15.3:0/1671524861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 msgr2=0x5623cc92b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb680a00640 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc92b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb680a00640 1 -- 172.21.15.3:0/1671524861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623cc8472c0 con 0x5623cc92b6c0 2024-03-21T08:37:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb67a000640 1 -- 172.21.15.3:0/2752792704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fb6680f1cd0 con 0x7fb67c104230 2024-03-21T08:37:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb681e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb6640a3080 0x7fb6640a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.199+0000 7fb680a00640 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc92dab0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb6680ef080 tx=0x7fb66804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed99573c25fab132 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.200+0000 7fb66e600640 1 -- 172.21.15.3:0/1671524861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680bdb80 con 0x5623cc92b6c0 2024-03-21T08:37:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.200+0000 7fb66e600640 1 -- 172.21.15.3:0/1671524861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb66803c350 con 0x5623cc92b6c0 2024-03-21T08:37:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.200+0000 7fb66e600640 1 -- 172.21.15.3:0/1671524861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66802cca0 con 0x5623cc92b6c0 2024-03-21T08:37:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 -- 172.21.15.3:0/1671524861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 msgr2=0x5623cc92dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc92dab0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb6680ef080 tx=0x7fb66804f000 comp rx=0 tx=0).stop 2024-03-21T08:37:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb681e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb6640a3080 0x7fb6640a5540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb674005fd0 tx=0x7fb674004a40 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 -- 172.21.15.3:0/1671524861 shutdown_connections 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc92dab0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc92b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 --2- 172.21.15.3:0/1671524861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc9223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 -- 172.21.15.3:0/1671524861 >> 172.21.15.3:0/1671524861 conn(0x5623cc91c8f0 msgr2=0x5623cc91ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 -- 172.21.15.3:0/1671524861 shutdown_connections 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 -- 172.21.15.3:0/1671524861 wait complete. 2024-03-21T08:37:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 -- start start 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.201+0000 7fb686bc7740 1 Processor -- start 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc928640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc928b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc9290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb686bc7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc842a20 con 0x5623cc92b6c0 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb686bc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc8828d0 con 0x5623cc9228f0 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb686bc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc92e0b0 con 0x5623cc921fd0 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb686bc7740 10 client.0 _ll_register_callbacks cb 0x5623cc87f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb681e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc9290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb681e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc9290c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49866/0 (socket says 172.21.15.3:49866) 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb681e00640 1 -- 172.21.15.3:0/3381081203 learned_addr learned my addr 172.21.15.3:0/3381081203 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.202+0000 7fb680a00640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc928b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb681400640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc928640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb681e00640 1 -- 172.21.15.3:0/3381081203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 msgr2=0x5623cc928640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb681e00640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc928640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb681e00640 1 -- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 msgr2=0x5623cc928b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb681e00640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc928b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb681e00640 1 -- 172.21.15.3:0/3381081203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623cc59b210 con 0x5623cc92b6c0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb680a00640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc928b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb681e00640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc9290c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb674012850 tx=0x7fb674004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7518e12b94a9a51 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.203+0000 7fb662a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.204+0000 7fb662a00640 1 -- 172.21.15.3:0/3381081203 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67401db60 con 0x5623cc92b6c0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.204+0000 7fb662a00640 1 -- 172.21.15.3:0/3381081203 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fb67401dd00 con 0x5623cc92b6c0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.204+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb66803c350 con 0x5623cc92b6c0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.204+0000 7fb662a00640 1 -- 172.21.15.3:0/3381081203 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb674027470 con 0x5623cc92b6c0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.204+0000 7fb686bc7740 10 client.16719 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.204+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5623cc667620 con 0x5623cc92b6c0 2024-03-21T08:37:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.204+0000 7fb662a00640 1 -- 172.21.15.3:0/3381081203 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb66803c350 con 0x5623cc92b6c0 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb662a00640 1 -- 172.21.15.3:0/3381081203 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb67400f040 con 0x5623cc92b6c0 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb686bc7740 10 client.16719 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb686bc7740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb686bc7740 20 client.16719 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc93b350 0x5623cc93d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb686bc7740 4 client.16719 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623cc893d70 con 0x5623cc93b350 2024-03-21T08:37:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.205+0000 7fb681400640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc93b350 0x5623cc93d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.206+0000 7fb681400640 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc93b350 0x5623cc93d740 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710de6a14b0b0b65 server_cookie=be71893da60f3676 in_seq=0 out_seq=0 2024-03-21T08:37:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.206+0000 7fb662a00640 10 client.16719 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb662a00640 1 -- 172.21.15.3:0/3381081203 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5623cc893d70 con 0x5623cc93b350 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb662a00640 10 client.16719 handle_command_reply: tid=0 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 client.16719 shutdown 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 msgr2=0x5623cc9290c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc9290c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb674012850 tx=0x7fb674004ab0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 shutdown_connections 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc92b6c0 0x5623cc9290c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc928b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc93b350 0x5623cc93d740 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3381081203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc921fd0 0x5623cc928640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 >> 172.21.15.3:0/3381081203 conn(0x5623cc91c8f0 msgr2=0x5623cc91e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb662a00640 0 client.16719 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 shutdown_connections 2024-03-21T08:37:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 1 -- 172.21.15.3:0/3381081203 wait complete. 2024-03-21T08:37:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.208+0000 7fb686bc7740 20 client.16719 trim_cache size 0 max 16384 2024-03-21T08:37:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.225+0000 7fb686bc7740 1 Processor -- start 2024-03-21T08:37:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.225+0000 7fb686bc7740 1 -- start start 2024-03-21T08:37:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc91e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc92b6c0 0x5623cc9366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb686bc7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1b3410 con 0x5623cc921fd0 2024-03-21T08:37:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb686bc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1b0eb0 con 0x5623cc9228f0 2024-03-21T08:37:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb686bc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1aef50 con 0x5623cc92b6c0 2024-03-21T08:37:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49878/0 (socket says 172.21.15.3:49878) 2024-03-21T08:37:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 -- 172.21.15.3:0/2141073753 learned_addr learned my addr 172.21.15.3:0/2141073753 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681e00640 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc92b6c0 0x5623cc9366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb680a00640 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc91e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 -- 172.21.15.3:0/2141073753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc92b6c0 msgr2=0x5623cc9366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc92b6c0 0x5623cc9366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 -- 172.21.15.3:0/2141073753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 msgr2=0x5623cc91e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc91e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.226+0000 7fb681400640 1 -- 172.21.15.3:0/2141073753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623cc91c720 con 0x5623cc921fd0 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.227+0000 7fb680a00640 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc91e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.227+0000 7fb681400640 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92ad70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb670009b30 tx=0x7fb670005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dcf07688ea81ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.227+0000 7fb66dc00640 1 -- 172.21.15.3:0/2141073753 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700202f0 con 0x5623cc921fd0 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.227+0000 7fb66dc00640 1 -- 172.21.15.3:0/2141073753 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb670024030 con 0x5623cc921fd0 2024-03-21T08:37:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.227+0000 7fb66dc00640 1 -- 172.21.15.3:0/2141073753 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670004e60 con 0x5623cc921fd0 2024-03-21T08:37:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 -- 172.21.15.3:0/2141073753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 msgr2=0x5623cc92ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92ad70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb670009b30 tx=0x7fb670005f90 comp rx=0 tx=0).stop 2024-03-21T08:37:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 -- 172.21.15.3:0/2141073753 shutdown_connections 2024-03-21T08:37:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc92b6c0 0x5623cc9366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc9228f0 0x5623cc91e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 --2- 172.21.15.3:0/2141073753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92ad70 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 -- 172.21.15.3:0/2141073753 >> 172.21.15.3:0/2141073753 conn(0x5623cc91c8f0 msgr2=0x5623cc91dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 -- 172.21.15.3:0/2141073753 shutdown_connections 2024-03-21T08:37:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.228+0000 7fb686bc7740 1 -- 172.21.15.3:0/2141073753 wait complete. 2024-03-21T08:37:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.229+0000 7fb686bc7740 1 -- start start 2024-03-21T08:37:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.229+0000 7fb686bc7740 1 Processor -- start 2024-03-21T08:37:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc9228f0 0x5623cc92ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb686bc7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc92b6c0 0x5623cc92f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb686bc7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1af550 con 0x5623cc921fd0 2024-03-21T08:37:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb686bc7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1b0850 con 0x5623cc92b6c0 2024-03-21T08:37:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb686bc7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1afb90 con 0x5623cc9228f0 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb686bc7740 10 client.0 _ll_register_callbacks cb 0x7fb674012550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb681400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb681400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49886/0 (socket says 172.21.15.3:49886) 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb681400640 1 -- 172.21.15.3:0/3007377937 learned_addr learned my addr 172.21.15.3:0/3007377937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb681e00640 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc92b6c0 0x5623cc92f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.230+0000 7fb680a00640 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc9228f0 0x5623cc92ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb681400640 1 -- 172.21.15.3:0/3007377937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc9228f0 msgr2=0x5623cc92ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb681400640 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc9228f0 0x5623cc92ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb681400640 1 -- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc92b6c0 msgr2=0x5623cc92f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb681400640 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc92b6c0 0x5623cc92f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb681400640 1 -- 172.21.15.3:0/3007377937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623cc995f10 con 0x5623cc921fd0 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb681400640 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92e900 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb670004ce0 tx=0x7fb6700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4ea21a35b19d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb662a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb662a00640 1 -- 172.21.15.3:0/3007377937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700200f0 con 0x5623cc921fd0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb662a00640 1 -- 172.21.15.3:0/3007377937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fb670020290 con 0x5623cc921fd0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.231+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb670024030 con 0x5623cc921fd0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb662a00640 1 -- 172.21.15.3:0/3007377937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb670038050 con 0x5623cc921fd0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb686bc7740 10 client.16725 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5623cc997190 con 0x5623cc921fd0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb662a00640 1 -- 172.21.15.3:0/3007377937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb670024030 con 0x5623cc921fd0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb662a00640 1 -- 172.21.15.3:0/3007377937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb670012b50 con 0x5623cc921fd0 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb686bc7740 10 client.16725 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb686bc7740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb686bc7740 20 client.16725 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.232+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc997850 0x5623cc999c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.233+0000 7fb686bc7740 4 client.16725 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:37:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.233+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623cc88e0c0 con 0x5623cc997850 2024-03-21T08:37:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.233+0000 7fb680a00640 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc997850 0x5623cc999c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.234+0000 7fb680a00640 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc997850 0x5623cc999c70 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da9368a29d510cd server_cookie=d1de2ae75a30452e in_seq=0 out_seq=0 2024-03-21T08:37:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.234+0000 7fb662a00640 10 client.16725 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb662a00640 1 -- 172.21.15.3:0/3007377937 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623cc88e0c0 con 0x5623cc997850 2024-03-21T08:37:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb662a00640 10 client.16725 handle_command_reply: tid=0 2024-03-21T08:37:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 client.16725 shutdown 2024-03-21T08:37:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 msgr2=0x5623cc92e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92e900 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb670004ce0 tx=0x7fb6700047b0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 shutdown_connections 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5623cc92b6c0 0x5623cc92f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5623cc997850 0x5623cc999c70 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623cc9228f0 0x5623cc92ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 --2- 172.21.15.3:0/3007377937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5623cc921fd0 0x5623cc92e900 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 >> 172.21.15.3:0/3007377937 conn(0x5623cc91c8f0 msgr2=0x5623cc928600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb662a00640 0 client.16725 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 shutdown_connections 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.270+0000 7fb686bc7740 1 -- 172.21.15.3:0/3007377937 wait complete. 2024-03-21T08:37:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb686bc7740 20 client.16725 trim_cache size 0 max 16384 2024-03-21T08:37:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:37:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:37:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:37:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:37:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb6640a3080 msgr2=0x7fb6640a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb6640a3080 0x7fb6640a5540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb674005fd0 tx=0x7fb674004a40 comp rx=0 tx=0).stop 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c104230 msgr2=0x7fb67c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c104230 0x7fb67c19a620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb668009230 tx=0x7fb66802c710 comp rx=0 tx=0).stop 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 shutdown_connections 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb67c109880 0x7fb67c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb67c104230 0x7fb67c19a620 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.271+0000 7fb683e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fb6640a3080 0x7fb6640a5540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.272+0000 7fb683e00640 1 --2- 172.21.15.3:0/2752792704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb67c103870 0x7fb67c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.272+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 >> 172.21.15.3:0/2752792704 conn(0x7fb67c0ff320 msgr2=0x7fb67c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.272+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 shutdown_connections 2024-03-21T08:37:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:44.272+0000 7fb683e00640 1 -- 172.21.15.3:0/2752792704 wait complete. 2024-03-21T08:37:46.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:45 smithi084 ceph-mon[35477]: pgmap v180: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 159 KiB/s wr, 28 op/s 2024-03-21T08:37:46.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:45 smithi171 ceph-mon[35388]: pgmap v180: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 159 KiB/s wr, 28 op/s 2024-03-21T08:37:46.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:45 smithi003 ceph-mon[31675]: pgmap v180: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 159 KiB/s wr, 28 op/s 2024-03-21T08:37:48.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:47 smithi084 ceph-mon[35477]: pgmap v181: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-03-21T08:37:48.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:47 smithi171 ceph-mon[35388]: pgmap v181: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-03-21T08:37:48.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:47 smithi003 ceph-mon[31675]: pgmap v181: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-03-21T08:37:50.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:49 smithi084 ceph-mon[35477]: pgmap v182: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-03-21T08:37:50.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:49 smithi171 ceph-mon[35388]: pgmap v182: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-03-21T08:37:50.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:49 smithi003 ceph-mon[31675]: pgmap v182: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-03-21T08:37:52.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:51 smithi084 ceph-mon[35477]: pgmap v183: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-03-21T08:37:52.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:51 smithi171 ceph-mon[35388]: pgmap v183: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-03-21T08:37:52.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:51 smithi003 ceph-mon[31675]: pgmap v183: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-03-21T08:37:54.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:54 smithi084 ceph-mon[35477]: pgmap v184: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 93 KiB/s wr, 21 op/s 2024-03-21T08:37:54.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:54 smithi171 ceph-mon[35388]: pgmap v184: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 93 KiB/s wr, 21 op/s 2024-03-21T08:37:54.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:54 smithi003 ceph-mon[31675]: pgmap v184: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 93 KiB/s wr, 21 op/s 2024-03-21T08:37:56.089 INFO:tasks.workunit.client.0.smithi003.stdout: [DEP] 2024-03-21T08:37:56.091 INFO:tasks.workunit.client.0.smithi003.stdout: [DEP] 2024-03-21T08:37:56.101 INFO:tasks.workunit.client.0.smithi003.stdout: [DEP] 2024-03-21T08:37:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:56 smithi084 ceph-mon[35477]: pgmap v185: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-03-21T08:37:56.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:56 smithi171 ceph-mon[35388]: pgmap v185: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-03-21T08:37:56.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:56 smithi003 ceph-mon[31675]: pgmap v185: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-03-21T08:37:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 -- 172.21.15.3:0/931161353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 msgr2=0x7f939c109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 --2- 172.21.15.3:0/931161353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c109a70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f938c009930 tx=0x7f938c02bc20 comp rx=0 tx=0).stop 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 -- 172.21.15.3:0/931161353 shutdown_connections 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 --2- 172.21.15.3:0/931161353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f939c104040 0x7f939c110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 --2- 172.21.15.3:0/931161353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f939c103680 0x7f939c103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 --2- 172.21.15.3:0/931161353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c109a70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 -- 172.21.15.3:0/931161353 >> 172.21.15.3:0/931161353 conn(0x7f939c0ff110 msgr2=0x7f939c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 -- 172.21.15.3:0/931161353 shutdown_connections 2024-03-21T08:37:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.431+0000 7f939be00640 1 -- 172.21.15.3:0/931161353 wait complete. 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.432+0000 7f939be00640 1 Processor -- start 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.432+0000 7f939be00640 1 -- start start 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f939c103680 0x7f939c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f939c104040 0x7f939c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c113950 con 0x7f939c109690 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1137d0 con 0x7f939c103680 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c113ad0 con 0x7f939c104040 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33456/0 (socket says 172.21.15.3:33456) 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.433+0000 7f939b400640 1 -- 172.21.15.3:0/1798452118 learned_addr learned my addr 172.21.15.3:0/1798452118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939aa00640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f939c103680 0x7f939c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939a000640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f939c104040 0x7f939c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939b400640 1 -- 172.21.15.3:0/1798452118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f939c104040 msgr2=0x7f939c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939b400640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f939c104040 0x7f939c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939b400640 1 -- 172.21.15.3:0/1798452118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f939c103680 msgr2=0x7f939c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939b400640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f939c103680 0x7f939c199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939b400640 1 -- 172.21.15.3:0/1798452118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c19eee0 con 0x7f939c109690 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f939b400640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c19e7b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f939400eaa0 tx=0x7f939400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.434+0000 7f9393400640 1 -- 172.21.15.3:0/1798452118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400cdb0 con 0x7f939c109690 2024-03-21T08:37:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.435+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c19f1d0 con 0x7f939c109690 2024-03-21T08:37:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.435+0000 7f9393400640 1 -- 172.21.15.3:0/1798452118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9394005020 con 0x7f939c109690 2024-03-21T08:37:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.435+0000 7f9393400640 1 -- 172.21.15.3:0/1798452118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940116d0 con 0x7f939c109690 2024-03-21T08:37:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.435+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c1a6a60 con 0x7f939c109690 2024-03-21T08:37:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.438+0000 7f93a483a740 1 Processor -- start 2024-03-21T08:37:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.438+0000 7f9393400640 1 -- 172.21.15.3:0/1798452118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f93940051d0 con 0x7f939c109690 2024-03-21T08:37:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f93a483a740 1 -- start start 2024-03-21T08:37:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f93a483a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f93a483a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf51390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f93a483a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f93a483a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571aaf0a3b0 con 0x5571aaf481e0 2024-03-21T08:37:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f93a483a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571aaeac830 con 0x5571aaf48b00 2024-03-21T08:37:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f93a483a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571aae555d0 con 0x5571aaf518d0 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.439+0000 7f939a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33460/0 (socket says 172.21.15.3:33460) 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf51390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 -- 172.21.15.3:0/2265920020 learned_addr learned my addr 172.21.15.3:0/2265920020 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 -- 172.21.15.3:0/2265920020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 msgr2=0x5571aaf53cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf53cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 -- 172.21.15.3:0/2265920020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 msgr2=0x5571aaf51390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf51390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f9393400640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f93800a3070 0x7f93800a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f939a000640 1 -- 172.21.15.3:0/2265920020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571aae6d2c0 con 0x5571aaf481e0 2024-03-21T08:37:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.440+0000 7f9393400640 1 -- 172.21.15.3:0/1798452118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f93940cfdf0 con 0x7f939c109690 2024-03-21T08:37:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.441+0000 7f939a000640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f93800a3070 0x7f93800a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.441+0000 7f939a000640 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf485c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f93880058c0 tx=0x7f938800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df8929e1ce2402f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.441+0000 7f9387e00640 1 -- 172.21.15.3:0/2265920020 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388019040 con 0x5571aaf481e0 2024-03-21T08:37:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.442+0000 7f9387e00640 1 -- 172.21.15.3:0/2265920020 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9388022e10 con 0x5571aaf481e0 2024-03-21T08:37:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.442+0000 7f9387e00640 1 -- 172.21.15.3:0/2265920020 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388020b30 con 0x5571aaf481e0 2024-03-21T08:37:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.442+0000 7f93a483a740 1 -- 172.21.15.3:0/2265920020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 msgr2=0x5571aaf485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.442+0000 7f93a483a740 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf485c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f93880058c0 tx=0x7f938800b6b0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.442+0000 7f939a000640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f93800a3070 0x7f93800a5530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9388021000 tx=0x7f938801e000 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 -- 172.21.15.3:0/2265920020 shutdown_connections 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf53cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf51390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 --2- 172.21.15.3:0/2265920020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf485c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 -- 172.21.15.3:0/2265920020 >> 172.21.15.3:0/2265920020 conn(0x5571aaf42b00 msgr2=0x5571aaf42ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 -- 172.21.15.3:0/2265920020 shutdown_connections 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 -- 172.21.15.3:0/2265920020 wait complete. 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 -- start start 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.443+0000 7f93a483a740 1 Processor -- start 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f93a483a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf481e0 0x5571aaf4e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f93a483a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf48b00 0x5571aaf4ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f93a483a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf4f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f93a483a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571aae68a20 con 0x5571aaf48b00 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f93a483a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571aaea88d0 con 0x5571aaf481e0 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f93a483a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571aaf54500 con 0x5571aaf518d0 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f93a483a740 10 client.0 _ll_register_callbacks cb 0x5571aaea51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f939a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf48b00 0x5571aaf4ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f939a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf48b00 0x5571aaf4ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33464/0 (socket says 172.21.15.3:33464) 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.444+0000 7f939a000640 1 -- 172.21.15.3:0/3551992759 learned_addr learned my addr 172.21.15.3:0/3551992759 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939aa00640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf481e0 0x5571aaf4e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939b400640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf4f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939a000640 1 -- 172.21.15.3:0/3551992759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 msgr2=0x5571aaf4f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939a000640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf4f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939a000640 1 -- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf481e0 msgr2=0x5571aaf4e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939a000640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf481e0 0x5571aaf4e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939a000640 1 -- 172.21.15.3:0/3551992759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571aabc1210 con 0x5571aaf48b00 2024-03-21T08:37:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939b400640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf4f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:37:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939aa00640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf481e0 0x5571aaf4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:37:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f939a000640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf48b00 0x5571aaf4ee40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f938800b440 tx=0x7f9388005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a918da6c995374 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.445+0000 7f9384c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.446+0000 7f9384c00640 1 -- 172.21.15.3:0/3551992759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388011760 con 0x5571aaf48b00 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.446+0000 7f9384c00640 1 -- 172.21.15.3:0/3551992759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f93880203d0 con 0x5571aaf48b00 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.446+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9388022e10 con 0x5571aaf48b00 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.446+0000 7f9384c00640 1 -- 172.21.15.3:0/3551992759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938803d050 con 0x5571aaf48b00 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.446+0000 7f93a483a740 10 client.16737 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.446+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5571aac8d620 con 0x5571aaf48b00 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.446+0000 7f9384c00640 1 -- 172.21.15.3:0/3551992759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9388022e10 con 0x5571aaf48b00 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f9384c00640 1 -- 172.21.15.3:0/3551992759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9388005040 con 0x5571aaf48b00 2024-03-21T08:37:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f93a483a740 10 client.16737 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f93a483a740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f93a483a740 20 client.16737 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f93a483a740 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aaf617c0 0x5571aaf63bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f93a483a740 4 client.16737 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571aaeb9d70 con 0x5571aaf617c0 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.447+0000 7f939aa00640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aaf617c0 0x5571aaf63bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.448+0000 7f939aa00640 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aaf617c0 0x5571aaf63bb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb80e248270f4ffb server_cookie=e7cc8fabaeaa09fa in_seq=0 out_seq=0 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.448+0000 7f9384c00640 10 client.16737 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f9384c00640 1 -- 172.21.15.3:0/3551992759 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5571aaeb9d70 con 0x5571aaf617c0 2024-03-21T08:37:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f9384c00640 10 client.16737 handle_command_reply: tid=0 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 client.16737 shutdown 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf48b00 msgr2=0x5571aaf4ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf48b00 0x5571aaf4ee40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f938800b440 tx=0x7f9388005850 comp rx=0 tx=0).stop 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 shutdown_connections 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aaf617c0 0x5571aaf63bb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aaf4f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf48b00 0x5571aaf4ee40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 --2- 172.21.15.3:0/3551992759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf481e0 0x5571aaf4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 >> 172.21.15.3:0/3551992759 conn(0x5571aaf42b00 msgr2=0x5571aaf44850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f9384c00640 0 client.16737 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 shutdown_connections 2024-03-21T08:37:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.450+0000 7f93a483a740 1 -- 172.21.15.3:0/3551992759 wait complete. 2024-03-21T08:37:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.451+0000 7f93a483a740 20 client.16737 trim_cache size 0 max 16384 2024-03-21T08:37:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.469+0000 7f93a483a740 1 Processor -- start 2024-03-21T08:37:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f93a483a740 1 -- start start 2024-03-21T08:37:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f93a483a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf4efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f93a483a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf48b00 0x5571aaf5cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f93a483a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf518d0 0x5571aaf5d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f93a483a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1b32b0 con 0x5571aaf481e0 2024-03-21T08:37:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f93a483a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1b0d50 con 0x5571aaf518d0 2024-03-21T08:37:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f93a483a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1aedf0 con 0x5571aaf48b00 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f939aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf4efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f939aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf4efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33480/0 (socket says 172.21.15.3:33480) 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.470+0000 7f939aa00640 1 -- 172.21.15.3:0/1689986879 learned_addr learned my addr 172.21.15.3:0/1689986879 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939a000640 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf48b00 0x5571aaf5cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939b400640 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf518d0 0x5571aaf5d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939aa00640 1 -- 172.21.15.3:0/1689986879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf48b00 msgr2=0x5571aaf5cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939aa00640 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf48b00 0x5571aaf5cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939aa00640 1 -- 172.21.15.3:0/1689986879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf518d0 msgr2=0x5571aaf5d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939aa00640 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf518d0 0x5571aaf5d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939aa00640 1 -- 172.21.15.3:0/1689986879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571aaf5d690 con 0x5571aaf481e0 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.471+0000 7f939aa00640 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf4efd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f938c0096c0 tx=0x7f938c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923f2cc79a2f69b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.472+0000 7f9387400640 1 -- 172.21.15.3:0/1689986879 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c04d020 con 0x5571aaf481e0 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.472+0000 7f9387400640 1 -- 172.21.15.3:0/1689986879 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f938c02cc80 con 0x5571aaf481e0 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.472+0000 7f9387400640 1 -- 172.21.15.3:0/1689986879 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c047070 con 0x5571aaf481e0 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.472+0000 7f93a483a740 1 -- 172.21.15.3:0/1689986879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 msgr2=0x5571aaf4efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.472+0000 7f93a483a740 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf4efd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f938c0096c0 tx=0x7f938c005a40 comp rx=0 tx=0).stop 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 -- 172.21.15.3:0/1689986879 shutdown_connections 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf518d0 0x5571aaf5d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf48b00 0x5571aaf5cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 --2- 172.21.15.3:0/1689986879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf4efd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 -- 172.21.15.3:0/1689986879 >> 172.21.15.3:0/1689986879 conn(0x5571aaf42b00 msgr2=0x5571aaf4e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 -- 172.21.15.3:0/1689986879 shutdown_connections 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 -- 172.21.15.3:0/1689986879 wait complete. 2024-03-21T08:37:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 -- start start 2024-03-21T08:37:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.473+0000 7f93a483a740 1 Processor -- start 2024-03-21T08:37:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f93a483a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf55250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f93a483a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf55790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f93a483a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aafbbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f93a483a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1af3f0 con 0x5571aaf481e0 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f93a483a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1b06f0 con 0x5571aaf48b00 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f93a483a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1afa30 con 0x5571aaf518d0 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f93a483a740 10 client.0 _ll_register_callbacks cb 0x7f938c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f939aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf55250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f939aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf55250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33482/0 (socket says 172.21.15.3:33482) 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f939aa00640 1 -- 172.21.15.3:0/4177623792 learned_addr learned my addr 172.21.15.3:0/4177623792 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:37:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.474+0000 7f939a000640 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf55790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f939b400640 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aafbbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f939aa00640 1 -- 172.21.15.3:0/4177623792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 msgr2=0x5571aafbbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f939aa00640 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aafbbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f939aa00640 1 -- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 msgr2=0x5571aaf55790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f939aa00640 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf55790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f939aa00640 1 -- 172.21.15.3:0/4177623792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571aafbc340 con 0x5571aaf481e0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f939aa00640 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf55250 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f938c02c940 tx=0x7f938c02c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a20b6f08b13b889 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f9384c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f9384c00640 1 -- 172.21.15.3:0/4177623792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c055020 con 0x5571aaf481e0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f9384c00640 1 -- 172.21.15.3:0/4177623792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f938c0040c0 con 0x5571aaf481e0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.475+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f938c02cc80 con 0x5571aaf481e0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f9384c00640 1 -- 172.21.15.3:0/4177623792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938c04d040 con 0x5571aaf481e0 2024-03-21T08:37:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 10 client.16743 fetch_fsmap learned FSMap version 30 2024-03-21T08:37:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5571aafbd620 con 0x5571aaf481e0 2024-03-21T08:37:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f9384c00640 1 -- 172.21.15.3:0/4177623792 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f938c02cc80 con 0x5571aaf481e0 2024-03-21T08:37:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f9384c00640 1 -- 172.21.15.3:0/4177623792 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f938c0035f0 con 0x5571aaf481e0 2024-03-21T08:37:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 10 client.16743 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T08:37:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.h 2024-03-21T08:37:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 20 client.16743 populate_metadata read hostname 'smithi003' 2024-03-21T08:37:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aafbdd70 0x5571aafc0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:37:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 4 client.16743 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T08:37:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.476+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 --> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5571aaeb40c0 con 0x5571aafbdd70 2024-03-21T08:37:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.477+0000 7f939a000640 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aafbdd70 0x5571aafc0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:37:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.478+0000 7f939a000640 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aafbdd70 0x5571aafc0190 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0afcf36e9dd7b44 server_cookie=b39dc443270bc9cc in_seq=0 out_seq=0 2024-03-21T08:37:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.478+0000 7f9384c00640 10 client.16743 ms_handle_connect on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.503+0000 7f9384c00640 1 -- 172.21.15.3:0/4177623792 <== mds.0 v2:172.21.15.84:6836/3332280865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5571aaeb40c0 con 0x5571aafbdd70 2024-03-21T08:37:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.503+0000 7f9384c00640 10 client.16743 handle_command_reply: tid=0 2024-03-21T08:37:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 client.16743 shutdown 2024-03-21T08:37:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 msgr2=0x5571aaf55250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf55250 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f938c02c940 tx=0x7f938c02c970 comp rx=0 tx=0).stop 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 shutdown_connections 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:6836/3332280865,v1:172.21.15.84:6837/3332280865] conn(0x5571aafbdd70 0x5571aafc0190 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5571aaf518d0 0x5571aafbbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571aaf48b00 0x5571aaf55790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f9384c00640 0 client.16743 ms_handle_reset on v2:172.21.15.84:6836/3332280865 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 --2- 172.21.15.3:0/4177623792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571aaf481e0 0x5571aaf55250 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 >> 172.21.15.3:0/4177623792 conn(0x5571aaf42b00 msgr2=0x5571aaf41b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 shutdown_connections 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 1 -- 172.21.15.3:0/4177623792 wait complete. 2024-03-21T08:37:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.504+0000 7f93a483a740 20 client.16743 trim_cache size 0 max 16384 2024-03-21T08:37:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-03-21T08:37:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-03-21T08:37:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-03-21T08:37:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-03-21T08:37:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f93800a3070 msgr2=0x7f93800a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f93800a3070 0x7f93800a5530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9388021000 tx=0x7f938801e000 comp rx=0 tx=0).stop 2024-03-21T08:37:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 msgr2=0x7f939c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:37:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c19e7b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f939400eaa0 tx=0x7f939400ef60 comp rx=0 tx=0).stop 2024-03-21T08:37:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 shutdown_connections 2024-03-21T08:37:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f939c109690 0x7f939c19e7b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f93800a3070 0x7f93800a5530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f939c104040 0x7f939c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.505+0000 7f939be00640 1 --2- 172.21.15.3:0/1798452118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f939c103680 0x7f939c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:37:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.506+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 >> 172.21.15.3:0/1798452118 conn(0x7f939c0ff110 msgr2=0x7f939c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:37:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.506+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 shutdown_connections 2024-03-21T08:37:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-03-21T08:37:57.506+0000 7f939be00640 1 -- 172.21.15.3:0/1798452118 wait complete. 2024-03-21T08:37:58.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:37:58 smithi084 ceph-mon[35477]: pgmap v186: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 18 op/s 2024-03-21T08:37:58.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:37:58 smithi171 ceph-mon[35388]: pgmap v186: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 18 op/s 2024-03-21T08:37:58.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:37:58 smithi003 ceph-mon[31675]: pgmap v186: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 18 op/s 2024-03-21T08:37:59.002 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] dirstress 2024-03-21T08:37:59.029 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] fill 2024-03-21T08:37:59.029 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] fill2 2024-03-21T08:37:59.039 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] getpagesize 2024-03-21T08:37:59.899 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] holes 2024-03-21T08:37:59.932 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] lstat64 2024-03-21T08:38:00.111 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] nametest 2024-03-21T08:38:00.128 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] permname 2024-03-21T08:38:00.625 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] randholes 2024-03-21T08:38:00.699 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] runas 2024-03-21T08:38:00.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:00 smithi084 ceph-mon[35477]: pgmap v187: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-03-21T08:38:00.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:00 smithi171 ceph-mon[35388]: pgmap v187: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-03-21T08:38:00.910 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] truncfile 2024-03-21T08:38:00.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:00 smithi003 ceph-mon[31675]: pgmap v187: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-03-21T08:38:01.021 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] usemem 2024-03-21T08:38:01.490 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] mmapcat 2024-03-21T08:38:01.549 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] append_reader 2024-03-21T08:38:01.643 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] append_writer 2024-03-21T08:38:01.670 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] dirperf 2024-03-21T08:38:01.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:01 smithi084 ceph-mon[35477]: pgmap v188: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-03-21T08:38:01.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:01 smithi171 ceph-mon[35388]: pgmap v188: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-03-21T08:38:01.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:01 smithi003 ceph-mon[31675]: pgmap v188: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-03-21T08:38:02.088 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] metaperf 2024-03-21T08:38:02.202 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] devzero 2024-03-21T08:38:02.272 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] feature 2024-03-21T08:38:02.419 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] alloc 2024-03-21T08:38:02.892 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] fault 2024-03-21T08:38:02.916 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] fstest 2024-03-21T08:38:02.979 INFO:tasks.workunit.client.0.smithi003.stderr:alloc.c: In function 'main': 2024-03-21T08:38:02.979 INFO:tasks.workunit.client.0.smithi003.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: 154 | bozo! 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~ 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: 154 | bozo! 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: | ; 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: 184 | rflag = 1; 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~ 2024-03-21T08:38:02.980 INFO:tasks.workunit.client.0.smithi003.stderr: | nflag 2024-03-21T08:38:02.981 INFO:tasks.workunit.client.0.smithi003.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-03-21T08:38:02.981 INFO:tasks.workunit.client.0.smithi003.stderr: 320 | bozo! 2024-03-21T08:38:02.981 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-03-21T08:38:02.981 INFO:tasks.workunit.client.0.smithi003.stderr: | ; 2024-03-21T08:38:02.981 INFO:tasks.workunit.client.0.smithi003.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-03-21T08:38:02.981 INFO:tasks.workunit.client.0.smithi003.stderr: 128 | struct flock64 f; 2024-03-21T08:38:02.981 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-03-21T08:38:02.989 INFO:tasks.workunit.client.0.smithi003.stderr:gmake[3]: *** [Makefile:82: alloc] Error 1 2024-03-21T08:38:02.989 INFO:tasks.workunit.client.0.smithi003.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-03-21T08:38:02.989 INFO:tasks.workunit.client.0.smithi003.stdout: [CC] t_access_root 2024-03-21T08:38:03.561 INFO:tasks.workunit.client.0.smithi003.stderr:fstest.c: In function 'run_child': 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~ 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~ 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~ 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-03-21T08:38:03.562 INFO:tasks.workunit.client.0.smithi003.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-21T08:38:03.563 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~ 2024-03-21T08:38:03.563 INFO:tasks.workunit.client.0.smithi003.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-03-21T08:38:03.563 INFO:tasks.workunit.client.0.smithi003.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-21T08:38:03.563 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-21T08:38:03.762 INFO:tasks.workunit.client.0.smithi003.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-03-21T08:38:03.764 INFO:tasks.workunit.client.0.smithi003.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-03-21T08:38:03.767 INFO:tasks.workunit.client.0.smithi003.stderr:make: *** [Makefile:51: default] Error 2 2024-03-21T08:38:03.771 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-21T08:38:03.771 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-03-21T08:38:03.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T08:38:04.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:03 smithi084 ceph-mon[35477]: pgmap v189: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-03-21T08:38:04.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:03 smithi171 ceph-mon[35388]: pgmap v189: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-03-21T08:38:04.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:03 smithi003 ceph-mon[31675]: pgmap v189: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-03-21T08:38:04.897 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi003 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-03-21T08:38:05.174 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0877337574d84672afd3a07a001c742d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi003 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-03-21T08:38:05.178 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-21T08:38:05.189 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:38:05.216 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-21T08:38:05.227 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-21T08:38:05.227 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T08:38:05.227 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:38:05.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-21T08:38:05.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-21T08:38:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:05 smithi084 ceph-mon[35477]: pgmap v190: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 123 KiB/s wr, 44 op/s 2024-03-21T08:38:06.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:05 smithi171 ceph-mon[35388]: pgmap v190: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 123 KiB/s wr, 44 op/s 2024-03-21T08:38:06.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:05 smithi003 ceph-mon[31675]: pgmap v190: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 123 KiB/s wr, 44 op/s 2024-03-21T08:38:08.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:07 smithi084 ceph-mon[35477]: pgmap v191: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 167 KiB/s wr, 62 op/s 2024-03-21T08:38:08.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:07 smithi084 ceph-mon[35477]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T08:38:08.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:07 smithi171 ceph-mon[35388]: pgmap v191: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 167 KiB/s wr, 62 op/s 2024-03-21T08:38:08.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:07 smithi171 ceph-mon[35388]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T08:38:08.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:07 smithi003 ceph-mon[31675]: pgmap v191: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 167 KiB/s wr, 62 op/s 2024-03-21T08:38:08.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:07 smithi003 ceph-mon[31675]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T08:38:10.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:09 smithi084 ceph-mon[35477]: pgmap v193: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-03-21T08:38:10.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:09 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:38:10.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:09 smithi171 ceph-mon[35388]: pgmap v193: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-03-21T08:38:10.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:09 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:38:10.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:09 smithi003 ceph-mon[31675]: pgmap v193: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-03-21T08:38:10.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:09 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:38:11.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:10 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:38:11.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:10 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:38:11.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:10 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:38:11.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:10 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:38:11.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:10 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:38:11.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:10 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:38:11.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:10 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:38:11.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:10 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:38:11.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:10 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:38:12.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:11 smithi084 ceph-mon[35477]: pgmap v194: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-03-21T08:38:12.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:11 smithi171 ceph-mon[35388]: pgmap v194: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-03-21T08:38:12.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:11 smithi003 ceph-mon[31675]: pgmap v194: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-03-21T08:38:14.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:13 smithi084 ceph-mon[35477]: pgmap v195: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-03-21T08:38:14.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:13 smithi171 ceph-mon[35388]: pgmap v195: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-03-21T08:38:14.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:13 smithi003 ceph-mon[31675]: pgmap v195: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-03-21T08:38:16.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:15 smithi171 ceph-mon[35388]: pgmap v196: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-03-21T08:38:16.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:15 smithi003 ceph-mon[31675]: pgmap v196: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-03-21T08:38:16.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:15 smithi084 ceph-mon[35477]: pgmap v196: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-03-21T08:38:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:18 smithi084 ceph-mon[35477]: pgmap v197: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:18 smithi084 ceph-mon[35477]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T08:38:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:18 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:38:18.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:18 smithi171 ceph-mon[35388]: pgmap v197: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:18.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:18 smithi171 ceph-mon[35388]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T08:38:18.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:18 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:38:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:18 smithi003 ceph-mon[31675]: pgmap v197: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:18 smithi003 ceph-mon[31675]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T08:38:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:18 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:38:20.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:20 smithi084 ceph-mon[35477]: pgmap v199: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:20.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:20 smithi171 ceph-mon[35388]: pgmap v199: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:20.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:20 smithi003 ceph-mon[31675]: pgmap v199: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:22.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:22 smithi084 ceph-mon[35477]: pgmap v200: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:22.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:22 smithi171 ceph-mon[35388]: pgmap v200: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:22.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:22 smithi003 ceph-mon[31675]: pgmap v200: 129 pgs: 129 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-21T08:38:23.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:23 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:38:23.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:23 smithi084 ceph-mon[35477]: pgmap v201: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 28 op/s 2024-03-21T08:38:23.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:23 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:38:23.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:23 smithi171 ceph-mon[35388]: pgmap v201: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 28 op/s 2024-03-21T08:38:23.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:23 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:38:23.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:23 smithi003 ceph-mon[31675]: pgmap v201: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 28 op/s 2024-03-21T08:38:26.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:25 smithi084 ceph-mon[35477]: pgmap v202: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 28 op/s 2024-03-21T08:38:26.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:25 smithi171 ceph-mon[35388]: pgmap v202: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 28 op/s 2024-03-21T08:38:26.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:25 smithi003 ceph-mon[31675]: pgmap v202: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 28 op/s 2024-03-21T08:38:28.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:27 smithi084 ceph-mon[35477]: pgmap v203: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 270 KiB/s wr, 27 op/s 2024-03-21T08:38:28.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:27 smithi171 ceph-mon[35388]: pgmap v203: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 270 KiB/s wr, 27 op/s 2024-03-21T08:38:28.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:27 smithi003 ceph-mon[31675]: pgmap v203: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 270 KiB/s wr, 27 op/s 2024-03-21T08:38:30.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:29 smithi084 ceph-mon[35477]: pgmap v204: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 240 KiB/s wr, 24 op/s 2024-03-21T08:38:30.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:29 smithi171 ceph-mon[35388]: pgmap v204: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 240 KiB/s wr, 24 op/s 2024-03-21T08:38:30.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:29 smithi003 ceph-mon[31675]: pgmap v204: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 240 KiB/s wr, 24 op/s 2024-03-21T08:38:32.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:31 smithi084 ceph-mon[35477]: pgmap v205: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 225 KiB/s wr, 23 op/s 2024-03-21T08:38:32.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:31 smithi171 ceph-mon[35388]: pgmap v205: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 225 KiB/s wr, 23 op/s 2024-03-21T08:38:32.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:31 smithi003 ceph-mon[31675]: pgmap v205: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 225 KiB/s wr, 23 op/s 2024-03-21T08:38:34.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:33 smithi084 ceph-mon[35477]: pgmap v206: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 225 KiB/s wr, 25 op/s 2024-03-21T08:38:34.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:33 smithi171 ceph-mon[35388]: pgmap v206: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 225 KiB/s wr, 25 op/s 2024-03-21T08:38:34.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:33 smithi003 ceph-mon[31675]: pgmap v206: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 225 KiB/s wr, 25 op/s 2024-03-21T08:38:36.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:35 smithi084 ceph-mon[35477]: pgmap v207: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:38:36.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:35 smithi171 ceph-mon[35388]: pgmap v207: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:38:36.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:35 smithi003 ceph-mon[31675]: pgmap v207: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-21T08:38:38.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:37 smithi171 ceph-mon[35388]: pgmap v208: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:38:38.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:37 smithi003 ceph-mon[31675]: pgmap v208: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:38:38.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:37 smithi084 ceph-mon[35477]: pgmap v208: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-21T08:38:40.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:39 smithi003 ceph-mon[31675]: pgmap v209: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:40.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:39 smithi084 ceph-mon[35477]: pgmap v209: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:40.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:39 smithi171 ceph-mon[35388]: pgmap v209: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:42.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:41 smithi003 ceph-mon[31675]: pgmap v210: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:42.178 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:41 smithi084 ceph-mon[35477]: pgmap v210: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:42.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:41 smithi171 ceph-mon[35388]: pgmap v210: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:44.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:43 smithi003 ceph-mon[31675]: pgmap v211: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:44.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:43 smithi084 ceph-mon[35477]: pgmap v211: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:44.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:43 smithi171 ceph-mon[35388]: pgmap v211: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:46.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:45 smithi003 ceph-mon[31675]: pgmap v212: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:46.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:45 smithi084 ceph-mon[35477]: pgmap v212: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:46.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:45 smithi171 ceph-mon[35388]: pgmap v212: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:48.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:47 smithi003 ceph-mon[31675]: pgmap v213: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:48.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:47 smithi084 ceph-mon[35477]: pgmap v213: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:48.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:47 smithi171 ceph-mon[35388]: pgmap v213: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:50.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:49 smithi003 ceph-mon[31675]: pgmap v214: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:50.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:49 smithi084 ceph-mon[35477]: pgmap v214: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:50.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:49 smithi171 ceph-mon[35388]: pgmap v214: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:52.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:51 smithi003 ceph-mon[31675]: pgmap v215: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:52.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:51 smithi084 ceph-mon[35477]: pgmap v215: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:51 smithi171 ceph-mon[35388]: pgmap v215: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:54.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:53 smithi084 ceph-mon[35477]: pgmap v216: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:54.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:53 smithi171 ceph-mon[35388]: pgmap v216: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:54.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:53 smithi003 ceph-mon[31675]: pgmap v216: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:56.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:55 smithi084 ceph-mon[35477]: pgmap v217: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:56.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:55 smithi171 ceph-mon[35388]: pgmap v217: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:56.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:55 smithi003 ceph-mon[31675]: pgmap v217: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:38:58.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:57 smithi084 ceph-mon[35477]: pgmap v218: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:58.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:57 smithi171 ceph-mon[35388]: pgmap v218: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:38:58.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:57 smithi003 ceph-mon[31675]: pgmap v218: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:00.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:38:59 smithi084 ceph-mon[35477]: pgmap v219: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:00.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:38:59 smithi171 ceph-mon[35388]: pgmap v219: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:00.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:38:59 smithi003 ceph-mon[31675]: pgmap v219: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:01.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:00 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:39:01.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:00 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:39:01.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:00 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:39:02.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:01 smithi084 ceph-mon[35477]: pgmap v220: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:02.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:01 smithi171 ceph-mon[35388]: pgmap v220: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:02.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:01 smithi003 ceph-mon[31675]: pgmap v220: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:04.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:03 smithi084 ceph-mon[35477]: pgmap v221: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:39:04.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:03 smithi171 ceph-mon[35388]: pgmap v221: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:39:04.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:03 smithi003 ceph-mon[31675]: pgmap v221: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:39:06.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:05 smithi084 ceph-mon[35477]: pgmap v222: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:39:06.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:05 smithi171 ceph-mon[35388]: pgmap v222: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:39:06.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:05 smithi003 ceph-mon[31675]: pgmap v222: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:39:08.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:08 smithi084 ceph-mon[35477]: pgmap v223: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:39:08.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:08 smithi084 ceph-mon[35477]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T08:39:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:08 smithi171 ceph-mon[35388]: pgmap v223: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:39:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:08 smithi171 ceph-mon[35388]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T08:39:08.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:08 smithi003 ceph-mon[31675]: pgmap v223: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:39:08.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:08 smithi003 ceph-mon[31675]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T08:39:10.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:10 smithi084 ceph-mon[35477]: pgmap v225: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:39:10.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:10 smithi171 ceph-mon[35388]: pgmap v225: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:39:10.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:10 smithi003 ceph-mon[31675]: pgmap v225: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:39:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:12 smithi084 ceph-mon[35477]: pgmap v226: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:39:12.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:12 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:39:12.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:12 smithi171 ceph-mon[35388]: pgmap v226: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:39:12.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:12 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:39:12.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:12 smithi003 ceph-mon[31675]: pgmap v226: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:39:12.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:12 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:39:13.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:13 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:39:13.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:13 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:39:13.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:13 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:39:13.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:13 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:39:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:13 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:39:13.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:13 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:39:13.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:13 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:39:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:13 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:39:13.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:13 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:39:14.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:14 smithi084 ceph-mon[35477]: pgmap v227: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:39:14.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:14 smithi171 ceph-mon[35388]: pgmap v227: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:39:14.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:14 smithi003 ceph-mon[31675]: pgmap v227: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:39:16.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:16 smithi084 ceph-mon[35477]: pgmap v228: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:39:16.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:16 smithi171 ceph-mon[35388]: pgmap v228: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:39:16.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:16 smithi003 ceph-mon[31675]: pgmap v228: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:39:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:18 smithi084 ceph-mon[35477]: pgmap v229: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:18.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:18 smithi084 ceph-mon[35477]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T08:39:18.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:18 smithi171 ceph-mon[35388]: pgmap v229: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:18.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:18 smithi171 ceph-mon[35388]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T08:39:18.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:18 smithi003 ceph-mon[31675]: pgmap v229: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:18.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:18 smithi003 ceph-mon[31675]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T08:39:19.879 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:19 smithi084 ceph-mon[35477]: pgmap v231: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:19.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:19 smithi171 ceph-mon[35388]: pgmap v231: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:19 smithi003 ceph-mon[31675]: pgmap v231: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:22.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:21 smithi084 ceph-mon[35477]: pgmap v232: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:22.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:21 smithi171 ceph-mon[35388]: pgmap v232: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:22.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:21 smithi003 ceph-mon[31675]: pgmap v232: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:39:24.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:23 smithi171 ceph-mon[35388]: pgmap v233: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:24.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:23 smithi003 ceph-mon[31675]: pgmap v233: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:24.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:23 smithi084 ceph-mon[35477]: pgmap v233: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:26.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:25 smithi003 ceph-mon[31675]: pgmap v234: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:25 smithi084 ceph-mon[35477]: pgmap v234: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:26.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:25 smithi171 ceph-mon[35388]: pgmap v234: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:28.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:27 smithi003 ceph-mon[31675]: pgmap v235: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:28.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:27 smithi084 ceph-mon[35477]: pgmap v235: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:28.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:27 smithi171 ceph-mon[35388]: pgmap v235: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:39:30.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:29 smithi003 ceph-mon[31675]: pgmap v236: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:39:30.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:29 smithi171 ceph-mon[35388]: pgmap v236: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:39:30.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:29 smithi084 ceph-mon[35477]: pgmap v236: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:39:32.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:31 smithi003 ceph-mon[31675]: pgmap v237: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:32.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:31 smithi084 ceph-mon[35477]: pgmap v237: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:32.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:31 smithi171 ceph-mon[35388]: pgmap v237: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:34.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:33 smithi003 ceph-mon[31675]: pgmap v238: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:34.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:33 smithi084 ceph-mon[35477]: pgmap v238: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:34.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:33 smithi171 ceph-mon[35388]: pgmap v238: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:36.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:35 smithi003 ceph-mon[31675]: pgmap v239: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:36.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:35 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:39:36.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:35 smithi084 ceph-mon[35477]: pgmap v239: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:36.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:35 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:39:36.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:35 smithi171 ceph-mon[35388]: pgmap v239: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:36.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:35 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:39:38.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:37 smithi084 ceph-mon[35477]: pgmap v240: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:38.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:37 smithi171 ceph-mon[35388]: pgmap v240: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:38.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:37 smithi003 ceph-mon[31675]: pgmap v240: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:40.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:39 smithi084 ceph-mon[35477]: pgmap v241: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:40.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:39 smithi171 ceph-mon[35388]: pgmap v241: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:40.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:39 smithi003 ceph-mon[31675]: pgmap v241: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:42.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:41 smithi084 ceph-mon[35477]: pgmap v242: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:42.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:41 smithi171 ceph-mon[35388]: pgmap v242: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:42.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:41 smithi003 ceph-mon[31675]: pgmap v242: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:44.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:43 smithi084 ceph-mon[35477]: pgmap v243: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:44.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:43 smithi171 ceph-mon[35388]: pgmap v243: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:44.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:43 smithi003 ceph-mon[31675]: pgmap v243: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:46.229 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:45 smithi003 ceph-mon[31675]: pgmap v244: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:46.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:45 smithi084 ceph-mon[35477]: pgmap v244: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:46.385 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:45 smithi171 ceph-mon[35388]: pgmap v244: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:48.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:47 smithi084 ceph-mon[35477]: pgmap v245: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:48.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:47 smithi171 ceph-mon[35388]: pgmap v245: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:48.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:47 smithi003 ceph-mon[31675]: pgmap v245: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:50.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:49 smithi084 ceph-mon[35477]: pgmap v246: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:50.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:49 smithi171 ceph-mon[35388]: pgmap v246: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:50.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:49 smithi003 ceph-mon[31675]: pgmap v246: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:52.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:51 smithi084 ceph-mon[35477]: pgmap v247: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:52.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:51 smithi171 ceph-mon[35388]: pgmap v247: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:52.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:51 smithi003 ceph-mon[31675]: pgmap v247: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:53.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:53 smithi084 ceph-mon[35477]: pgmap v248: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:53.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:53 smithi171 ceph-mon[35388]: pgmap v248: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:53.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:53 smithi003 ceph-mon[31675]: pgmap v248: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:56.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:55 smithi003 ceph-mon[31675]: pgmap v249: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:56.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:55 smithi084 ceph-mon[35477]: pgmap v249: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:56.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:55 smithi171 ceph-mon[35388]: pgmap v249: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:39:58.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:57 smithi003 ceph-mon[31675]: pgmap v250: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:58.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:57 smithi084 ceph-mon[35477]: pgmap v250: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:39:58.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:57 smithi171 ceph-mon[35388]: pgmap v250: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:00.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:39:59 smithi003 ceph-mon[31675]: pgmap v251: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:00.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31671]: 2024-03-21T08:39:59.999+0000 7efc8f000700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-21T08:40:00.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31671]: 2024-03-21T08:39:59.999+0000 7efc8f000700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-21T08:40:00.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31671]: 2024-03-21T08:39:59.999+0000 7efc8f000700 -1 log_channel(cluster) log [ERR] : mds.k(mds.2): Metadata damage detected 2024-03-21T08:40:00.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:39:59 smithi171 ceph-mon[35388]: pgmap v251: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:00.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:39:59 smithi084 ceph-mon[35477]: pgmap v251: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:01.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:00 smithi003 ceph-mon[31675]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-21T08:40:01.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:00 smithi003 ceph-mon[31675]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-21T08:40:01.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:00 smithi003 ceph-mon[31675]: mds.k(mds.2): Metadata damage detected 2024-03-21T08:40:01.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:00 smithi084 ceph-mon[35477]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-21T08:40:01.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:00 smithi084 ceph-mon[35477]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-21T08:40:01.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:00 smithi084 ceph-mon[35477]: mds.k(mds.2): Metadata damage detected 2024-03-21T08:40:01.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:00 smithi171 ceph-mon[35388]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-21T08:40:01.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:00 smithi171 ceph-mon[35388]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-21T08:40:01.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:00 smithi171 ceph-mon[35388]: mds.k(mds.2): Metadata damage detected 2024-03-21T08:40:02.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:01 smithi003 ceph-mon[31675]: pgmap v252: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:02.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:01 smithi084 ceph-mon[35477]: pgmap v252: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:02.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:01 smithi171 ceph-mon[35388]: pgmap v252: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:03.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:02 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:03.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:02 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:03.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:02 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:04.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:03 smithi084 ceph-mon[35477]: pgmap v253: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:40:04.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:03 smithi171 ceph-mon[35388]: pgmap v253: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:40:04.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:03 smithi003 ceph-mon[31675]: pgmap v253: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:40:06.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:05 smithi084 ceph-mon[35477]: pgmap v254: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:40:06.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:05 smithi171 ceph-mon[35388]: pgmap v254: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:40:06.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:05 smithi003 ceph-mon[31675]: pgmap v254: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:40:08.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:07 smithi084 ceph-mon[35477]: pgmap v255: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:40:08.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:07 smithi084 ceph-mon[35477]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T08:40:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:07 smithi171 ceph-mon[35388]: pgmap v255: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:40:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:07 smithi171 ceph-mon[35388]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T08:40:08.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:07 smithi003 ceph-mon[31675]: pgmap v255: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:40:08.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:07 smithi003 ceph-mon[31675]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T08:40:10.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:09 smithi084 ceph-mon[35477]: pgmap v257: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:40:10.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:09 smithi171 ceph-mon[35388]: pgmap v257: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:40:10.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:09 smithi003 ceph-mon[31675]: pgmap v257: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:40:12.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:11 smithi084 ceph-mon[35477]: pgmap v258: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:40:12.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:11 smithi171 ceph-mon[35388]: pgmap v258: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:40:12.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:11 smithi003 ceph-mon[31675]: pgmap v258: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:40:13.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:12 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:40:13.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:12 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:40:13.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:12 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:40:14.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:13 smithi084 ceph-mon[35477]: pgmap v259: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:40:14.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:13 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:14.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:13 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:13 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.127 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:13 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.128 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:13 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:13 smithi003 ceph-mon[31675]: pgmap v259: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:40:14.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:13 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:14.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:13 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:13 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:13 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:13 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:13 smithi171 ceph-mon[35388]: pgmap v259: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:40:14.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:13 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:14.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:13 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:13 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:13 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:14.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:13 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:16.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:15 smithi084 ceph-mon[35477]: pgmap v260: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:40:16.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:15 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:40:16.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:15 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:40:16.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:15 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:16.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:15 smithi171 ceph-mon[35388]: pgmap v260: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:40:16.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:15 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:40:16.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:15 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:40:16.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:15 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:16.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:15 smithi003 ceph-mon[31675]: pgmap v260: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:40:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:15 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:40:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:15 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:40:16.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:15 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:40:18.377 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:18 smithi084 ceph-mon[35477]: pgmap v261: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:18.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:18 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:18.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:18 smithi084 ceph-mon[35477]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T08:40:18.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:18 smithi171 ceph-mon[35388]: pgmap v261: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:18.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:18 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:18.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:18 smithi171 ceph-mon[35388]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T08:40:18.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:18 smithi003 ceph-mon[31675]: pgmap v261: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:18.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:18 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:40:18.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:18 smithi003 ceph-mon[31675]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T08:40:20.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:20 smithi084 ceph-mon[35477]: pgmap v263: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:20.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:20 smithi171 ceph-mon[35388]: pgmap v263: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:20.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:20 smithi003 ceph-mon[31675]: pgmap v263: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:22.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:22 smithi084 ceph-mon[35477]: pgmap v264: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:22.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:22 smithi171 ceph-mon[35388]: pgmap v264: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:22.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:22 smithi003 ceph-mon[31675]: pgmap v264: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:40:24.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:24 smithi084 ceph-mon[35477]: pgmap v265: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:24.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:24 smithi171 ceph-mon[35388]: pgmap v265: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:24.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:24 smithi003 ceph-mon[31675]: pgmap v265: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:26.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:26 smithi084 ceph-mon[35477]: pgmap v266: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:26.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:26 smithi171 ceph-mon[35388]: pgmap v266: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:26.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:26 smithi003 ceph-mon[31675]: pgmap v266: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:28.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:28 smithi084 ceph-mon[35477]: pgmap v267: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:28.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:28 smithi171 ceph-mon[35388]: pgmap v267: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:28.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:28 smithi003 ceph-mon[31675]: pgmap v267: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:40:30.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:30 smithi084 ceph-mon[35477]: pgmap v268: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:40:30.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:30 smithi171 ceph-mon[35388]: pgmap v268: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:40:30.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:30 smithi003 ceph-mon[31675]: pgmap v268: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:40:32.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:32 smithi084 ceph-mon[35477]: pgmap v269: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:32.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:32 smithi171 ceph-mon[35388]: pgmap v269: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:32.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:32 smithi003 ceph-mon[31675]: pgmap v269: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:34.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:34 smithi084 ceph-mon[35477]: pgmap v270: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:34.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:34 smithi171 ceph-mon[35388]: pgmap v270: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:34.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:34 smithi003 ceph-mon[31675]: pgmap v270: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:36.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:36 smithi084 ceph-mon[35477]: pgmap v271: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:36.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:36 smithi171 ceph-mon[35388]: pgmap v271: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:36.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:36 smithi003 ceph-mon[31675]: pgmap v271: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:38.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:38 smithi084 ceph-mon[35477]: pgmap v272: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:38.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:38 smithi171 ceph-mon[35388]: pgmap v272: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:38.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:38 smithi003 ceph-mon[31675]: pgmap v272: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:40.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:40 smithi084 ceph-mon[35477]: pgmap v273: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:40.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:40 smithi171 ceph-mon[35388]: pgmap v273: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:40.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:40 smithi003 ceph-mon[31675]: pgmap v273: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:42.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:42 smithi084 ceph-mon[35477]: pgmap v274: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:42.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:42 smithi171 ceph-mon[35388]: pgmap v274: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:42.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:42 smithi003 ceph-mon[31675]: pgmap v274: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:44.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:44 smithi084 ceph-mon[35477]: pgmap v275: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:44.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:44 smithi171 ceph-mon[35388]: pgmap v275: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:44.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:44 smithi003 ceph-mon[31675]: pgmap v275: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:46.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:46 smithi084 ceph-mon[35477]: pgmap v276: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:46.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:46 smithi171 ceph-mon[35388]: pgmap v276: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:46.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:46 smithi003 ceph-mon[31675]: pgmap v276: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:48.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:48 smithi084 ceph-mon[35477]: pgmap v277: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:48.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:48 smithi171 ceph-mon[35388]: pgmap v277: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:48.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:48 smithi003 ceph-mon[31675]: pgmap v277: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:50.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:50 smithi084 ceph-mon[35477]: pgmap v278: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:50.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:50 smithi171 ceph-mon[35388]: pgmap v278: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:50.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:50 smithi003 ceph-mon[31675]: pgmap v278: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:52.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:52 smithi171 ceph-mon[35388]: pgmap v279: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:52.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:52 smithi003 ceph-mon[31675]: pgmap v279: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:52.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:52 smithi084 ceph-mon[35477]: pgmap v279: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:54.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:54 smithi003 ceph-mon[31675]: pgmap v280: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:54.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:54 smithi084 ceph-mon[35477]: pgmap v280: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:54.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:54 smithi171 ceph-mon[35388]: pgmap v280: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:56.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:56 smithi003 ceph-mon[31675]: pgmap v281: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:56.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:56 smithi084 ceph-mon[35477]: pgmap v281: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:56.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:56 smithi171 ceph-mon[35388]: pgmap v281: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:40:58.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:40:58 smithi003 ceph-mon[31675]: pgmap v282: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:58.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:40:58 smithi084 ceph-mon[35477]: pgmap v282: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:40:58.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:40:58 smithi171 ceph-mon[35388]: pgmap v282: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:00.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:00 smithi003 ceph-mon[31675]: pgmap v283: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:00.437 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:00 smithi171 ceph-mon[35388]: pgmap v283: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:00.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:00 smithi084 ceph-mon[35477]: pgmap v283: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:02.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:02 smithi171 ceph-mon[35388]: pgmap v284: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:02.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:02 smithi003 ceph-mon[31675]: pgmap v284: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:02 smithi084 ceph-mon[35477]: pgmap v284: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:04.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:04 smithi003 ceph-mon[31675]: pgmap v285: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:41:04.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:04 smithi084 ceph-mon[35477]: pgmap v285: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:41:04.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:04 smithi171 ceph-mon[35388]: pgmap v285: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T08:41:06.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:06 smithi003 ceph-mon[31675]: pgmap v286: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:41:06.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:06 smithi084 ceph-mon[35477]: pgmap v286: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:41:06.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:06 smithi171 ceph-mon[35388]: pgmap v286: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-21T08:41:07.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:07 smithi084 ceph-mon[35477]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T08:41:07.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:07 smithi171 ceph-mon[35388]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T08:41:07.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:07 smithi003 ceph-mon[31675]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T08:41:08.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:08 smithi084 ceph-mon[35477]: pgmap v288: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:08.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:08 smithi171 ceph-mon[35388]: pgmap v288: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:08.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:08 smithi003 ceph-mon[31675]: pgmap v288: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:10.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:10 smithi084 ceph-mon[35477]: pgmap v289: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:10.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:10 smithi171 ceph-mon[35388]: pgmap v289: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:10.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:10 smithi003 ceph-mon[31675]: pgmap v289: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:12.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:12 smithi171 ceph-mon[35388]: pgmap v290: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:12.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:12 smithi084 ceph-mon[35477]: pgmap v290: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:12.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:12 smithi003 ceph-mon[31675]: pgmap v290: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-21T08:41:14.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:14 smithi084 ceph-mon[35477]: pgmap v291: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:41:14.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:14 smithi171 ceph-mon[35388]: pgmap v291: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:41:14.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:14 smithi003 ceph-mon[31675]: pgmap v291: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:41:16.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:16 smithi003 ceph-mon[31675]: pgmap v292: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:41:16.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:16 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:41:16.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:16 smithi084 ceph-mon[35477]: pgmap v292: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:41:16.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:16 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:41:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:16 smithi171 ceph-mon[35388]: pgmap v292: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:41:16.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:16 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:41:17.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:17 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:41:17.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:17 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:41:17.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:17 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:41:17.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:17 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:41:17.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:17 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:41:17.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:17 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:41:17.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:17 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:41:17.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:17 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:41:17.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:17 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:41:18.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:18 smithi084 ceph-mon[35477]: pgmap v293: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 191 B/s wr, 8 op/s 2024-03-21T08:41:18.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:18 smithi084 ceph-mon[35477]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T08:41:18.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:18 smithi171 ceph-mon[35388]: pgmap v293: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 191 B/s wr, 8 op/s 2024-03-21T08:41:18.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:18 smithi171 ceph-mon[35388]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T08:41:18.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:18 smithi003 ceph-mon[31675]: pgmap v293: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 191 B/s wr, 8 op/s 2024-03-21T08:41:18.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:18 smithi003 ceph-mon[31675]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T08:41:20.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:20 smithi084 ceph-mon[35477]: pgmap v295: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:41:20.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:20 smithi171 ceph-mon[35388]: pgmap v295: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:41:20.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:20 smithi003 ceph-mon[31675]: pgmap v295: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:41:22.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:22 smithi084 ceph-mon[35477]: pgmap v296: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:41:22.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:22 smithi171 ceph-mon[35388]: pgmap v296: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:41:22.678 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:22 smithi003 ceph-mon[31675]: pgmap v296: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:41:24.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:24 smithi084 ceph-mon[35477]: pgmap v297: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:24.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:24 smithi171 ceph-mon[35388]: pgmap v297: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:24.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:24 smithi003 ceph-mon[31675]: pgmap v297: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:26.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:26 smithi084 ceph-mon[35477]: pgmap v298: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:26.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:26 smithi171 ceph-mon[35388]: pgmap v298: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:26.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:26 smithi003 ceph-mon[31675]: pgmap v298: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:28.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:28 smithi084 ceph-mon[35477]: pgmap v299: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:28.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:28 smithi171 ceph-mon[35388]: pgmap v299: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:28.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:28 smithi003 ceph-mon[31675]: pgmap v299: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:41:30.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:30 smithi084 ceph-mon[35477]: pgmap v300: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:41:30.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:30 smithi171 ceph-mon[35388]: pgmap v300: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:41:30.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:30 smithi003 ceph-mon[31675]: pgmap v300: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:41:32.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:32 smithi084 ceph-mon[35477]: pgmap v301: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:32.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:32 smithi171 ceph-mon[35388]: pgmap v301: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:32.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:32 smithi003 ceph-mon[31675]: pgmap v301: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:34.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:34 smithi084 ceph-mon[35477]: pgmap v302: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:34.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:34 smithi171 ceph-mon[35388]: pgmap v302: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:34.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:34 smithi003 ceph-mon[31675]: pgmap v302: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:36.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:36 smithi084 ceph-mon[35477]: pgmap v303: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:36.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:36 smithi171 ceph-mon[35388]: pgmap v303: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:36.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:36 smithi003 ceph-mon[31675]: pgmap v303: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:38.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:38 smithi084 ceph-mon[35477]: pgmap v304: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:38.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:38 smithi084 ceph-mon[35477]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:41:38.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:38 smithi171 ceph-mon[35388]: pgmap v304: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:38.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:38 smithi171 ceph-mon[35388]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:41:38.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:38 smithi003 ceph-mon[31675]: pgmap v304: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:38.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:38 smithi003 ceph-mon[31675]: fsmap cephfs:3 {0=h=up:active,1=e=up:active,2=k=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:40 smithi084 ceph-mon[35477]: pgmap v305: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:40.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:40 smithi171 ceph-mon[35388]: pgmap v305: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:40.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:40 smithi003 ceph-mon[31675]: pgmap v305: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:42.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:42 smithi084 ceph-mon[35477]: pgmap v306: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:42.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:42 smithi171 ceph-mon[35388]: pgmap v306: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:42.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:42 smithi003 ceph-mon[31675]: pgmap v306: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:44.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:44 smithi084 ceph-mon[35477]: pgmap v307: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:44.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:44 smithi171 ceph-mon[35388]: pgmap v307: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:44.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:44 smithi003 ceph-mon[31675]: pgmap v307: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:46.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:46 smithi084 ceph-mon[35477]: pgmap v308: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:46.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:46 smithi171 ceph-mon[35388]: pgmap v308: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:46.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:46 smithi003 ceph-mon[31675]: pgmap v308: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:48.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:48 smithi084 ceph-mon[35477]: pgmap v309: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:48.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:48 smithi171 ceph-mon[35388]: pgmap v309: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:48.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:48 smithi003 ceph-mon[31675]: pgmap v309: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:50.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:50 smithi084 ceph-mon[35477]: pgmap v310: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:50.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:50 smithi171 ceph-mon[35388]: pgmap v310: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:50.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:50 smithi003 ceph-mon[31675]: pgmap v310: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:52.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:52 smithi084 ceph-mon[35477]: pgmap v311: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:52.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:52 smithi171 ceph-mon[35388]: pgmap v311: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:52.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:52 smithi003 ceph-mon[31675]: pgmap v311: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:53.481 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-21T08:41:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 -- 172.21.15.3:0/3710245692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 msgr2=0x7fa11410e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 --2- 172.21.15.3:0/3710245692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11410e330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009ca0 tx=0x7fa10c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 -- 172.21.15.3:0/3710245692 shutdown_connections 2024-03-21T08:41:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 --2- 172.21.15.3:0/3710245692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11410e330 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 --2- 172.21.15.3:0/3710245692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa114068e90 0x7fa114069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 --2- 172.21.15.3:0/3710245692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1141075b0 0x7fa114107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 -- 172.21.15.3:0/3710245692 >> 172.21.15.3:0/3710245692 conn(0x7fa114075f00 msgr2=0x7fa114076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 -- 172.21.15.3:0/3710245692 shutdown_connections 2024-03-21T08:41:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.711+0000 7fa118e00640 1 -- 172.21.15.3:0/3710245692 wait complete. 2024-03-21T08:41:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 Processor -- start 2024-03-21T08:41:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 -- start start 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa114068e90 0x7fa114199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1141075b0 0x7fa11419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa114111700 con 0x7fa114069850 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa114111580 con 0x7fa114068e90 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.712+0000 7fa118e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa114111880 con 0x7fa1141075b0 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43846/0 (socket says 172.21.15.3:43846) 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 -- 172.21.15.3:0/2560475215 learned_addr learned my addr 172.21.15.3:0/2560475215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa113400640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa114068e90 0x7fa114199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa113e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1141075b0 0x7fa11419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 -- 172.21.15.3:0/2560475215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1141075b0 msgr2=0x7fa11419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1141075b0 0x7fa11419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 -- 172.21.15.3:0/2560475215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa114068e90 msgr2=0x7fa114199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa114068e90 0x7fa114199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa10aa00640 1 -- 172.21.15.3:0/2560475215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11419eda0 con 0x7fa114069850 2024-03-21T08:41:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.713+0000 7fa113e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1141075b0 0x7fa11419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:41:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.714+0000 7fa10aa00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11419a340 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00e9f0 tx=0x7fa0fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.714+0000 7fa110c00640 1 -- 172.21.15.3:0/2560475215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00cd80 con 0x7fa114069850 2024-03-21T08:41:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.714+0000 7fa110c00640 1 -- 172.21.15.3:0/2560475215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0fc0054d0 con 0x7fa114069850 2024-03-21T08:41:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.714+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11419f090 con 0x7fa114069850 2024-03-21T08:41:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.714+0000 7fa110c00640 1 -- 172.21.15.3:0/2560475215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc010490 con 0x7fa114069850 2024-03-21T08:41:53.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.714+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1141a6920 con 0x7fa114069850 2024-03-21T08:41:53.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.715+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa114067f10 con 0x7fa114069850 2024-03-21T08:41:53.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.717+0000 7fa110c00640 1 -- 172.21.15.3:0/2560475215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fa0fc005020 con 0x7fa114069850 2024-03-21T08:41:53.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.718+0000 7fa110c00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa0f80a30f0 0x7fa0f80a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:53.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.719+0000 7fa110c00640 1 -- 172.21.15.3:0/2560475215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7fa0fc0d00d0 con 0x7fa114069850 2024-03-21T08:41:53.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.719+0000 7fa113400640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa0f80a30f0 0x7fa0f80a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:53.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.720+0000 7fa113400640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa0f80a30f0 0x7fa0f80a55b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa104005fd0 tx=0x7fa10400b590 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:53.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.724+0000 7fa110c00640 1 -- 172.21.15.3:0/2560475215 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0fc09c6e0 con 0x7fa114069850 2024-03-21T08:41:53.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:53 smithi084 ceph-mon[35477]: pgmap v312: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:53.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:53 smithi171 ceph-mon[35388]: pgmap v312: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:53.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:53 smithi003 ceph-mon[31675]: pgmap v312: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:41:53.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.951+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 --> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1140ff700 con 0x7fa0f80a30f0 2024-03-21T08:41:53.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.970+0000 7fa110c00640 1 -- 172.21.15.3:0/2560475215 <== mgr.15654 v2:172.21.15.3:6800/1070134681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7fa1140ff700 con 0x7fa0f80a30f0 2024-03-21T08:41:53.973 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-21T00:00:00", "created": "2024-03-21T08:31:51", "first": "2024-03-21T08:32:00", "last": "2024-03-21T08:41:00", "last_pruned": "2024-03-21T08:41:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-21T08:41:53.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa0f80a30f0 msgr2=0x7fa0f80a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa0f80a30f0 0x7fa0f80a55b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa104005fd0 tx=0x7fa10400b590 comp rx=0 tx=0).stop 2024-03-21T08:41:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 msgr2=0x7fa11419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11419a340 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00e9f0 tx=0x7fa0fc00edf0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 shutdown_connections 2024-03-21T08:41:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fa0f80a30f0 0x7fa0f80a55b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1141075b0 0x7fa11419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa114069850 0x7fa11419a340 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 --2- 172.21.15.3:0/2560475215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa114068e90 0x7fa114199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:53.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 >> 172.21.15.3:0/2560475215 conn(0x7fa114075f00 msgr2=0x7fa1140feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:53.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 shutdown_connections 2024-03-21T08:41:53.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:53.973+0000 7fa118e00640 1 -- 172.21.15.3:0/2560475215 wait complete. 2024-03-21T08:41:53.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-21T08:41:54.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 -- 172.21.15.3:0/1694256926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a01096d0 msgr2=0x7ff0a0109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 --2- 172.21.15.3:0/1694256926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a01096d0 0x7ff0a0109ab0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff090009cd0 tx=0x7ff09002c0e0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 -- 172.21.15.3:0/1694256926 shutdown_connections 2024-03-21T08:41:54.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 --2- 172.21.15.3:0/1694256926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a0104080 0x7ff0a0110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 --2- 172.21.15.3:0/1694256926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01036c0 0x7ff0a0103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 --2- 172.21.15.3:0/1694256926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a01096d0 0x7ff0a0109ab0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 -- 172.21.15.3:0/1694256926 >> 172.21.15.3:0/1694256926 conn(0x7ff0a00ff1b0 msgr2=0x7ff0a01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:54.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 -- 172.21.15.3:0/1694256926 shutdown_connections 2024-03-21T08:41:54.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.214+0000 7ff09fe00640 1 -- 172.21.15.3:0/1694256926 wait complete. 2024-03-21T08:41:54.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.215+0000 7ff09fe00640 1 Processor -- start 2024-03-21T08:41:54.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09fe00640 1 -- start start 2024-03-21T08:41:54.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01036c0 0x7ff0a0199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0104080 0x7ff0a019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a01096d0 0x7ff0a019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0113990 con 0x7ff0a0104080 2024-03-21T08:41:54.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0113810 con 0x7ff0a01036c0 2024-03-21T08:41:54.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0113b10 con 0x7ff0a01096d0 2024-03-21T08:41:54.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0104080 0x7ff0a019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0104080 0x7ff0a019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43866/0 (socket says 172.21.15.3:43866) 2024-03-21T08:41:54.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09e000640 1 -- 172.21.15.3:0/2399419433 learned_addr learned my addr 172.21.15.3:0/2399419433 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:54.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.216+0000 7ff09ea00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01036c0 0x7ff0a0199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09f400640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a01096d0 0x7ff0a019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09e000640 1 -- 172.21.15.3:0/2399419433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a01096d0 msgr2=0x7ff0a019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09e000640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a01096d0 0x7ff0a019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09e000640 1 -- 172.21.15.3:0/2399419433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01036c0 msgr2=0x7ff0a0199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09e000640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01036c0 0x7ff0a0199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09e000640 1 -- 172.21.15.3:0/2399419433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a019ef20 con 0x7ff0a0104080 2024-03-21T08:41:54.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09e000640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0104080 0x7ff0a019a470 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9f0 tx=0x7ff08800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:54.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff097400640 1 -- 172.21.15.3:0/2399419433 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800cd80 con 0x7ff0a0104080 2024-03-21T08:41:54.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff097400640 1 -- 172.21.15.3:0/2399419433 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0880108e0 con 0x7ff0a0104080 2024-03-21T08:41:54.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff097400640 1 -- 172.21.15.3:0/2399419433 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088018920 con 0x7ff0a0104080 2024-03-21T08:41:54.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.217+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a019f210 con 0x7ff0a0104080 2024-03-21T08:41:54.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.218+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a01a6aa0 con 0x7ff0a0104080 2024-03-21T08:41:54.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.219+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a0067f10 con 0x7ff0a0104080 2024-03-21T08:41:54.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.221+0000 7ff097400640 1 -- 172.21.15.3:0/2399419433 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7ff088010a90 con 0x7ff0a0104080 2024-03-21T08:41:54.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.222+0000 7ff097400640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff0840a30f0 0x7ff0840a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.222+0000 7ff097400640 1 -- 172.21.15.3:0/2399419433 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7ff0880cff90 con 0x7ff0a0104080 2024-03-21T08:41:54.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.222+0000 7ff09ea00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff0840a30f0 0x7ff0840a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.223+0000 7ff09ea00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff0840a30f0 0x7ff0840a55b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff090009ca0 tx=0x7ff090005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:54.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.227+0000 7ff097400640 1 -- 172.21.15.3:0/2399419433 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff08809c5a0 con 0x7ff0a0104080 2024-03-21T08:41:54.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.453+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 --> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff0a0061ae0 con 0x7ff0840a30f0 2024-03-21T08:41:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.471+0000 7ff097400640 1 -- 172.21.15.3:0/2399419433 <== mgr.15654 v2:172.21.15.3:6800/1070134681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ff0a0061ae0 con 0x7ff0840a30f0 2024-03-21T08:41:54.473 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-03-21T08:41:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff0840a30f0 msgr2=0x7ff0840a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff0840a30f0 0x7ff0840a55b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff090009ca0 tx=0x7ff090005f90 comp rx=0 tx=0).stop 2024-03-21T08:41:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0104080 msgr2=0x7ff0a019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0104080 0x7ff0a019a470 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9f0 tx=0x7ff08800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 shutdown_connections 2024-03-21T08:41:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff0840a30f0 0x7ff0840a55b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a01096d0 0x7ff0a019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0a0104080 0x7ff0a019a470 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 --2- 172.21.15.3:0/2399419433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01036c0 0x7ff0a0199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.474+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 >> 172.21.15.3:0/2399419433 conn(0x7ff0a00ff1b0 msgr2=0x7ff0a00ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.475+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 shutdown_connections 2024-03-21T08:41:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.475+0000 7ff09fe00640 1 -- 172.21.15.3:0/2399419433 wait complete. 2024-03-21T08:41:54.497 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-21T08:41:54.514 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-21T08:41:54.514 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-21T08:41:54.524 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-21T08:41:54.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:41:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 -- 172.21.15.3:0/3326788862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6950108800 msgr2=0x7f695010c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 --2- 172.21.15.3:0/3326788862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6950108800 0x7f695010c7c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6938009900 tx=0x7f693802bc40 comp rx=0 tx=0).stop 2024-03-21T08:41:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 -- 172.21.15.3:0/3326788862 shutdown_connections 2024-03-21T08:41:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 --2- 172.21.15.3:0/3326788862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f695010cea0 0x7f6950114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 --2- 172.21.15.3:0/3326788862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6950108800 0x7f695010c7c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 --2- 172.21.15.3:0/3326788862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6950107e50 0x7f6950108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 -- 172.21.15.3:0/3326788862 >> 172.21.15.3:0/3326788862 conn(0x7f6950077470 msgr2=0x7f69501064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:54.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 -- 172.21.15.3:0/3326788862 shutdown_connections 2024-03-21T08:41:54.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.761+0000 7f694f200640 1 -- 172.21.15.3:0/3326788862 wait complete. 2024-03-21T08:41:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.762+0000 7f694f200640 1 Processor -- start 2024-03-21T08:41:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.762+0000 7f694f200640 1 -- start start 2024-03-21T08:41:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6950107e50 0x7f695019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f695010cea0 0x7f695019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69501a2b70 0x7f69501a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6950117e10 con 0x7f69501a2b70 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6950117c90 con 0x7f6950107e50 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6950117f90 con 0x7f695010cea0 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69501a2b70 0x7f69501a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69501a2b70 0x7f69501a3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43876/0 (socket says 172.21.15.3:43876) 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694e800640 1 -- 172.21.15.3:0/4131622494 learned_addr learned my addr 172.21.15.3:0/4131622494 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694de00640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6950107e50 0x7f695019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.763+0000 7f694d400640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f695010cea0 0x7f695019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694e800640 1 -- 172.21.15.3:0/4131622494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f695010cea0 msgr2=0x7f695019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694e800640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f695010cea0 0x7f695019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694e800640 1 -- 172.21.15.3:0/4131622494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6950107e50 msgr2=0x7f695019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694e800640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6950107e50 0x7f695019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694e800640 1 -- 172.21.15.3:0/4131622494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69501a3660 con 0x7f69501a2b70 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694de00640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6950107e50 0x7f695019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694e800640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69501a2b70 0x7f69501a3020 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f694800eaa0 tx=0x7f694800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f6946a00640 1 -- 172.21.15.3:0/4131622494 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694800cdb0 con 0x7f69501a2b70 2024-03-21T08:41:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f6946a00640 1 -- 172.21.15.3:0/4131622494 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6948005020 con 0x7f69501a2b70 2024-03-21T08:41:54.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f6946a00640 1 -- 172.21.15.3:0/4131622494 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69480116d0 con 0x7f69501a2b70 2024-03-21T08:41:54.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69501a3950 con 0x7f69501a2b70 2024-03-21T08:41:54.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.764+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69501ab0c0 con 0x7f69501a2b70 2024-03-21T08:41:54.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.766+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695004f470 con 0x7f69501a2b70 2024-03-21T08:41:54.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.768+0000 7f6946a00640 1 -- 172.21.15.3:0/4131622494 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f69480051d0 con 0x7f69501a2b70 2024-03-21T08:41:54.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.769+0000 7f6946a00640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f69340a3100 0x7f69340a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:54.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.769+0000 7f6946a00640 1 -- 172.21.15.3:0/4131622494 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f69480d0210 con 0x7f69501a2b70 2024-03-21T08:41:54.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.769+0000 7f694de00640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f69340a3100 0x7f69340a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:54.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.770+0000 7f694de00640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f69340a3100 0x7f69340a55c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f694000abe0 tx=0x7f694000a5c0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:54.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.772+0000 7f6946a00640 1 -- 172.21.15.3:0/4131622494 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6948016030 con 0x7f69501a2b70 2024-03-21T08:41:54.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:54 smithi084 ceph-mon[35477]: from='client.16749 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:41:54.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:54 smithi084 ceph-mon[35477]: from='client.16755 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:41:54.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:54 smithi171 ceph-mon[35388]: from='client.16749 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:41:54.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:54 smithi171 ceph-mon[35388]: from='client.16755 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:41:54.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:54 smithi003 ceph-mon[31675]: from='client.16749 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:41:54.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:54 smithi003 ceph-mon[31675]: from='client.16755 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:41:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.981+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f695019f600 con 0x7f69501a2b70 2024-03-21T08:41:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.983+0000 7f6946a00640 1 -- 172.21.15.3:0/4131622494 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25436 (secure 0 0 0) 0x7f695019f600 con 0x7f69501a2b70 2024-03-21T08:41:54.985 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:41:54.986 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":105,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:41:17.585844+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3880262252":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/62217793":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/1599875703":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/1520535032":"2024-03-22T08:31:50.557523+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1115678781":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6801/1520535032":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:41:54.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f69340a3100 msgr2=0x7f69340a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f69340a3100 0x7f69340a55c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f694000abe0 tx=0x7f694000a5c0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69501a2b70 msgr2=0x7f69501a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69501a2b70 0x7f69501a3020 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f694800eaa0 tx=0x7f694800ef60 comp rx=0 tx=0).stop 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 shutdown_connections 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69501a2b70 0x7f69501a3020 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f69340a3100 0x7f69340a55c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f695010cea0 0x7f695019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 --2- 172.21.15.3:0/4131622494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6950107e50 0x7f695019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 >> 172.21.15.3:0/4131622494 conn(0x7f6950077470 msgr2=0x7f69500790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 shutdown_connections 2024-03-21T08:41:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:54.986+0000 7f694f200640 1 -- 172.21.15.3:0/4131622494 wait complete. 2024-03-21T08:41:55.009 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:41:55.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:41:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.224+0000 7f770d200640 1 -- 172.21.15.3:0/1344391982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 msgr2=0x7f7708103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.224+0000 7f770d200640 1 --2- 172.21.15.3:0/1344391982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708103ae0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009cd0 tx=0x7f76f002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.225+0000 7f770d200640 1 -- 172.21.15.3:0/1344391982 shutdown_connections 2024-03-21T08:41:55.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.225+0000 7f770d200640 1 --2- 172.21.15.3:0/1344391982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7708104020 0x7f7708110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.225+0000 7f770d200640 1 --2- 172.21.15.3:0/1344391982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708103ae0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.225+0000 7f770d200640 1 --2- 172.21.15.3:0/1344391982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7708109670 0x7f7708109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.225+0000 7f770d200640 1 -- 172.21.15.3:0/1344391982 >> 172.21.15.3:0/1344391982 conn(0x7f77080ff150 msgr2=0x7f7708101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:55.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.225+0000 7f770d200640 1 -- 172.21.15.3:0/1344391982 shutdown_connections 2024-03-21T08:41:55.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.225+0000 7f770d200640 1 -- 172.21.15.3:0/1344391982 wait complete. 2024-03-21T08:41:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.226+0000 7f770d200640 1 Processor -- start 2024-03-21T08:41:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.226+0000 7f770d200640 1 -- start start 2024-03-21T08:41:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f770d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f770d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7708104020 0x7f770819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f770d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7708109670 0x7f770819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f770d200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77081138f0 con 0x7f7708103660 2024-03-21T08:41:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f770d200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7708113770 con 0x7f7708104020 2024-03-21T08:41:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f770d200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7708113a70 con 0x7f7708109670 2024-03-21T08:41:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43892/0 (socket says 172.21.15.3:43892) 2024-03-21T08:41:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 -- 172.21.15.3:0/294322948 learned_addr learned my addr 172.21.15.3:0/294322948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7706a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7708104020 0x7f770819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707e00640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7708109670 0x7f770819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 -- 172.21.15.3:0/294322948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7708109670 msgr2=0x7f770819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7708109670 0x7f770819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 -- 172.21.15.3:0/294322948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7708104020 msgr2=0x7f770819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7708104020 0x7f770819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707400640 1 -- 172.21.15.3:0/294322948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770819ee90 con 0x7f7708103660 2024-03-21T08:41:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.227+0000 7f7707e00640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7708109670 0x7f770819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:41:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.228+0000 7f7707400640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708199ea0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f76f800e9f0 tx=0x7f76f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.228+0000 7f76ffe00640 1 -- 172.21.15.3:0/294322948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800cd80 con 0x7f7708103660 2024-03-21T08:41:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.228+0000 7f76ffe00640 1 -- 172.21.15.3:0/294322948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76f80108e0 con 0x7f7708103660 2024-03-21T08:41:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.228+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770819f180 con 0x7f7708103660 2024-03-21T08:41:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.228+0000 7f76ffe00640 1 -- 172.21.15.3:0/294322948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8018920 con 0x7f7708103660 2024-03-21T08:41:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.228+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77081a6a10 con 0x7f7708103660 2024-03-21T08:41:55.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.230+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770804f400 con 0x7f7708103660 2024-03-21T08:41:55.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.231+0000 7f76ffe00640 1 -- 172.21.15.3:0/294322948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f76f8010a90 con 0x7f7708103660 2024-03-21T08:41:55.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.233+0000 7f76ffe00640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f76ec0a3090 0x7f76ec0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.233+0000 7f76ffe00640 1 -- 172.21.15.3:0/294322948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f76f80cffc0 con 0x7f7708103660 2024-03-21T08:41:55.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.233+0000 7f7706a00640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f76ec0a3090 0x7f76ec0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.234+0000 7f7706a00640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f76ec0a3090 0x7f76ec0a5550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009ca0 tx=0x7f76f0005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:55.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.238+0000 7f76ffe00640 1 -- 172.21.15.3:0/294322948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f76f809c5d0 con 0x7f7708103660 2024-03-21T08:41:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.442+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f770802d070 con 0x7f7708103660 2024-03-21T08:41:55.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.443+0000 7f76ffe00640 1 -- 172.21.15.3:0/294322948 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76f80bc0c0 con 0x7f7708103660 2024-03-21T08:41:55.445 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:41:55.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f76ec0a3090 msgr2=0x7f76ec0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f76ec0a3090 0x7f76ec0a5550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009ca0 tx=0x7f76f0005f90 comp rx=0 tx=0).stop 2024-03-21T08:41:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 msgr2=0x7f7708199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708199ea0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f76f800e9f0 tx=0x7f76f800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 shutdown_connections 2024-03-21T08:41:55.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f76ec0a3090 0x7f76ec0a5550 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7708109670 0x7f770819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7708104020 0x7f770819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 --2- 172.21.15.3:0/294322948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7708103660 0x7f7708199ea0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 >> 172.21.15.3:0/294322948 conn(0x7f77080ff150 msgr2=0x7f77080ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:55.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 shutdown_connections 2024-03-21T08:41:55.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.446+0000 7f770d200640 1 -- 172.21.15.3:0/294322948 wait complete. 2024-03-21T08:41:55.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:41:55.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 -- 172.21.15.3:0/2221540274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c109850 msgr2=0x7f069c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 --2- 172.21.15.3:0/2221540274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c109850 0x7f069c109c30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f068c009930 tx=0x7f068c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:41:55.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 -- 172.21.15.3:0/2221540274 shutdown_connections 2024-03-21T08:41:55.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 --2- 172.21.15.3:0/2221540274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c104200 0x7f069c110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 --2- 172.21.15.3:0/2221540274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069c103840 0x7f069c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 --2- 172.21.15.3:0/2221540274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c109850 0x7f069c109c30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 -- 172.21.15.3:0/2221540274 >> 172.21.15.3:0/2221540274 conn(0x7f069c0ff310 msgr2=0x7f069c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:55.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 -- 172.21.15.3:0/2221540274 shutdown_connections 2024-03-21T08:41:55.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.693+0000 7f06a0c00640 1 -- 172.21.15.3:0/2221540274 wait complete. 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.694+0000 7f06a0c00640 1 Processor -- start 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f06a0c00640 1 -- start start 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f06a0c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c103840 0x7f069c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f06a0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c104200 0x7f069c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f06a0c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069c109850 0x7f069c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f06a0c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c113ad0 con 0x7f069c104200 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f06a0c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c113950 con 0x7f069c109850 2024-03-21T08:41:55.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f06a0c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c113c50 con 0x7f069c103840 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f0699400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c104200 0x7f069c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.695+0000 7f0699400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c104200 0x7f069c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43916/0 (socket says 172.21.15.3:43916) 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699400640 1 -- 172.21.15.3:0/511522948 learned_addr learned my addr 172.21.15.3:0/511522948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f069a800640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069c109850 0x7f069c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699e00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c103840 0x7f069c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699400640 1 -- 172.21.15.3:0/511522948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c103840 msgr2=0x7f069c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699400640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c103840 0x7f069c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699400640 1 -- 172.21.15.3:0/511522948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069c109850 msgr2=0x7f069c19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699400640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069c109850 0x7f069c19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699400640 1 -- 172.21.15.3:0/511522948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c19f090 con 0x7f069c104200 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f069a800640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069c109850 0x7f069c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.696+0000 7f0699400640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c104200 0x7f069c19a5e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f068800e9f0 tx=0x7f068800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.697+0000 7f0692a00640 1 -- 172.21.15.3:0/511522948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068800cd80 con 0x7f069c104200 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.697+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c19f380 con 0x7f069c104200 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.697+0000 7f0692a00640 1 -- 172.21.15.3:0/511522948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f06880108e0 con 0x7f069c104200 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.697+0000 7f0692a00640 1 -- 172.21.15.3:0/511522948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688018920 con 0x7f069c104200 2024-03-21T08:41:55.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.697+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c1a6c10 con 0x7f069c104200 2024-03-21T08:41:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.698+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c068520 con 0x7f069c104200 2024-03-21T08:41:55.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.701+0000 7f0692a00640 1 -- 172.21.15.3:0/511522948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f0688010a90 con 0x7f069c104200 2024-03-21T08:41:55.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.702+0000 7f0692a00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f06800a3090 0x7f06800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:55.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.702+0000 7f0692a00640 1 -- 172.21.15.3:0/511522948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f06880d0100 con 0x7f069c104200 2024-03-21T08:41:55.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.702+0000 7f0699e00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f06800a3090 0x7f06800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:55.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.703+0000 7f0699e00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f06800a3090 0x7f06800a5550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f068c006010 tx=0x7f068c02c670 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:55.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.707+0000 7f0692a00640 1 -- 172.21.15.3:0/511522948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f068809c710 con 0x7f069c104200 2024-03-21T08:41:55.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:55 smithi084 ceph-mon[35477]: pgmap v313: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:55.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:55 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/4131622494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:41:55.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:55 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/294322948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:41:55.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:55 smithi171 ceph-mon[35388]: pgmap v313: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:55.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:55 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/4131622494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:41:55.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:55 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/294322948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:41:55.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.911+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f069c10b820 con 0x7f069c104200 2024-03-21T08:41:55.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.912+0000 7f0692a00640 1 -- 172.21.15.3:0/511522948 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f068809c010 con 0x7f069c104200 2024-03-21T08:41:55.914 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:41:55.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f06800a3090 msgr2=0x7f06800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f06800a3090 0x7f06800a5550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f068c006010 tx=0x7f068c02c670 comp rx=0 tx=0).stop 2024-03-21T08:41:55.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c104200 msgr2=0x7f069c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:55.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c104200 0x7f069c19a5e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f068800e9f0 tx=0x7f068800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 shutdown_connections 2024-03-21T08:41:55.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069c109850 0x7f069c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f069c104200 0x7f069c19a5e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f06800a3090 0x7f06800a5550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 --2- 172.21.15.3:0/511522948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c103840 0x7f069c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:55.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 >> 172.21.15.3:0/511522948 conn(0x7f069c0ff310 msgr2=0x7f069c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:55.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 shutdown_connections 2024-03-21T08:41:55.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:55.915+0000 7f06a0c00640 1 -- 172.21.15.3:0/511522948 wait complete. 2024-03-21T08:41:55.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:55 smithi003 ceph-mon[31675]: pgmap v313: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:41:55.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:55 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/4131622494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:41:55.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:55 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/294322948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:41:55.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:41:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 -- 172.21.15.3:0/501182455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 msgr2=0x7ff16010cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 --2- 172.21.15.3:0/501182455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff16010cd60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff150009ca0 tx=0x7ff15002c180 comp rx=0 tx=0).stop 2024-03-21T08:41:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 -- 172.21.15.3:0/501182455 shutdown_connections 2024-03-21T08:41:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 --2- 172.21.15.3:0/501182455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff160108440 0x7ff160114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 --2- 172.21.15.3:0/501182455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff160107aa0 0x7ff160107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 --2- 172.21.15.3:0/501182455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff16010cd60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 -- 172.21.15.3:0/501182455 >> 172.21.15.3:0/501182455 conn(0x7ff1600772a0 msgr2=0x7ff1600796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 -- 172.21.15.3:0/501182455 shutdown_connections 2024-03-21T08:41:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.163+0000 7ff165200640 1 -- 172.21.15.3:0/501182455 wait complete. 2024-03-21T08:41:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.164+0000 7ff165200640 1 Processor -- start 2024-03-21T08:41:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.164+0000 7ff165200640 1 -- start start 2024-03-21T08:41:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff165200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff160107aa0 0x7ff16019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff165200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff160108440 0x7ff16019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff165200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff1601a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff165200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff160117cf0 con 0x7ff16010c980 2024-03-21T08:41:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff165200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff160117b70 con 0x7ff160107aa0 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff165200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff160117e70 con 0x7ff160108440 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff15fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff1601a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff15fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff1601a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43936/0 (socket says 172.21.15.3:43936) 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff15fe00640 1 -- 172.21.15.3:0/165601487 learned_addr learned my addr 172.21.15.3:0/165601487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff15f400640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff160107aa0 0x7ff16019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.165+0000 7ff156a00640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff160108440 0x7ff16019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff15fe00640 1 -- 172.21.15.3:0/165601487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff160108440 msgr2=0x7ff16019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff15fe00640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff160108440 0x7ff16019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff15fe00640 1 -- 172.21.15.3:0/165601487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff160107aa0 msgr2=0x7ff16019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff15fe00640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff160107aa0 0x7ff16019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff15fe00640 1 -- 172.21.15.3:0/165601487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1601a31f0 con 0x7ff16010c980 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff156a00640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff160108440 0x7ff16019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff15f400640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff160107aa0 0x7ff16019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.166+0000 7ff15fe00640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff1601a2ac0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c970 tx=0x7ff15800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.167+0000 7ff15cc00640 1 -- 172.21.15.3:0/165601487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158007bb0 con 0x7ff16010c980 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.167+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1601a34e0 con 0x7ff16010c980 2024-03-21T08:41:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.167+0000 7ff15cc00640 1 -- 172.21.15.3:0/165601487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff158007d50 con 0x7ff16010c980 2024-03-21T08:41:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.167+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1601aad70 con 0x7ff16010c980 2024-03-21T08:41:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.167+0000 7ff15cc00640 1 -- 172.21.15.3:0/165601487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158018720 con 0x7ff16010c980 2024-03-21T08:41:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.168+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16004f400 con 0x7ff16010c980 2024-03-21T08:41:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.170+0000 7ff15cc00640 1 -- 172.21.15.3:0/165601487 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7ff15800fa20 con 0x7ff16010c980 2024-03-21T08:41:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.172+0000 7ff15cc00640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff1440a3090 0x7ff1440a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.172+0000 7ff15cc00640 1 -- 172.21.15.3:0/165601487 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7ff1580cfc80 con 0x7ff16010c980 2024-03-21T08:41:56.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.172+0000 7ff15f400640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff1440a3090 0x7ff1440a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.173+0000 7ff15f400640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff1440a3090 0x7ff1440a5550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff150000f80 tx=0x7ff150032340 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:56.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.176+0000 7ff15cc00640 1 -- 172.21.15.3:0/165601487 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff158014030 con 0x7ff16010c980 2024-03-21T08:41:56.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.382+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff16002d070 con 0x7ff16010c980 2024-03-21T08:41:56.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.383+0000 7ff15cc00640 1 -- 172.21.15.3:0/165601487 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff15809c2f0 con 0x7ff16010c980 2024-03-21T08:41:56.385 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:41:56.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff1440a3090 msgr2=0x7ff1440a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff1440a3090 0x7ff1440a5550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff150000f80 tx=0x7ff150032340 comp rx=0 tx=0).stop 2024-03-21T08:41:56.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 msgr2=0x7ff1601a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff1601a2ac0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c970 tx=0x7ff15800ce30 comp rx=0 tx=0).stop 2024-03-21T08:41:56.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 shutdown_connections 2024-03-21T08:41:56.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff16010c980 0x7ff1601a2ac0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7ff1440a3090 0x7ff1440a5550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff160108440 0x7ff16019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 --2- 172.21.15.3:0/165601487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff160107aa0 0x7ff16019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.387+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 >> 172.21.15.3:0/165601487 conn(0x7ff1600772a0 msgr2=0x7ff160077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:56.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.388+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 shutdown_connections 2024-03-21T08:41:56.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.388+0000 7ff165200640 1 -- 172.21.15.3:0/165601487 wait complete. 2024-03-21T08:41:56.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:41:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 -- 172.21.15.3:0/149172079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650107500 msgr2=0x7f36501078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 --2- 172.21.15.3:0/149172079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650107500 0x7f36501078e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3640009930 tx=0x7f364002bc30 comp rx=0 tx=0).stop 2024-03-21T08:41:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 -- 172.21.15.3:0/149172079 shutdown_connections 2024-03-21T08:41:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 --2- 172.21.15.3:0/149172079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3650101e20 0x7f365010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 --2- 172.21.15.3:0/149172079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3650101460 0x7f36501018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 --2- 172.21.15.3:0/149172079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650107500 0x7f36501078e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 -- 172.21.15.3:0/149172079 >> 172.21.15.3:0/149172079 conn(0x7f36500fcff0 msgr2=0x7f36500ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 -- 172.21.15.3:0/149172079 shutdown_connections 2024-03-21T08:41:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.638+0000 7f364fe00640 1 -- 172.21.15.3:0/149172079 wait complete. 2024-03-21T08:41:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.639+0000 7f364fe00640 1 Processor -- start 2024-03-21T08:41:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.639+0000 7f364fe00640 1 -- start start 2024-03-21T08:41:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650101460 0x7f36500739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3650101e20 0x7f3650073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3650107500 0x7f3650074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501116b0 con 0x7f3650101460 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3650111530 con 0x7f3650107500 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3650111830 con 0x7f3650101e20 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650101460 0x7f36500739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650101460 0x7f36500739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43956/0 (socket says 172.21.15.3:43956) 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 -- 172.21.15.3:0/2275672592 learned_addr learned my addr 172.21.15.3:0/2275672592 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364f400640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3650107500 0x7f3650074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364e000640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3650101e20 0x7f3650073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 -- 172.21.15.3:0/2275672592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3650101e20 msgr2=0x7f3650073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3650101e20 0x7f3650073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 -- 172.21.15.3:0/2275672592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3650107500 msgr2=0x7f3650074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3650107500 0x7f3650074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364ea00640 1 -- 172.21.15.3:0/2275672592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36501a61b0 con 0x7f3650101460 2024-03-21T08:41:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364e000640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3650101e20 0x7f3650073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:41:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.640+0000 7f364f400640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3650107500 0x7f3650074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:41:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.641+0000 7f364ea00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650101460 0x7f36500739a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3640009230 tx=0x7f364002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.641+0000 7f3647400640 1 -- 172.21.15.3:0/2275672592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640043020 con 0x7f3650101460 2024-03-21T08:41:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.641+0000 7f3647400640 1 -- 172.21.15.3:0/2275672592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f364002caf0 con 0x7f3650101460 2024-03-21T08:41:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.641+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36501a6440 con 0x7f3650101460 2024-03-21T08:41:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.641+0000 7f3647400640 1 -- 172.21.15.3:0/2275672592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640003ae0 con 0x7f3650101460 2024-03-21T08:41:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.641+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36501a6990 con 0x7f3650101460 2024-03-21T08:41:56.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.643+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365004f400 con 0x7f3650101460 2024-03-21T08:41:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.645+0000 7f3647400640 1 -- 172.21.15.3:0/2275672592 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f3640046020 con 0x7f3650101460 2024-03-21T08:41:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.646+0000 7f3647400640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f36340a3090 0x7f36340a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.646+0000 7f3647400640 1 -- 172.21.15.3:0/2275672592 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f36400f1d60 con 0x7f3650101460 2024-03-21T08:41:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.646+0000 7f364e000640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f36340a3090 0x7f36340a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.647+0000 7f364e000640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f36340a3090 0x7f36340a5550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3650075430 tx=0x7f3638005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.650+0000 7f3647400640 1 -- 172.21.15.3:0/2275672592 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f364004d020 con 0x7f3650101460 2024-03-21T08:41:56.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:56 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/511522948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:41:56.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:56 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/165601487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:41:56.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.865+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36501099d0 con 0x7f3650101460 2024-03-21T08:41:56.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.866+0000 7f3647400640 1 -- 172.21.15.3:0/2275672592 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7f36400be480 con 0x7f3650101460 2024-03-21T08:41:56.871 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:41:56.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f36340a3090 msgr2=0x7f36340a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f36340a3090 0x7f36340a5550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3650075430 tx=0x7f3638005f90 comp rx=0 tx=0).stop 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650101460 msgr2=0x7f36500739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650101460 0x7f36500739a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3640009230 tx=0x7f364002c710 comp rx=0 tx=0).stop 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 shutdown_connections 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3650107500 0x7f3650074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f36340a3090 0x7f36340a5550 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3650101e20 0x7f3650073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 --2- 172.21.15.3:0/2275672592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3650101460 0x7f36500739a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:56.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.868+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 >> 172.21.15.3:0/2275672592 conn(0x7f36500fcff0 msgr2=0x7f36500fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:56.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.869+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 shutdown_connections 2024-03-21T08:41:56.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:56.869+0000 7f364fe00640 1 -- 172.21.15.3:0/2275672592 wait complete. 2024-03-21T08:41:56.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-21T08:41:57.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 -- 172.21.15.3:0/3095063200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c103760 msgr2=0x7f3b1c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:57.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 --2- 172.21.15.3:0/3095063200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c103760 0x7f3b1c103be0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009940 tx=0x7f3b0802bc20 comp rx=0 tx=0).stop 2024-03-21T08:41:57.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 -- 172.21.15.3:0/3095063200 shutdown_connections 2024-03-21T08:41:57.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 --2- 172.21.15.3:0/3095063200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b1c104120 0x7f3b1c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:57.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 --2- 172.21.15.3:0/3095063200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c103760 0x7f3b1c103be0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:57.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 --2- 172.21.15.3:0/3095063200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c109770 0x7f3b1c109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:57.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 -- 172.21.15.3:0/3095063200 >> 172.21.15.3:0/3095063200 conn(0x7f3b1c0ff1d0 msgr2=0x7f3b1c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:57.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 -- 172.21.15.3:0/3095063200 shutdown_connections 2024-03-21T08:41:57.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.111+0000 7f3b22400640 1 -- 172.21.15.3:0/3095063200 wait complete. 2024-03-21T08:41:57.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 Processor -- start 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 -- start start 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b1c103760 0x7f3b1c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c104120 0x7f3b1c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c109770 0x7f3b1c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c113a30 con 0x7f3b1c104120 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1138b0 con 0x7f3b1c109770 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.112+0000 7f3b22400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c113bb0 con 0x7f3b1c103760 2024-03-21T08:41:57.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c104120 0x7f3b1c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c104120 0x7f3b1c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43980/0 (socket says 172.21.15.3:43980) 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 -- 172.21.15.3:0/3326450538 learned_addr learned my addr 172.21.15.3:0/3326450538 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b21a00640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c109770 0x7f3b1c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b21000640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b1c103760 0x7f3b1c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 -- 172.21.15.3:0/3326450538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b1c103760 msgr2=0x7f3b1c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b1c103760 0x7f3b1c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 -- 172.21.15.3:0/3326450538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c109770 msgr2=0x7f3b1c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:57.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c109770 0x7f3b1c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:57.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 -- 172.21.15.3:0/3326450538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c19efc0 con 0x7f3b1c104120 2024-03-21T08:41:57.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.113+0000 7f3b1be00640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c104120 0x7f3b1c19a510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3b0802c160 tx=0x7f3b08005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:57.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.114+0000 7f3b19600640 1 -- 172.21.15.3:0/3326450538 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08039040 con 0x7f3b1c104120 2024-03-21T08:41:57.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.114+0000 7f3b19600640 1 -- 172.21.15.3:0/3326450538 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b08044070 con 0x7f3b1c104120 2024-03-21T08:41:57.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.114+0000 7f3b19600640 1 -- 172.21.15.3:0/3326450538 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08003ba0 con 0x7f3b1c104120 2024-03-21T08:41:57.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.114+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c19f250 con 0x7f3b1c104120 2024-03-21T08:41:57.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.114+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c1a6b40 con 0x7f3b1c104120 2024-03-21T08:41:57.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.115+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c04f470 con 0x7f3b1c104120 2024-03-21T08:41:57.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.117+0000 7f3b19600640 1 -- 172.21.15.3:0/3326450538 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f3b0802c5c0 con 0x7f3b1c104120 2024-03-21T08:41:57.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.119+0000 7f3b19600640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3b040a31d0 0x7f3b040a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:41:57.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.119+0000 7f3b19600640 1 -- 172.21.15.3:0/3326450538 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f3b080f17f0 con 0x7f3b1c104120 2024-03-21T08:41:57.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.120+0000 7f3b21000640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3b040a31d0 0x7f3b040a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:41:57.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.120+0000 7f3b21000640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3b040a31d0 0x7f3b040a5690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009a20 tx=0x7f3b10005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:41:57.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.124+0000 7f3b19600640 1 -- 172.21.15.3:0/3326450538 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b0802ce20 con 0x7f3b1c104120 2024-03-21T08:41:57.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:56 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/511522948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:41:57.126 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:56 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/165601487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:41:57.133 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:56 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/511522948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:41:57.134 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:56 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/165601487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:41:57.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.380+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f3b1c10bcf0 con 0x7f3b1c104120 2024-03-21T08:41:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.408+0000 7f3b19600640 1 -- 172.21.15.3:0/3326450538 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+742364 (secure 0 0 0) 0x7f3b080bdf10 con 0x7f3b1c104120 2024-03-21T08:41:57.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:41:57.414 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 26, 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "active_gid": 15654, 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "active_name": "x", 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addrs": { 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6800", 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1070134681 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6801", 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1070134681 2024-03-21T08:41:57.416 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addr": "172.21.15.3:6801/1070134681", 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "active_change": "2024-03-21T08:31:50.557775+0000", 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "active_mgr_features": 4540701547738038271, 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "available": true, 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "standbys": [ 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.417 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 15648, 2024-03-21T08:41:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "y", 2024-03-21T08:41:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-03-21T08:41:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-03-21T08:41:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-03-21T08:41:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.418 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:57.419 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.421 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.426 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.428 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-03-21T08:41:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-03-21T08:41:57.429 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.430 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-03-21T08:41:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-03-21T08:41:57.432 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.433 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.434 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.437 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.438 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-03-21T08:41:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-03-21T08:41:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.444 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T08:41:57.445 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.446 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-03-21T08:41:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-03-21T08:41:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T08:41:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T08:41:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-03-21T08:41:57.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-03-21T08:41:57.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T08:41:57.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-03-21T08:41:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-03-21T08:41:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-03-21T08:41:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-03-21T08:41:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T08:41:57.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-03-21T08:41:57.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.456 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.459 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.464 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.467 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.469 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.474 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-03-21T08:41:57.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T08:41:57.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-03-21T08:41:57.478 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-03-21T08:41:57.479 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.480 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.481 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-03-21T08:41:57.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.485 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T08:41:57.487 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T08:41:57.488 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-03-21T08:41:57.490 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-03-21T08:41:57.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.492 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.493 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.494 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.495 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.497 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-03-21T08:41:57.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-03-21T08:41:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-03-21T08:41:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T08:41:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.505 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-03-21T08:41:57.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-03-21T08:41:57.508 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-03-21T08:41:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-03-21T08:41:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.509 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T08:41:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-03-21T08:41:57.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.511 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.512 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-03-21T08:41:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-03-21T08:41:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T08:41:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-03-21T08:41:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-03-21T08:41:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-03-21T08:41:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-03-21T08:41:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T08:41:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T08:41:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-03-21T08:41:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-03-21T08:41:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T08:41:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-03-21T08:41:57.523 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T08:41:57.524 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T08:41:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.526 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.529 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-03-21T08:41:57.530 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-03-21T08:41:57.531 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.532 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.533 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:57.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.535 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.538 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.539 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.552 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-03-21T08:41:57.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T08:41:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-03-21T08:41:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-03-21T08:41:57.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.560 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-03-21T08:41:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-03-21T08:41:57.562 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-03-21T08:41:57.563 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-03-21T08:41:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.564 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.565 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.566 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.567 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.568 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.569 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-03-21T08:41:57.570 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T08:41:57.571 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.572 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.573 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-03-21T08:41:57.575 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-03-21T08:41:57.576 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.577 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.580 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T08:41:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-03-21T08:41:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-03-21T08:41:57.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T08:41:57.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.587 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.589 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.590 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.594 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.600 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.602 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T08:41:57.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.608 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T08:41:57.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.612 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T08:41:57.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.617 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T08:41:57.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T08:41:57.624 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.625 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T08:41:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T08:41:57.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.629 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.633 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.634 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.635 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T08:41:57.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.637 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.638 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T08:41:57.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.644 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.645 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T08:41:57.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.648 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.650 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T08:41:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.652 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.653 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.655 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.656 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-03-21T08:41:57.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.661 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T08:41:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.665 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.666 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T08:41:57.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.671 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.676 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T08:41:57.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.679 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-03-21T08:41:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-03-21T08:41:57.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.688 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-03-21T08:41:57.689 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.690 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-03-21T08:41:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-03-21T08:41:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.691 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.692 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.693 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.694 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.696 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.698 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.701 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T08:41:57.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.707 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-03-21T08:41:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.709 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-03-21T08:41:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-03-21T08:41:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.710 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-03-21T08:41:57.712 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-03-21T08:41:57.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.715 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.717 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-03-21T08:41:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-03-21T08:41:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-03-21T08:41:57.718 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.719 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.720 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.722 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.723 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.725 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.726 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-03-21T08:41:57.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T08:41:57.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-03-21T08:41:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.734 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-03-21T08:41:57.736 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.743 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.744 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.746 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.747 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-03-21T08:41:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.752 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:57.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.755 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-03-21T08:41:57.756 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-03-21T08:41:57.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.758 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-03-21T08:41:57.762 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.763 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.764 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.765 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.769 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.773 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-03-21T08:41:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.779 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-03-21T08:41:57.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.782 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.784 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.785 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.786 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.788 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.791 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.793 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.794 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.795 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.796 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.797 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.798 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.799 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.801 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.803 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.804 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.805 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.806 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.810 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.811 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-03-21T08:41:57.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-03-21T08:41:57.816 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.818 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.822 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-03-21T08:41:57.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-03-21T08:41:57.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T08:41:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.830 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.831 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.832 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.834 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.836 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.840 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.845 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.849 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.852 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.853 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.855 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.859 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.862 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.866 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.870 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.873 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.876 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.882 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.884 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.885 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.887 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.889 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.894 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.896 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-03-21T08:41:57.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.907 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-03-21T08:41:57.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T08:41:57.909 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.911 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.913 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.916 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.917 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:57.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.923 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-03-21T08:41:57.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T08:41:57.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.929 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.930 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.931 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.933 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.935 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-03-21T08:41:57.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.941 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:57.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T08:41:57.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-03-21T08:41:57.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.954 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.955 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.956 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.957 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.958 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.963 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.964 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.967 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-03-21T08:41:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-03-21T08:41:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.971 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.975 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.981 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.982 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.983 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.985 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.986 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.988 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.989 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-03-21T08:41:57.991 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.992 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.993 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.994 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:57.996 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:57.997 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:57.998 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:57.999 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-03-21T08:41:58.000 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T08:41:58.001 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-03-21T08:41:58.002 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.004 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-03-21T08:41:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-03-21T08:41:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-03-21T08:41:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.023 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.025 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.026 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.029 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-03-21T08:41:58.030 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.031 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-03-21T08:41:58.032 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.037 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.038 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.039 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.040 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.041 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.044 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.051 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-03-21T08:41:58.054 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.056 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.060 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.061 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.064 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.066 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.069 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.071 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.073 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.076 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.077 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-03-21T08:41:58.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T08:41:58.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T08:41:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T08:41:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.084 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T08:41:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-03-21T08:41:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-03-21T08:41:58.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T08:41:58.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-03-21T08:41:58.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.092 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-03-21T08:41:58.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.096 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.101 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.105 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-03-21T08:41:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-03-21T08:41:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.110 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.115 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.116 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.122 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.123 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.129 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.135 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.139 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-03-21T08:41:58.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.141 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.146 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.147 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.150 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.153 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.158 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 25201, 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "z", 2024-03-21T08:41:58.159 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.163 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.166 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-03-21T08:41:58.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-03-21T08:41:58.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.179 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-03-21T08:41:58.181 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.183 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.184 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.185 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.187 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-03-21T08:41:58.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T08:41:58.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T08:41:58.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T08:41:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T08:41:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-03-21T08:41:58.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-03-21T08:41:58.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T08:41:58.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.201 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.203 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.205 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.206 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-03-21T08:41:58.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.210 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-03-21T08:41:58.211 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.212 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.213 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T08:41:58.215 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.217 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-03-21T08:41:58.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.221 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.222 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-03-21T08:41:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-03-21T08:41:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.226 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.227 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.228 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-03-21T08:41:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T08:41:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T08:41:58.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T08:41:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-03-21T08:41:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.237 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T08:41:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-03-21T08:41:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-03-21T08:41:58.238 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-03-21T08:41:58.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:58.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T08:41:58.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.248 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-03-21T08:41:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-03-21T08:41:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T08:41:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-03-21T08:41:58.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.253 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.254 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-03-21T08:41:58.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-03-21T08:41:58.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.259 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T08:41:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T08:41:58.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-03-21T08:41:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-03-21T08:41:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-03-21T08:41:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T08:41:58.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.264 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-03-21T08:41:58.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T08:41:58.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.270 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-03-21T08:41:58.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-03-21T08:41:58.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.276 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T08:41:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-03-21T08:41:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-03-21T08:41:58.278 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-03-21T08:41:58.279 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-03-21T08:41:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-03-21T08:41:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.280 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.281 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-03-21T08:41:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-03-21T08:41:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.282 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.285 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-03-21T08:41:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T08:41:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.289 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.290 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T08:41:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.292 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.293 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.296 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.297 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T08:41:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T08:41:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.298 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-03-21T08:41:58.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.302 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T08:41:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-03-21T08:41:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-03-21T08:41:58.303 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-03-21T08:41:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-03-21T08:41:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-03-21T08:41:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.308 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.309 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T08:41:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T08:41:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-03-21T08:41:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-03-21T08:41:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-03-21T08:41:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-03-21T08:41:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.314 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-03-21T08:41:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-03-21T08:41:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-03-21T08:41:58.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T08:41:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-03-21T08:41:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-03-21T08:41:58.318 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T08:41:58.319 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.322 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-03-21T08:41:58.323 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T08:41:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.329 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T08:41:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.330 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-03-21T08:41:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-03-21T08:41:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.332 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-03-21T08:41:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.333 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.334 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.339 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.341 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-03-21T08:41:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-03-21T08:41:58.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T08:41:58.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.345 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.346 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T08:41:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T08:41:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.347 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.348 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T08:41:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T08:41:58.350 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.352 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T08:41:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T08:41:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.355 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.356 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T08:41:58.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T08:41:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T08:41:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T08:41:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T08:41:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T08:41:58.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T08:41:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T08:41:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T08:41:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T08:41:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-03-21T08:41:58.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.376 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.380 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.381 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T08:41:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T08:41:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.383 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.386 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T08:41:58.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T08:41:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T08:41:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.395 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.396 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T08:41:58.397 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T08:41:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T08:41:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.404 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T08:41:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T08:41:58.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T08:41:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T08:41:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-03-21T08:41:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.411 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.413 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T08:41:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T08:41:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.414 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T08:41:58.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.418 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T08:41:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T08:41:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T08:41:58.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T08:41:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T08:41:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.426 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.427 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.428 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-03-21T08:41:58.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-03-21T08:41:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.432 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.433 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-03-21T08:41:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.434 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.435 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.436 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.438 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.442 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-03-21T08:41:58.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.446 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.447 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.448 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.450 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-03-21T08:41:58.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.452 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-03-21T08:41:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-03-21T08:41:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-03-21T08:41:58.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-03-21T08:41:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-03-21T08:41:58.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-03-21T08:41:58.457 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-03-21T08:41:58.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.460 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-03-21T08:41:58.462 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-03-21T08:41:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-03-21T08:41:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.466 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.467 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.470 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.471 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-03-21T08:41:58.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.475 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-03-21T08:41:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-03-21T08:41:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-03-21T08:41:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-03-21T08:41:58.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.478 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-03-21T08:41:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-03-21T08:41:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-03-21T08:41:58.479 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T08:41:58.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T08:41:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.482 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.483 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.484 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-03-21T08:41:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-03-21T08:41:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-03-21T08:41:58.485 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T08:41:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.486 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.487 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.488 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.490 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.494 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.496 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.499 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-03-21T08:41:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-03-21T08:41:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.500 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-03-21T08:41:58.502 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.503 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-03-21T08:41:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-03-21T08:41:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-03-21T08:41:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-03-21T08:41:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-03-21T08:41:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-03-21T08:41:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T08:41:58.507 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.508 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-03-21T08:41:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T08:41:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-03-21T08:41:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T08:41:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.513 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.515 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.519 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.520 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.522 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-03-21T08:41:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-03-21T08:41:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.526 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-03-21T08:41:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T08:41:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.527 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:58.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.529 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-03-21T08:41:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-03-21T08:41:58.530 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3b040a31d0 msgr2=0x7f3b040a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3b040a31d0 0x7f3b040a5690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009a20 tx=0x7f3b10005f90 comp rx=0 tx=0).stop 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c104120 msgr2=0x7f3b1c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c104120 0x7f3b1c19a510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3b0802c160 tx=0x7f3b08005d70 comp rx=0 tx=0).stop 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 shutdown_connections 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c109770 0x7f3b1c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b1c104120 0x7f3b1c19a510 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f3b040a31d0 0x7f3b040a5690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 --2- 172.21.15.3:0/3326450538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b1c103760 0x7f3b1c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:41:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.430+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 >> 172.21.15.3:0/3326450538 conn(0x7f3b1c0ff1d0 msgr2=0x7f3b1c0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:41:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.431+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 shutdown_connections 2024-03-21T08:41:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:41:57.431+0000 7f3b22400640 1 -- 172.21.15.3:0/3326450538 wait complete. 2024-03-21T08:41:58.534 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:57 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2275672592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:41:58.535 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:57 smithi171 ceph-mon[35388]: pgmap v314: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T08:41:58.535 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:57 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3326450538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T08:41:58.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:57 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2275672592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:41:58.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:57 smithi084 ceph-mon[35477]: pgmap v314: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T08:41:58.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:57 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3326450538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T08:41:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-03-21T08:41:58.538 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T08:41:58.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.540 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.541 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.542 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.543 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.544 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.545 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.546 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.547 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.551 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.556 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.557 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.558 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-03-21T08:41:58.559 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.560 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-03-21T08:41:58.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.563 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.564 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.569 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.572 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.573 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-03-21T08:41:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-03-21T08:41:58.574 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.575 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.576 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.578 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.580 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.581 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.582 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-03-21T08:41:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-03-21T08:41:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-03-21T08:41:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T08:41:58.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-03-21T08:41:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-03-21T08:41:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.587 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.588 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-03-21T08:41:58.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-03-21T08:41:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.590 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-03-21T08:41:58.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T08:41:58.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.593 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.594 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.595 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.596 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.597 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.598 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.600 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.601 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.602 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.604 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.607 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.610 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-03-21T08:41:58.612 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.619 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.620 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.621 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.622 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.625 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.626 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.629 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.630 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.631 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.632 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.633 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-03-21T08:41:58.634 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.635 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.640 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.641 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.642 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.643 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.645 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.646 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.647 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.648 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.649 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.650 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.651 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.652 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-03-21T08:41:58.654 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.655 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.656 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.660 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.664 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.665 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.666 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-03-21T08:41:58.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.669 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.674 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.675 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.676 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.677 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-03-21T08:41:58.679 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.681 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.682 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.683 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.685 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.686 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.687 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.688 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.691 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.694 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T08:41:58.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T08:41:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-03-21T08:41:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.698 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.699 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.700 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-03-21T08:41:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T08:41:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.702 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.703 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-03-21T08:41:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-03-21T08:41:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-03-21T08:41:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-03-21T08:41:58.707 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.710 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.711 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.712 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.717 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.718 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.719 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-03-21T08:41:58.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-03-21T08:41:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-03-21T08:41:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-03-21T08:41:58.726 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.730 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.732 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.736 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.737 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.741 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.742 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.743 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.744 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.745 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.747 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.748 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-03-21T08:41:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T08:41:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-03-21T08:41:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.767 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.768 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.770 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-03-21T08:41:58.773 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.774 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.775 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.779 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.780 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.782 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-03-21T08:41:58.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.784 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.785 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.788 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.793 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-03-21T08:41:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T08:41:58.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.799 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.803 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.806 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.807 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.808 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.810 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.811 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.813 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.814 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.823 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.825 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-03-21T08:41:58.826 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T08:41:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-03-21T08:41:58.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.830 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.832 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.833 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.836 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.839 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-03-21T08:41:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.840 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-03-21T08:41:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-03-21T08:41:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-03-21T08:41:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-03-21T08:41:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-03-21T08:41:58.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.852 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.854 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.856 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-03-21T08:41:58.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.859 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.860 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-03-21T08:41:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-03-21T08:41:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.864 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.864 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.864 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.865 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.866 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.867 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.868 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.869 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.870 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-03-21T08:41:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-03-21T08:41:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T08:41:58.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.878 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.879 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-03-21T08:41:58.879 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.879 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.879 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.879 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.880 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.881 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.882 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.884 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.885 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.886 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.886 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.886 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.886 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.886 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.886 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.886 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.890 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.891 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.892 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.893 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.894 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.895 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.896 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-03-21T08:41:58.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.901 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T08:41:58.901 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.902 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-03-21T08:41:58.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-03-21T08:41:58.902 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.902 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.903 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-03-21T08:41:58.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.907 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.908 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-03-21T08:41:58.908 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.908 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.908 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.908 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-03-21T08:41:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-03-21T08:41:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-03-21T08:41:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.910 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.910 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.910 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.910 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.910 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.912 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.914 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.915 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.916 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.916 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.916 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.916 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.916 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.916 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.917 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.918 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.918 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.918 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.918 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.918 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.919 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.920 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.923 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-03-21T08:41:58.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-03-21T08:41:58.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.928 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.929 INFO:teuthology.orchestra.run.smithi003.stdout: "modules": [ 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard", 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: "iostat", 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: "nfs", 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: "restful", 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_schedule" 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-03-21T08:41:58.930 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-03-21T08:41:58.931 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.931 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.931 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.931 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:58.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:58.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.932 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.932 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.932 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.932 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.934 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.934 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.934 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.934 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.935 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.935 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.935 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.935 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.935 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.938 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.938 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.939 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-03-21T08:41:58.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.947 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-03-21T08:41:58.947 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.947 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.947 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.948 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.949 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.952 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.953 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.954 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.957 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.958 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:58.958 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T08:41:58.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.960 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-03-21T08:41:58.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-03-21T08:41:58.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T08:41:58.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-03-21T08:41:58.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-03-21T08:41:58.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.968 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.969 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.972 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:58.973 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.974 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:58.976 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.977 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-03-21T08:41:58.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.980 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.981 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-03-21T08:41:58.982 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-03-21T08:41:58.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.985 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.986 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:58.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-03-21T08:41:58.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:58.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-03-21T08:41:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.995 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.996 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.997 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:58.998 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-03-21T08:41:58.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T08:41:59.000 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:59.000 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.000 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.000 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-03-21T08:41:59.000 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-03-21T08:41:59.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-03-21T08:41:59.002 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-03-21T08:41:59.003 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-03-21T08:41:59.003 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T08:41:59.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T08:41:59.005 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.005 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-03-21T08:41:59.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-03-21T08:41:59.006 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.006 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.006 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.006 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-03-21T08:41:59.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-03-21T08:41:59.008 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.009 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.009 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T08:41:59.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.009 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.010 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-03-21T08:41:59.010 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-03-21T08:41:59.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T08:41:59.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-03-21T08:41:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-03-21T08:41:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T08:41:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-03-21T08:41:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-03-21T08:41:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-03-21T08:41:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T08:41:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-03-21T08:41:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-03-21T08:41:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.031 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T08:41:59.032 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.033 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-03-21T08:41:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-03-21T08:41:59.035 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.042 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-03-21T08:41:59.044 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.045 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:59.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.049 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-03-21T08:41:59.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-03-21T08:41:59.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T08:41:59.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-03-21T08:41:59.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.057 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.058 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.058 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.058 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.058 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.059 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.059 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.059 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.059 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.060 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-03-21T08:41:59.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-03-21T08:41:59.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.062 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T08:41:59.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.064 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.066 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.066 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.066 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.066 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.066 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.066 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.067 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.068 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.071 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-03-21T08:41:59.072 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.074 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-03-21T08:41:59.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.077 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-03-21T08:41:59.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-03-21T08:41:59.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.082 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-03-21T08:41:59.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T08:41:59.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.088 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-03-21T08:41:59.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.091 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.092 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.093 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.094 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-03-21T08:41:59.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-03-21T08:41:59.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.095 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-03-21T08:41:59.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T08:41:59.097 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-03-21T08:41:59.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.104 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.105 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.108 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.110 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.110 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.110 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.110 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.110 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.112 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.114 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.114 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.114 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.114 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.114 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.115 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.116 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-03-21T08:41:59.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T08:41:59.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.119 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.119 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.119 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.119 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T08:41:59.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.125 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.126 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T08:41:59.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.133 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T08:41:59.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.137 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T08:41:59.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T08:41:59.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.139 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T08:41:59.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T08:41:59.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T08:41:59.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:59.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.151 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T08:41:59.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-03-21T08:41:59.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.163 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.168 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.170 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T08:41:59.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T08:41:59.170 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.173 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T08:41:59.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T08:41:59.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.174 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T08:41:59.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T08:41:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.185 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T08:41:59.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T08:41:59.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-03-21T08:41:59.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T08:41:59.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T08:41:59.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.193 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.194 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.195 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.196 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.200 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-03-21T08:41:59.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-03-21T08:41:59.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-03-21T08:41:59.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.203 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.204 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-03-21T08:41:59.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-03-21T08:41:59.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.206 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.206 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.206 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.206 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.207 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.209 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.210 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.211 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.212 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.215 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:59.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.222 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-03-21T08:41:59.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-03-21T08:41:59.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-03-21T08:41:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-03-21T08:41:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-03-21T08:41:59.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-03-21T08:41:59.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.228 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.230 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.231 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.231 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.231 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.231 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.231 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.231 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-03-21T08:41:59.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-03-21T08:41:59.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.235 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.236 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.237 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.238 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.239 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.240 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.241 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.243 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.244 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T08:41:59.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.244 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.246 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-03-21T08:41:59.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.249 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.254 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.256 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.261 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.266 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.268 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.270 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.271 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-03-21T08:41:59.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-03-21T08:41:59.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.278 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.282 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.283 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.284 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.285 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.286 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.286 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.286 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-03-21T08:41:59.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.286 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-03-21T08:41:59.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-03-21T08:41:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.292 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.293 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.293 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-03-21T08:41:59.293 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.293 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.293 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.294 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.295 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.297 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.298 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.299 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.299 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.299 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.299 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.301 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.301 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.301 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.301 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.301 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.301 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.302 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.304 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.304 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.305 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.305 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.305 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.305 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.305 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.305 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.306 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-03-21T08:41:59.306 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.306 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.306 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.306 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.307 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.308 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.308 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.308 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.308 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.308 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.308 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.308 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.309 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.313 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.315 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.320 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.325 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.326 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.327 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.332 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.333 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.334 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.334 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.334 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.336 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.336 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.336 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.339 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-03-21T08:41:59.342 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.342 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.342 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-03-21T08:41:59.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.343 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-03-21T08:41:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.346 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.347 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.348 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T08:41:59.348 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T08:41:59.348 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.348 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.348 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.348 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.351 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.352 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.357 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.357 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-03-21T08:41:59.358 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.359 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.359 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.359 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.360 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.360 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.361 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.361 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.361 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.361 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.361 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.362 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.362 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.363 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-03-21T08:41:59.369 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.371 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.375 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.375 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.375 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.375 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.376 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.377 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.377 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.379 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.380 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-03-21T08:41:59.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-03-21T08:41:59.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.381 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.383 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.384 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.384 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.384 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.384 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.386 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.387 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.387 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.387 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.387 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.388 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.391 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.393 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.395 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.396 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.398 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.399 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.400 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.404 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.408 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.409 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.409 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.409 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.409 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.409 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.409 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.410 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.410 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-03-21T08:41:59.412 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.414 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-03-21T08:41:59.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.426 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.430 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.434 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.435 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.436 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.436 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.436 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.437 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.438 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.438 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.438 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-03-21T08:41:59.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-03-21T08:41:59.438 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.439 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-03-21T08:41:59.439 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-03-21T08:41:59.441 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.441 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-03-21T08:41:59.441 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-03-21T08:41:59.442 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.443 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.444 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.446 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T08:41:59.446 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T08:41:59.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.446 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.447 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.448 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.449 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.449 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.449 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.449 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.450 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.450 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.451 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.451 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.451 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.452 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.453 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.453 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.453 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.453 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.453 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T08:41:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:59.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-03-21T08:41:59.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.467 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.469 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.470 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.471 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.474 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.474 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.476 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.476 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.476 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.476 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.477 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.477 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.477 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.477 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.479 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.479 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.479 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.479 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.479 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.479 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.479 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-03-21T08:41:59.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-03-21T08:41:59.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T08:41:59.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.482 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.482 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.482 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.482 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-03-21T08:41:59.483 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.484 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.485 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.485 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.485 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.486 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.489 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.493 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.495 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.496 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.497 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.498 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.499 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.500 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.503 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.505 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.506 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.506 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.508 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.514 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.515 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.515 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.515 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.515 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.515 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.515 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.515 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.516 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.517 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.518 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.518 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.518 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.518 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.519 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.519 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.519 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.519 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.520 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.520 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.520 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.520 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.520 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.520 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.520 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.521 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.522 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.522 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.522 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.522 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.525 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.527 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.530 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.531 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.531 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.531 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.531 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.532 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.533 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.534 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-03-21T08:41:59.535 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.536 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-03-21T08:41:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-03-21T08:41:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.539 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-03-21T08:41:59.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-03-21T08:41:59.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.545 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.545 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.545 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.545 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.545 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.546 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.546 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.546 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.546 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.546 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-03-21T08:41:59.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-03-21T08:41:59.547 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.547 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.547 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.547 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.547 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.547 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-03-21T08:41:59.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.550 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-03-21T08:41:59.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-03-21T08:41:59.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.551 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.551 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.551 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.552 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.553 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-03-21T08:41:59.555 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.556 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.557 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.558 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.560 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.562 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.567 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.568 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.569 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-03-21T08:41:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.572 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.573 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.574 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.575 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.576 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.577 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.578 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.579 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.580 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.581 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.581 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.581 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.581 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.582 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.582 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.582 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.582 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.582 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.582 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.582 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.583 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.583 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.583 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.583 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.584 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.585 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.585 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.586 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.589 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-03-21T08:41:59.590 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.591 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.592 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.593 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.595 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.596 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.599 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.600 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.601 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.602 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.604 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.606 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.606 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.606 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.606 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T08:41:59.606 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.607 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.608 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.609 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T08:41:59.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.610 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.610 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.610 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.610 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.611 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-03-21T08:41:59.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-03-21T08:41:59.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.619 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.619 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:59.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:59.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.622 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-03-21T08:41:59.622 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.622 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.622 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.625 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-03-21T08:41:59.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-03-21T08:41:59.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.626 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.628 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.629 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.630 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.632 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-03-21T08:41:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.654 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.656 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.657 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.657 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.657 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.657 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.657 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-03-21T08:41:59.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-03-21T08:41:59.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-03-21T08:41:59.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-03-21T08:41:59.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T08:41:59.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-03-21T08:41:59.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T08:41:59.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-03-21T08:41:59.667 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-03-21T08:41:59.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.669 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-03-21T08:41:59.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-03-21T08:41:59.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.671 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-03-21T08:41:59.673 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.674 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-03-21T08:41:59.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.681 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-03-21T08:41:59.681 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-03-21T08:41:59.681 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.681 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.681 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.681 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.681 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.682 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.683 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-03-21T08:41:59.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.685 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-03-21T08:41:59.686 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.687 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: "services": { 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard": "https://172.21.15.3:8443/" 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: "always_on_modules": { 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: "octopus": [ 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-03-21T08:41:59.688 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-03-21T08:41:59.689 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "pacific": [ 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-03-21T08:41:59.690 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-03-21T08:41:59.691 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-03-21T08:41:59.691 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-03-21T08:41:59.691 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-03-21T08:41:59.691 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.691 INFO:teuthology.orchestra.run.smithi003.stdout: "quincy": [ 2024-03-21T08:41:59.691 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-03-21T08:41:59.691 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-03-21T08:41:59.692 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "reef": [ 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-03-21T08:41:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-03-21T08:41:59.694 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-03-21T08:41:59.694 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-03-21T08:41:59.694 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-03-21T08:41:59.694 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-03-21T08:41:59.694 INFO:teuthology.orchestra.run.smithi003.stdout: "squid": [ 2024-03-21T08:41:59.694 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-03-21T08:41:59.694 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-03-21T08:41:59.695 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: "last_failure_osd_epoch": 97, 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: "active_clients": [ 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3709616841 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "libcephsqlite", 2024-03-21T08:41:59.697 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-03-21T08:41:59.698 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-03-21T08:41:59.698 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-03-21T08:41:59.698 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1972323283 2024-03-21T08:41:59.698 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.698 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.698 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2216280453 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.699 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 408108914 2024-03-21T08:41:59.700 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-03-21T08:41:59.701 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2457496165 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-03-21T08:41:59.702 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1454387590 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-03-21T08:41:59.703 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:41:59.712 DEBUG:tasks.check_counter:Getting stats from a 2024-03-21T08:41:59.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-21T08:41:59.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:57 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2275672592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:41:59.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:57 smithi003 ceph-mon[31675]: pgmap v314: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T08:41:59.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:57 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3326450538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T08:42:00.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:41:59 smithi003 ceph-mon[31675]: pgmap v315: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T08:42:00.187 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:42:00.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:41:59 smithi084 ceph-mon[35477]: pgmap v315: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T08:42:00.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:41:59 smithi171 ceph-mon[35388]: pgmap v315: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T08:42:02.216 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:01 smithi003 ceph-mon[31675]: pgmap v316: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T08:42:02.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:01 smithi084 ceph-mon[35477]: pgmap v316: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T08:42:02.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:01 smithi171 ceph-mon[35388]: pgmap v316: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T08:42:02.527 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:42:02.527 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:02.527 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 229, 2024-03-21T08:42:02.527 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 218, 2024-03-21T08:42:02.528 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1582784, 2024-03-21T08:42:02.528 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 131024, 2024-03-21T08:42:02.528 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-03-21T08:42:02.528 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-03-21T08:42:02.528 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.105351558, 2024-03-21T08:42:02.528 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.032977528, 2024-03-21T08:42:02.528 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.083064363, 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.010582306, 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 217, 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.023685028, 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000109147 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:02.529 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:02.530 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:02.530 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:02.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.530 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1582784, 2024-03-21T08:42:02.530 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 131024 2024-03-21T08:42:02.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 167, 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 184416, 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:02.531 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.036391379, 2024-03-21T08:42:02.532 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.024969521, 2024-03-21T08:42:02.532 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.004948613, 2024-03-21T08:42:02.532 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:02.532 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:02.532 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 164, 2024-03-21T08:42:02.532 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.040287949, 2024-03-21T08:42:02.533 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000245658 2024-03-21T08:42:02.533 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.533 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:02.533 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:02.533 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:02.534 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:02.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.534 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T08:42:02.534 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 184416 2024-03-21T08:42:02.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.535 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:02.535 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 4, 2024-03-21T08:42:02.535 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 4, 2024-03-21T08:42:02.535 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 2944, 2024-03-21T08:42:02.535 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 448, 2024-03-21T08:42:02.535 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-03-21T08:42:02.535 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-03-21T08:42:02.536 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.001657032, 2024-03-21T08:42:02.536 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000084165, 2024-03-21T08:42:02.536 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001483970, 2024-03-21T08:42:02.536 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:02.536 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:02.536 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3, 2024-03-21T08:42:02.536 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.001645518, 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000548506 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.537 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-21T08:42:02.538 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T08:42:02.538 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.538 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-03-21T08:42:02.538 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-03-21T08:42:02.538 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-03-21T08:42:02.538 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.538 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-03-21T08:42:02.539 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:02.539 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:02.539 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:02.539 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:02.539 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:02.539 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:02.539 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:02.540 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:02.540 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:02.540 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:02.540 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:02.540 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:02.540 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:02.540 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:02.541 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:02.541 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:02.541 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:02.541 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:02.541 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:02.541 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:02.541 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:02.542 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:02.542 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:02.542 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:02.542 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:02.542 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:02.542 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:02.542 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:02.543 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:02.544 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148754, 2024-03-21T08:42:02.544 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 31, 2024-03-21T08:42:02.544 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:02.544 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:02.544 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-03-21T08:42:02.544 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-03-21T08:42:02.544 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:02.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:02.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:02.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:02.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-03-21T08:42:02.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-03-21T08:42:02.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:02.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-03-21T08:42:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-03-21T08:42:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-03-21T08:42:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-03-21T08:42:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-03-21T08:42:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-03-21T08:42:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-03-21T08:42:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-03-21T08:42:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 231, 2024-03-21T08:42:02.548 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1550520, 2024-03-21T08:42:02.548 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:02.548 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 231, 2024-03-21T08:42:02.548 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-03-21T08:42:02.548 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-03-21T08:42:02.548 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 231, 2024-03-21T08:42:02.548 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1550520, 2024-03-21T08:42:02.549 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-03-21T08:42:02.549 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:02.549 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:02.549 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:02.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:02.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:02.549 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:42:03.282 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-21T08:42:03.282 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-21T08:42:03.282 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-21T08:42:03.283 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-21T08:42:03.283 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-21T08:42:03.283 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-03-21T08:42:03.283 DEBUG:tasks.check_counter:Getting stats from d 2024-03-21T08:42:03.283 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-21T08:42:03.751 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:42:04.156 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:03 smithi003 ceph-mon[31675]: pgmap v317: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-03-21T08:42:04.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:03 smithi084 ceph-mon[35477]: pgmap v317: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-03-21T08:42:04.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:03 smithi171 ceph-mon[35388]: pgmap v317: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-03-21T08:42:06.051 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:42:06.051 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:06.051 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-03-21T08:42:06.051 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.004086954, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000724678, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:06.052 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:06.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.054 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:06.054 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:06.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.054 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:06.054 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 237, 2024-03-21T08:42:06.054 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 226, 2024-03-21T08:42:06.054 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1584960, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 136016, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.110744389, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.035773617, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.088920877, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.011181314, 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:06.055 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 225, 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.021052407, 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000093566 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:06.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1584960, 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 136016 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 6, 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 175, 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 3264, 2024-03-21T08:42:06.057 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 185920, 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.037658074, 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.026808110, 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.004872718, 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:06.058 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 173, 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.036810674, 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000212778 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.059 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 185920 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:06.060 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:06.061 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:06.062 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:06.063 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:06.064 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148754, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 31, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-03-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 239, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1551560, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 239, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-03-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 239, 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1551560, 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:06.069 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:42:06.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:05 smithi003 ceph-mon[31675]: pgmap v318: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T08:42:06.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:05 smithi084 ceph-mon[35477]: pgmap v318: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T08:42:06.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:05 smithi171 ceph-mon[35388]: pgmap v318: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T08:42:06.979 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-21T08:42:06.979 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-21T08:42:06.979 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-21T08:42:06.979 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-21T08:42:06.980 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-21T08:42:06.980 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-03-21T08:42:06.980 DEBUG:tasks.check_counter:Getting stats from g 2024-03-21T08:42:06.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-21T08:42:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:42:08.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:07 smithi003 ceph-mon[31675]: pgmap v319: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-21T08:42:08.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:07 smithi003 ceph-mon[31675]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T08:42:08.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:07 smithi084 ceph-mon[35477]: pgmap v319: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-21T08:42:08.376 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:07 smithi084 ceph-mon[35477]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T08:42:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:07 smithi171 ceph-mon[35388]: pgmap v319: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-21T08:42:08.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:07 smithi171 ceph-mon[35388]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T08:42:10.231 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:42:10.231 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:10.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-03-21T08:42:10.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-03-21T08:42:10.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-03-21T08:42:10.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-03-21T08:42:10.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-03-21T08:42:10.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-03-21T08:42:10.232 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.004105986, 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000662792, 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:10.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:10.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.235 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:10.235 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 232, 2024-03-21T08:42:10.235 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 221, 2024-03-21T08:42:10.235 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1583600, 2024-03-21T08:42:10.235 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 132896, 2024-03-21T08:42:10.235 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-03-21T08:42:10.235 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.104801182, 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.034643866, 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.085257746, 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.010121396, 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 220, 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.021604043, 2024-03-21T08:42:10.236 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000098200 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1583600, 2024-03-21T08:42:10.237 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 132896 2024-03-21T08:42:10.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.238 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:10.238 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 6, 2024-03-21T08:42:10.238 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 171, 2024-03-21T08:42:10.238 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 3264, 2024-03-21T08:42:10.238 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 185216, 2024-03-21T08:42:10.238 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:10.239 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:10.239 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.038229369, 2024-03-21T08:42:10.239 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.026429727, 2024-03-21T08:42:10.239 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.005618240, 2024-03-21T08:42:10.239 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:10.239 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:10.239 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 169, 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.040520448, 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000239765 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:10.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 185216 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.241 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-03-21T08:42:10.242 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:10.242 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:10.242 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:10.242 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:10.242 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:10.242 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:10.242 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:10.243 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:10.243 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:10.243 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:10.243 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:10.243 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:10.243 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:10.243 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:10.244 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:10.244 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:10.244 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:10.244 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:10.244 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:10.244 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:10.244 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:10.245 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:10.246 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:10.246 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:10.246 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-03-21T08:42:10.246 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:10.246 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:10.246 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:10.246 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148714, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 30, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:10.247 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:10.248 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:10.250 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-03-21T08:42:10.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-03-21T08:42:10.250 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-03-21T08:42:10.250 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 234, 2024-03-21T08:42:10.250 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1550910, 2024-03-21T08:42:10.250 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 234, 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 234, 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1550910, 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:10.251 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:10.252 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:10.252 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:10.252 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:10.252 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:42:10.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:09 smithi003 ceph-mon[31675]: pgmap v321: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T08:42:10.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:09 smithi084 ceph-mon[35477]: pgmap v321: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T08:42:10.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:09 smithi171 ceph-mon[35388]: pgmap v321: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T08:42:11.278 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-21T08:42:11.278 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-21T08:42:11.278 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-21T08:42:11.279 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-21T08:42:11.279 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-21T08:42:11.279 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-03-21T08:42:11.279 DEBUG:tasks.check_counter:Getting stats from j 2024-03-21T08:42:11.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-21T08:42:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:42:12.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:11 smithi003 ceph-mon[31675]: pgmap v322: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T08:42:12.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:11 smithi084 ceph-mon[35477]: pgmap v322: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T08:42:12.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:11 smithi171 ceph-mon[35388]: pgmap v322: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T08:42:13.906 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-03-21T08:42:13.906 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:13.906 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.003729283, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T08:42:13.907 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000713963, 2024-03-21T08:42:13.908 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:13.908 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:13.908 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:13.908 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:13.908 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:13.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:13.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 244, 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 231, 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1588720, 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 138336, 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-03-21T08:42:13.910 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.103265393, 2024-03-21T08:42:13.911 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.036549126, 2024-03-21T08:42:13.911 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.075056300, 2024-03-21T08:42:13.911 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.011051668, 2024-03-21T08:42:13.911 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:13.911 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 231, 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.024005221, 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000103918 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.912 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1588720, 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 138336 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 175, 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 185824, 2024-03-21T08:42:13.913 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:13.914 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:13.914 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.039144923, 2024-03-21T08:42:13.914 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.026906812, 2024-03-21T08:42:13.914 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.006282385, 2024-03-21T08:42:13.914 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 172, 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.035805793, 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000208173 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:13.915 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 185824 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-03-21T08:42:13.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.917 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-03-21T08:42:13.917 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:13.917 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:13.917 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:13.917 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:13.917 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:13.918 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:13.919 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:13.919 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:13.919 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:13.919 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:13.919 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:13.919 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:13.919 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:13.920 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:13.920 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:13.920 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:13.920 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:13.920 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-03-21T08:42:13.921 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:13.922 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:13.922 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:13.922 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:13.922 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148794, 2024-03-21T08:42:13.922 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 32, 2024-03-21T08:42:13.922 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:13.922 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:13.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-03-21T08:42:13.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-03-21T08:42:13.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:13.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:13.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:13.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:13.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-03-21T08:42:13.924 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-03-21T08:42:13.924 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:13.924 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:13.924 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:13.924 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-03-21T08:42:13.924 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-03-21T08:42:13.924 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-03-21T08:42:13.925 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:13.925 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-03-21T08:42:13.925 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:13.925 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-03-21T08:42:13.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-03-21T08:42:13.925 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:13.925 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-03-21T08:42:13.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-03-21T08:42:13.926 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-03-21T08:42:13.926 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 242, 2024-03-21T08:42:13.926 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1551950, 2024-03-21T08:42:13.926 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:13.926 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 242, 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 242, 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1551950, 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-03-21T08:42:13.927 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-03-21T08:42:13.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:13.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-03-21T08:42:13.928 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-03-21T08:42:14.328 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:14 smithi003 ceph-mon[31675]: pgmap v323: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:42:14.375 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:14 smithi084 ceph-mon[35477]: pgmap v323: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:42:14.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:14 smithi171 ceph-mon[35388]: pgmap v323: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:42:14.715 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-21T08:42:14.715 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-21T08:42:14.715 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-21T08:42:14.715 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-21T08:42:14.716 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-21T08:42:14.716 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-03-21T08:42:14.716 DEBUG:tasks.check_counter:Getting stats from b 2024-03-21T08:42:14.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-21T08:42:15.195 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:42:16.280 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:16 smithi084 ceph-mon[35477]: pgmap v324: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:42:16.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:16 smithi171 ceph-mon[35388]: pgmap v324: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:42:16.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:16 smithi003 ceph-mon[31675]: pgmap v324: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T08:42:17.759 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-03-21T08:42:17.759 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:17.759 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 0, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 0, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 0, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 0, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.003802428, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.000859284, 2024-03-21T08:42:17.760 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:17.761 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 236, 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 223, 2024-03-21T08:42:17.762 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1586544, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 133344, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 2, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.068163272, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.016843920, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.057446391, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.008662327, 2024-03-21T08:42:17.763 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 223, 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.016802382, 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000075347 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:17.764 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1586544, 2024-03-21T08:42:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 133344 2024-03-21T08:42:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2, 2024-03-21T08:42:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 169, 2024-03-21T08:42:17.766 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 320, 2024-03-21T08:42:17.766 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 121536, 2024-03-21T08:42:17.766 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:17.766 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:17.766 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.019921790, 2024-03-21T08:42:17.766 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.011643919, 2024-03-21T08:42:17.767 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.003689496, 2024-03-21T08:42:17.767 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:17.767 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:17.767 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 166, 2024-03-21T08:42:17.767 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.029207883, 2024-03-21T08:42:17.767 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000175951 2024-03-21T08:42:17.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.768 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:17.768 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:17.768 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:17.768 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:17.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.768 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T08:42:17.769 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 121536 2024-03-21T08:42:17.769 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.769 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-03-21T08:42:17.769 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 0, 2024-03-21T08:42:17.769 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-03-21T08:42:17.769 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.770 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-03-21T08:42:17.770 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:17.770 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:17.770 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:17.770 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:17.770 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:17.771 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:17.772 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:17.773 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:17.774 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:17.774 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-03-21T08:42:17.774 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:17.774 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:17.774 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:17.774 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:17.774 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 148731, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 31, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:17.775 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:17.776 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 0, 2024-03-21T08:42:17.776 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 0, 2024-03-21T08:42:17.776 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:17.776 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:17.776 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:17.776 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-03-21T08:42:17.776 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 928, 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2, 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:17.777 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 234, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1550910, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 234, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:17.778 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:17.779 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 234, 2024-03-21T08:42:17.779 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1550910, 2024-03-21T08:42:17.779 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:17.779 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:17.779 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:17.779 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:17.779 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:17.780 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:17.780 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-03-21T08:42:18.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:18 smithi084 ceph-mon[35477]: pgmap v325: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:18.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:18 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:42:18.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:18 smithi084 ceph-mon[35477]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T08:42:18.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:18 smithi171 ceph-mon[35388]: pgmap v325: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:18.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:18 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:42:18.384 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:18 smithi171 ceph-mon[35388]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T08:42:18.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:18 smithi003 ceph-mon[31675]: pgmap v325: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:18.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:18 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:42:18.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:18 smithi003 ceph-mon[31675]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T08:42:18.565 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.b 2024-03-21T08:42:18.565 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.b 2024-03-21T08:42:18.565 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.b 2024-03-21T08:42:18.565 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.b 2024-03-21T08:42:18.566 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.b 2024-03-21T08:42:18.566 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.b 2024-03-21T08:42:18.566 DEBUG:tasks.check_counter:Getting stats from e 2024-03-21T08:42:18.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-21T08:42:19.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:19 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:19.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:19 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:19.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:19 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:19.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:19 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:19.769 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:42:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:19 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:19.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:19 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:20.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:20 smithi084 ceph-mon[35477]: pgmap v327: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:20.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:20 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:20.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:20 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:20.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:20 smithi171 ceph-mon[35388]: pgmap v327: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:20.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:20 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:20.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:20 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:20.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:20 smithi003 ceph-mon[31675]: pgmap v327: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:20.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:20 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:20.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:20 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:22.237 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-03-21T08:42:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 29362, 2024-03-21T08:42:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 24320, 2024-03-21T08:42:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 63288341, 2024-03-21T08:42:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 46587001, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 11, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 4.072471936, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 2.148118979, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 2.407552002, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.112213412, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 24323, 2024-03-21T08:42:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1.776010174, 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000073017 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 47804, 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.013929224, 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000291 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 6240, 2024-03-21T08:42:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 12640 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 9193, 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 17278, 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 2945846, 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 24320053, 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 24, 2024-03-21T08:42:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 8, 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 1.293680890, 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.630780243, 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.684255609, 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.048230428, 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 17271, 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1.824766917, 2024-03-21T08:42:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000105654 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 18446, 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.006488476, 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000351 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 2256, 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 395456 2024-03-21T08:42:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 3623, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 4258, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 5271354, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 23022239, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 9, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 6, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.542213701, 2024-03-21T08:42:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.252967349, 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.206361652, 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.097914895, 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4257, 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.461483764, 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000108405 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.245 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1838, 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.001290330, 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000702 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1690912, 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 147648 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.246 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5343, 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 8.320799980, 2024-03-21T08:42:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001557327 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2571, 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.261128191, 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000101566 2024-03-21T08:42:22.248 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 5413, 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 5315, 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5315, 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 17.518446719, 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003296038 2024-03-21T08:42:22.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 98, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 4, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 43, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 4, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-03-21T08:42:22.250 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 6882, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 352, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 5443, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 1087, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 6087, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 1130, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 3196, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 3196, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 15, 2024-03-21T08:42:22.251 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 15210, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 6037, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 92, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 28, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 3, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 2, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-03-21T08:42:22.252 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-03-21T08:42:22.253 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 29, 2024-03-21T08:42:22.253 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 968, 2024-03-21T08:42:22.253 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 185, 2024-03-21T08:42:22.253 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 6438, 2024-03-21T08:42:22.253 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-03-21T08:42:22.253 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 120, 2024-03-21T08:42:22.253 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 3452, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 26526745, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 120, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-03-21T08:42:22.254 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 13, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 13, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 107, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 48, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 2243, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T08:42:22.255 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 2329, 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 1489, 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 126, 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 3352 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-03-21T08:42:22.256 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 209, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 8, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 1, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 1130, 2024-03-21T08:42:22.257 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 1127, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 2, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-03-21T08:42:22.258 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 53, 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 4, 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-03-21T08:42:22.259 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 96858, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 96858, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 107, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-03-21T08:42:22.260 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 107, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 9, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 69886182, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 0, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2554, 2024-03-21T08:42:22.261 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 12.474465775, 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004884285 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 0 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 5827, 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 7871, 2024-03-21T08:42:22.262 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 2044, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 235, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 255, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 20, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 6882, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 9155, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 2273, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 3196, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 4862, 2024-03-21T08:42:22.263 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 1666, 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 84988, 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 223612 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 12480, 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 8, 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 5413, 2024-03-21T08:42:22.264 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 52, 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 2249, 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1085, 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 5.628106169, 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.005187194 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 22, 2024-03-21T08:42:22.265 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.010366767, 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000471216 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-03-21T08:42:22.266 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2, 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.008785116, 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004392558 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3053, 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1.738706054, 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000569507 2024-03-21T08:42:22.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.268 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-03-21T08:42:22.269 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.270 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-03-21T08:42:22.271 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.271 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.271 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-03-21T08:42:22.272 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.273 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4, 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.003682020, 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000920505 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 14, 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.042799947, 2024-03-21T08:42:22.274 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003057139 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5, 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3.108334830, 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.621666966 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-03-21T08:42:22.275 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1, 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.032593488, 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.032593488 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.277 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-03-21T08:42:22.278 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.280 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.281 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1129, 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 6.945072328, 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.006151525 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.282 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-03-21T08:42:22.283 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 2, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 3, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 1, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 2, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 1207, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 603, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 635, 2024-03-21T08:42:22.284 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:22.285 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:22.286 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:22.287 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:22.288 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 307959, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 120, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:22.289 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 10088, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 194, 2024-03-21T08:42:22.290 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 18517481, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 87758, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-03-21T08:42:22.291 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 5840, 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 66339541, 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 33, 2024-03-21T08:42:22.292 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 5840, 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5840, 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 45.717194676, 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.007828286 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5840, 2024-03-21T08:42:22.293 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 6234 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 5840, 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 362, 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 5478, 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 0, 2024-03-21T08:42:22.294 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 10, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 238, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 2792, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 150, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-03-21T08:42:22.295 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 2285, 2024-03-21T08:42:22.296 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-03-21T08:42:22.296 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-03-21T08:42:22.296 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-03-21T08:42:22.296 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 122, 2024-03-21T08:42:22.296 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 20, 2024-03-21T08:42:22.296 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-03-21T08:42:22.296 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-03-21T08:42:22.297 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-03-21T08:42:22.298 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-03-21T08:42:22.299 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-03-21T08:42:22.300 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 107, 2024-03-21T08:42:22.300 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-03-21T08:42:22.300 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 13, 2024-03-21T08:42:22.300 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 12, 2024-03-21T08:42:22.300 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 12, 2024-03-21T08:42:22.300 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-03-21T08:42:22.300 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-03-21T08:42:22.301 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 441, 2024-03-21T08:42:22.301 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 8, 2024-03-21T08:42:22.301 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 168 2024-03-21T08:42:22.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.301 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-03-21T08:42:22.301 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 1, 2024-03-21T08:42:22.301 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-03-21T08:42:22.302 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-03-21T08:42:22.302 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-03-21T08:42:22.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.302 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-03-21T08:42:22.302 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-03-21T08:42:22.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.302 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 16, 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 8, 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 2254, 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 1127, 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-03-21T08:42:22.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.304 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:22.304 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:22.304 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-03-21T08:42:22.304 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:22.304 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 42174, 2024-03-21T08:42:22.304 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 67690426, 2024-03-21T08:42:22.304 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 42174, 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 42174, 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 67690426, 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.305 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:22.306 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-03-21T08:42:22.307 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-03-21T08:42:22.307 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:22.307 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:22.307 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 5840, 2024-03-21T08:42:22.307 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 66478328, 2024-03-21T08:42:22.307 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 5840, 2024-03-21T08:42:22.307 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 66478328, 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-03-21T08:42:22.308 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 5840, 2024-03-21T08:42:22.309 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 5840, 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 5840, 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 5840, 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 1127, 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 138851, 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:22.311 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 1127, 2024-03-21T08:42:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 238, 2024-03-21T08:42:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 138851, 2024-03-21T08:42:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x5555cf0de460": { 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:22.313 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 96858, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 65691878, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 96858, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2554, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 65691878, 2024-03-21T08:42:22.314 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:22.315 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:22.315 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:22.315 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:22.315 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.315 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:22.315 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-03-21T08:42:22.316 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: pgmap v328: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:22.316 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:22.316 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:22.316 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.316 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.317 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.317 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.317 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.317 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.317 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.317 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:22 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: pgmap v328: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:22.427 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:22 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: pgmap v328: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T08:42:22.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:22.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:22.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:22.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:22 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.289 INFO:tasks.check_counter:Daemon mds.e mds.exported=29 2024-03-21T08:42:23.289 INFO:tasks.check_counter:Daemon mds.e mds.imported=185 2024-03-21T08:42:23.289 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-21T08:42:23.290 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-21T08:42:23.290 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T08:42:23.290 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=4 2024-03-21T08:42:23.290 DEBUG:tasks.check_counter:Getting stats from h 2024-03-21T08:42:23.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-21T08:42:23.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi003 to 177.8M 2024-03-21T08:42:23.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: Unable to set osd_memory_target on smithi003 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T08:42:23.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.428 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi171 to 176.8M 2024-03-21T08:42:23.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:23 smithi003 ceph-mon[31675]: Unable to set osd_memory_target on smithi171 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T08:42:23.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi003 to 177.8M 2024-03-21T08:42:23.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: Unable to set osd_memory_target on smithi003 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T08:42:23.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.626 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi171 to 176.8M 2024-03-21T08:42:23.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:23 smithi084 ceph-mon[35477]: Unable to set osd_memory_target on smithi171 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T08:42:23.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi003 to 177.8M 2024-03-21T08:42:23.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: Unable to set osd_memory_target on smithi003 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T08:42:23.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:23.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi171 to 176.8M 2024-03-21T08:42:23.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:23 smithi171 ceph-mon[35388]: Unable to set osd_memory_target on smithi171 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T08:42:23.842 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:42:24.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:24 smithi084 ceph-mon[35477]: pgmap v329: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:24.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:24 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:24.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:24 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:24.887 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:24 smithi171 ceph-mon[35388]: pgmap v329: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:24.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:24 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:24.888 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:24 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:24.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:24 smithi003 ceph-mon[31675]: pgmap v329: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:24.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:24 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:24.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:24 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:25.578 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.579 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.579 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.579 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.579 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.579 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.579 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.579 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.580 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:42:25.580 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:42:25.580 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:25 smithi084 ceph-mon[35477]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:25.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:42:25.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:42:25.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:25 smithi171 ceph-mon[35388]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:42:25.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:42:25.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 172.21.15.3:0/4230393274' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:42:25.928 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:25 smithi003 ceph-mon[31675]: from='mgr.15654 ' entity='mgr.x' 2024-03-21T08:42:26.186 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-03-21T08:42:26.186 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:26.186 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 25378, 2024-03-21T08:42:26.186 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 30953, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 19469300, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 86677944, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 14, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 4.041658776, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 2.134568168, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 1.987786898, 2024-03-21T08:42:26.187 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.225579674, 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 30952, 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3.755541041, 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000121334 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 44749, 2024-03-21T08:42:26.188 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.010846791, 2024-03-21T08:42:26.189 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000242 2024-03-21T08:42:26.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.189 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1703872, 2024-03-21T08:42:26.189 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 438128 2024-03-21T08:42:26.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.189 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:26.189 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 38812, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 51799, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 12581233, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 44279252, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 71, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 8, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 4.742224554, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 1.998472871, 2024-03-21T08:42:26.190 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 2.729526890, 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.313064668, 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 51796, 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 8.921449340, 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000172242 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:26.191 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 72666, 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.022634790, 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000311 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 428256, 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 250928 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:26.192 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 29180, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 34243, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 18537328, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 92196034, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 47, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 6, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 4.411318874, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 2.333166246, 2024-03-21T08:42:26.193 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 2.198616303, 2024-03-21T08:42:26.194 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.249975952, 2024-03-21T08:42:26.194 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:26.194 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 34242, 2024-03-21T08:42:26.194 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 7.646866754, 2024-03-21T08:42:26.194 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000223318 2024-03-21T08:42:26.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.194 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:26.195 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 48686, 2024-03-21T08:42:26.195 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.012192490, 2024-03-21T08:42:26.195 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000250 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-03-21T08:42:26.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.197 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-03-21T08:42:26.197 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-03-21T08:42:26.197 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-03-21T08:42:26.197 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 14207, 2024-03-21T08:42:26.197 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 19.742297386, 2024-03-21T08:42:26.197 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001389617 2024-03-21T08:42:26.197 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2207, 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.299702080, 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000135796 2024-03-21T08:42:26.198 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 21662, 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 16654, 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 16654, 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 102.891636927, 2024-03-21T08:42:26.199 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.006178193 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 5008, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 15, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 2, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 692, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 12, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-03-21T08:42:26.200 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 138, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 15, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 66, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 57, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 116, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 5241, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 103, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 104, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 21, 2024-03-21T08:42:26.201 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 33691, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 23293, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 4987, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 6, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 5, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-03-21T08:42:26.202 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 121, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 4736, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 20, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 618, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 1416, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 3452, 2024-03-21T08:42:26.203 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 26526745, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 1416, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 63, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 5, 2024-03-21T08:42:26.204 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 58, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 1353, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 5158, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 5318, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 8, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T08:42:26.205 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 9702, 2024-03-21T08:42:26.206 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 4149, 2024-03-21T08:42:26.206 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 114, 2024-03-21T08:42:26.206 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 6856 2024-03-21T08:42:26.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.206 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-03-21T08:42:26.206 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-03-21T08:42:26.206 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 12, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 137, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 3, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 967, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 964, 2024-03-21T08:42:26.207 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 5, 2024-03-21T08:42:26.208 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 145, 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 12, 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 16258, 2024-03-21T08:42:26.209 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 16178, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 16178, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 80, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 18, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 48, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 47, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 47, 2024-03-21T08:42:26.210 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 1, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 1, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 1, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 68743249, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 69213030, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 0, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 6316, 2024-03-21T08:42:26.211 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 38.140827377, 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.006038763 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 0 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 139, 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 5779, 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 5640, 2024-03-21T08:42:26.212 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 81, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 323, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 242, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 138, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 6513, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 6375, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 104, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 4842, 2024-03-21T08:42:26.213 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 4738, 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 80624, 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 231804 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 28323, 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 4583, 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 21662, 2024-03-21T08:42:26.214 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 154, 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 9128, 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4361, 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 13.896725981, 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003186591 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1148, 2024-03-21T08:42:26.215 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.710212573, 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000618652 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.216 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 9823, 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 82.531919936, 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.008401905 2024-03-21T08:42:26.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.218 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-03-21T08:42:26.219 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.221 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-03-21T08:42:26.221 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 11, 2024-03-21T08:42:26.222 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.004511963, 2024-03-21T08:42:26.222 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000410178 2024-03-21T08:42:26.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.222 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-03-21T08:42:26.222 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 81, 2024-03-21T08:42:26.222 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.418352197, 2024-03-21T08:42:26.222 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.005164841 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-03-21T08:42:26.223 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 11, 2024-03-21T08:42:26.224 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.174220590, 2024-03-21T08:42:26.224 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.015838235 2024-03-21T08:42:26.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.224 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-03-21T08:42:26.224 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 102, 2024-03-21T08:42:26.224 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.130009645, 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001274604 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 77, 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1.037783647, 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.013477709 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.225 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-03-21T08:42:26.226 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 74, 2024-03-21T08:42:26.226 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.291932127, 2024-03-21T08:42:26.226 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003945028 2024-03-21T08:42:26.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.226 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-03-21T08:42:26.226 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.226 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-03-21T08:42:26.227 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 21, 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.267442764, 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.012735369 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5, 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.047752153, 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.009550430 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.228 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 8, 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.024040034, 2024-03-21T08:42:26.229 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003005004 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 8, 2024-03-21T08:42:26.230 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.009074838, 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001134354 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.231 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 8, 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.038737862, 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004842232 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 7, 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.011604121, 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001657731 2024-03-21T08:42:26.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 909, 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3.297316496, 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003627410 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-03-21T08:42:26.233 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 2, 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 5, 2024-03-21T08:42:26.234 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 3, 2024-03-21T08:42:26.235 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 2, 2024-03-21T08:42:26.235 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-03-21T08:42:26.235 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 269, 2024-03-21T08:42:26.235 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 134, 2024-03-21T08:42:26.235 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 642, 2024-03-21T08:42:26.235 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-03-21T08:42:26.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 600, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 600, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:26.236 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:26.237 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:26.238 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-03-21T08:42:26.239 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 315760, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 126, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-03-21T08:42:26.240 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 10088, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 194, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:26.241 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 616146, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 4231, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-03-21T08:42:26.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 17852, 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 71992088, 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 31, 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 17852, 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-03-21T08:42:26.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 17852, 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1075.509734030, 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.060245895 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 17852, 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 28630 2024-03-21T08:42:26.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 17852, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 1629, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 16223, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 519, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 6041, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 191, 2024-03-21T08:42:26.245 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 6502, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 231, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 6, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 2465, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-03-21T08:42:26.246 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1435, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 9144, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-03-21T08:42:26.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 8, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-03-21T08:42:26.248 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-03-21T08:42:26.249 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 107, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 20, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 12, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 12, 2024-03-21T08:42:26.250 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 1790, 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 34, 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 264 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 1, 2024-03-21T08:42:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-03-21T08:42:26.252 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 17, 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 9, 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 1917, 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 964, 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:26.253 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 93366, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 42120376, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 93366, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:26.254 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 93366, 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 42120376, 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.255 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-03-21T08:42:26.256 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:26.256 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-03-21T08:42:26.256 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:26.256 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-03-21T08:42:26.256 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-03-21T08:42:26.256 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 17852, 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 72101116, 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 17852, 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 72101116, 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.257 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:26.258 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 17852, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 17852, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 17852, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 17852, 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:26.260 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 964, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 109276, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 964, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 186, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 109276, 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:26.261 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x5631715f2460": { 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-03-21T08:42:26.262 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 16258, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 65018726, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 16258, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 6316, 2024-03-21T08:42:26.263 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 65018726, 2024-03-21T08:42:26.264 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:26.264 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:26.264 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:26.264 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:26.264 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.264 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:26.264 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-03-21T08:42:26.740 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:26 smithi084 ceph-mon[35477]: pgmap v330: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:26.740 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:26 smithi084 ceph-mon[35477]: Adjusting osd_memory_target on smithi084 to 176.7M 2024-03-21T08:42:26.740 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:26 smithi084 ceph-mon[35477]: Unable to set osd_memory_target on smithi084 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-21T08:42:26.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:26 smithi171 ceph-mon[35388]: pgmap v330: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:26.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:26 smithi171 ceph-mon[35388]: Adjusting osd_memory_target on smithi084 to 176.7M 2024-03-21T08:42:26.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:26 smithi171 ceph-mon[35388]: Unable to set osd_memory_target on smithi084 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-21T08:42:26.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:26 smithi003 ceph-mon[31675]: pgmap v330: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:26.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:26 smithi003 ceph-mon[31675]: Adjusting osd_memory_target on smithi084 to 176.7M 2024-03-21T08:42:26.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:26 smithi003 ceph-mon[31675]: Unable to set osd_memory_target on smithi084 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-21T08:42:27.187 INFO:tasks.check_counter:Daemon mds.h mds.exported=121 2024-03-21T08:42:27.187 INFO:tasks.check_counter:Daemon mds.h mds.imported=20 2024-03-21T08:42:27.188 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-21T08:42:27.188 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=11 2024-03-21T08:42:27.188 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=5 2024-03-21T08:42:27.188 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=12 2024-03-21T08:42:27.188 DEBUG:tasks.check_counter:Getting stats from k 2024-03-21T08:42:27.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-21T08:42:27.672 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:42:28.582 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:28 smithi084 ceph-mon[35477]: pgmap v331: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:28.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:28 smithi003 ceph-mon[31675]: pgmap v331: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:28.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:28 smithi171 ceph-mon[35388]: pgmap v331: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-21T08:42:30.035 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-03-21T08:42:30.035 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:30.035 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 27348, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 22001, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 61313737, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 46684286, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 11, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 3.838645041, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 2.054138143, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 2.218513172, 2024-03-21T08:42:30.036 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.086521601, 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 22003, 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1.412840924, 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000064211 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 44518, 2024-03-21T08:42:30.037 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.011359974, 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000255 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 282832 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2826, 2024-03-21T08:42:30.038 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2810, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 2347845, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 36756673, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 18, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 7, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.455858274, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.200440391, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.143405716, 2024-03-21T08:42:30.039 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.109825373, 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2806, 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.371542127, 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000132409 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 24, 2024-03-21T08:42:30.040 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000004945, 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000206 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1690112, 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 159488 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 13885, 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 20603, 2024-03-21T08:42:30.041 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 4759853, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 16142822, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 15, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 6, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 1.883893669, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.831682070, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 1.061301999, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.147910126, 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:30.042 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 20600, 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1.943616241, 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000094350 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 27050, 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.009007618, 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000332 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.043 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 5280, 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 115488 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-03-21T08:42:30.044 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5474, 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 8.700315504, 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001589389 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-03-21T08:42:30.045 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2455, 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.300484523, 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000122396 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 8491, 2024-03-21T08:42:30.046 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 8479, 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 8479, 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 15.397902120, 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001816004 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 12, 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 1, 2024-03-21T08:42:30.047 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 83, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 9, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 323, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 2, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 1, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 320, 2024-03-21T08:42:30.048 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 320, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 3417, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 242, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 242, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 13, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 21629, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 7412, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 12, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 18, 2024-03-21T08:42:30.049 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 1, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 101, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 3258, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 46, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 1906, 2024-03-21T08:42:30.050 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 126, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 3452, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 26526745, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 126, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T08:42:30.051 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 6, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 6, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 120, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 44, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 2360, 2024-03-21T08:42:30.052 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 2164, 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 1745, 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 121, 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 3500 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-03-21T08:42:30.053 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 17, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 95, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-03-21T08:42:30.054 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 1136, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 1136, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-03-21T08:42:30.055 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 101, 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 9, 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-03-21T08:42:30.056 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 104409, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 104409, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 115, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-03-21T08:42:30.057 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 115, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 10, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 73102574, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 0, 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-03-21T08:42:30.058 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2598, 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 16.391651037, 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.006309334 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 0 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 309, 2024-03-21T08:42:30.059 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 4286, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 3977, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 143, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 158, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 15, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 323, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 4915, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 4592, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 242, 2024-03-21T08:42:30.060 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 2844, 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 2602, 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 73060, 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 223612 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 15576, 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 0, 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 8491, 2024-03-21T08:42:30.061 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 117, 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 2346, 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1220, 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 5.864670216, 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004807106 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-03-21T08:42:30.062 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 14, 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.007592201, 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000542300 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.063 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 6085, 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3.132468661, 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000514785 2024-03-21T08:42:30.064 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.065 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-03-21T08:42:30.066 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.067 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.068 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-03-21T08:42:30.068 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.068 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.069 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.069 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-03-21T08:42:30.069 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4, 2024-03-21T08:42:30.069 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.019242399, 2024-03-21T08:42:30.069 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004810599 2024-03-21T08:42:30.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.070 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1, 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000467932, 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000467932 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.071 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 17, 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.032598375, 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001917551 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2, 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.004447425, 2024-03-21T08:42:30.072 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.002223712 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-03-21T08:42:30.073 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.075 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-03-21T08:42:30.076 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.078 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-03-21T08:42:30.079 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1136, 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 6.336414911, 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.005577830 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.080 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 1, 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 3, 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 2, 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 1, 2024-03-21T08:42:30.081 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 2852, 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 2852, 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 660, 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:30.082 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:30.083 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:30.084 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:30.085 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 291229, 2024-03-21T08:42:30.086 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 109, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:30.087 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 10088, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 194, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 1213594, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 9345, 2024-03-21T08:42:30.088 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 5946, 2024-03-21T08:42:30.089 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 69918117, 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 28, 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 5946, 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5946, 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 39.536306144, 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.006649227 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-03-21T08:42:30.090 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5946, 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 6377 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 5946, 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 364, 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 5582, 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-03-21T08:42:30.091 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 0, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 10, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 237, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 2835, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 146, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-03-21T08:42:30.092 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 2309, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 126, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 20, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-03-21T08:42:30.093 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-03-21T08:42:30.094 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-03-21T08:42:30.095 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 107, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-03-21T08:42:30.096 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 13, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 12, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 12, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 521, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 2, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 171 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.097 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 1, 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-03-21T08:42:30.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 16, 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 8, 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 2272, 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 1136, 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-03-21T08:42:30.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 44055, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 64437250, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 44055, 2024-03-21T08:42:30.100 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 44055, 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 64437250, 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.101 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:30.102 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 5946, 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 70056653, 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 5946, 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 70056653, 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.103 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-03-21T08:42:30.104 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 5946, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 5946, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 5946, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 5946, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.105 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 1136, 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 138552, 2024-03-21T08:42:30.106 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 1136, 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 237, 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 138552, 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.107 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x55d2589743c0": { 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 104409, 2024-03-21T08:42:30.108 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 68908270, 2024-03-21T08:42:30.109 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:30.109 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 104409, 2024-03-21T08:42:30.109 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-03-21T08:42:30.109 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-03-21T08:42:30.109 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2598, 2024-03-21T08:42:30.109 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 68908270, 2024-03-21T08:42:30.109 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-03-21T08:42:30.110 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-03-21T08:42:30.110 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-03-21T08:42:30.110 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-03-21T08:42:30.110 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.110 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-03-21T08:42:30.110 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-03-21T08:42:30.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:30 smithi084 ceph-mon[35477]: pgmap v332: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:42:30.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:30 smithi171 ceph-mon[35388]: pgmap v332: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:42:30.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:30 smithi003 ceph-mon[31675]: pgmap v332: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-21T08:42:31.107 INFO:tasks.check_counter:Daemon mds.k mds.exported=101 2024-03-21T08:42:31.107 INFO:tasks.check_counter:Daemon mds.k mds.imported=46 2024-03-21T08:42:31.108 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-03-21T08:42:31.108 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-21T08:42:31.108 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T08:42:31.108 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=9 2024-03-21T08:42:31.108 DEBUG:tasks.check_counter:Getting stats from c 2024-03-21T08:42:31.108 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-21T08:42:31.577 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:42:32.771 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:32 smithi171 ceph-mon[35388]: pgmap v333: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:32.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:32 smithi084 ceph-mon[35477]: pgmap v333: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:32.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:32 smithi003 ceph-mon[31675]: pgmap v333: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:34.096 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-21T08:42:34.097 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:34.097 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2002, 2024-03-21T08:42:34.098 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2004, 2024-03-21T08:42:34.098 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 34126797, 2024-03-21T08:42:34.098 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 636643, 2024-03-21T08:42:34.099 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-03-21T08:42:34.099 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-21T08:42:34.099 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.496483639, 2024-03-21T08:42:34.100 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.141506714, 2024-03-21T08:42:34.100 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.192826816, 2024-03-21T08:42:34.100 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.121713507, 2024-03-21T08:42:34.101 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:34.101 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2002, 2024-03-21T08:42:34.101 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.176292301, 2024-03-21T08:42:34.101 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000088058 2024-03-21T08:42:34.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.102 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:34.102 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.103 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.103 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.104 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:34.104 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:34.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.104 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:34.105 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 763, 2024-03-21T08:42:34.105 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 736, 2024-03-21T08:42:34.105 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 25552790, 2024-03-21T08:42:34.105 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 296878, 2024-03-21T08:42:34.106 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:34.106 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-21T08:42:34.106 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.276969092, 2024-03-21T08:42:34.106 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.069380555, 2024-03-21T08:42:34.107 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.181974860, 2024-03-21T08:42:34.107 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.038859476, 2024-03-21T08:42:34.107 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:34.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 735, 2024-03-21T08:42:34.108 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.077188720, 2024-03-21T08:42:34.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000105018 2024-03-21T08:42:34.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.108 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:34.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.109 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.109 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1677456, 2024-03-21T08:42:34.110 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 141632 2024-03-21T08:42:34.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.110 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:34.110 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 51, 2024-03-21T08:42:34.111 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 223, 2024-03-21T08:42:34.111 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 8403268, 2024-03-21T08:42:34.111 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 410134, 2024-03-21T08:42:34.112 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-03-21T08:42:34.112 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-03-21T08:42:34.112 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.056660457, 2024-03-21T08:42:34.113 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.018979623, 2024-03-21T08:42:34.113 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.028755828, 2024-03-21T08:42:34.114 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.002712156, 2024-03-21T08:42:34.114 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:34.114 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 221, 2024-03-21T08:42:34.114 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.046020330, 2024-03-21T08:42:34.114 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000208236 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T08:42:34.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 395824 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-21T08:42:34.116 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4242, 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.229113494, 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000054010 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-21T08:42:34.117 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-03-21T08:42:34.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-03-21T08:42:34.119 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 8139, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 1353, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 6786, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 6206, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-03-21T08:42:34.120 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 13, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-03-21T08:42:34.121 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-03-21T08:42:34.122 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-03-21T08:42:34.123 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T08:42:34.124 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-03-21T08:42:34.125 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-03-21T08:42:34.126 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-03-21T08:42:34.127 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-03-21T08:42:34.128 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 96858, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 107, 2024-03-21T08:42:34.129 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 9, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 69886182, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 69886121, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.130 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 96858 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 5933, 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 7925, 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 1992, 2024-03-21T08:42:34.131 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 229, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 280, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 51, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 8139, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 9164, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 1025, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-03-21T08:42:34.132 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 72660, 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215420 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-03-21T08:42:34.133 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-03-21T08:42:34.136 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-03-21T08:42:34.137 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.140 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-03-21T08:42:34.140 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.140 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.141 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-03-21T08:42:34.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.141 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.143 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-03-21T08:42:34.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.145 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.146 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.147 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.148 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-03-21T08:42:34.149 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.150 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.151 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 3, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 1, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 2184, 2024-03-21T08:42:34.152 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:34.153 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:34.154 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:34.155 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:34.156 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-21T08:42:34.157 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:34.157 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:34.157 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:34.157 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:34.157 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 231369, 2024-03-21T08:42:34.157 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 75, 2024-03-21T08:42:34.158 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:34.158 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:34.158 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-21T08:42:34.158 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-21T08:42:34.158 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:34.158 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:34.159 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:34.159 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:34.159 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 10088, 2024-03-21T08:42:34.159 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 194, 2024-03-21T08:42:34.159 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:34.159 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:34.159 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 18179337, 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 55600, 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-21T08:42:34.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.161 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-03-21T08:42:34.161 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-03-21T08:42:34.161 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-03-21T08:42:34.161 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 2535, 2024-03-21T08:42:34.161 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-03-21T08:42:34.161 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 2, 2024-03-21T08:42:34.161 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 2535, 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2535, 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3.376933475, 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001332123 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-03-21T08:42:34.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2535, 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2536 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 2535, 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 2535, 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-03-21T08:42:34.163 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 1518, 2024-03-21T08:42:34.164 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-03-21T08:42:34.164 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1015, 2024-03-21T08:42:34.164 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-03-21T08:42:34.164 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-03-21T08:42:34.164 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-03-21T08:42:34.164 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-03-21T08:42:34.164 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-03-21T08:42:34.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-03-21T08:42:34.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-03-21T08:42:34.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-03-21T08:42:34.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-03-21T08:42:34.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-03-21T08:42:34.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-03-21T08:42:34.166 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-03-21T08:42:34.166 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-03-21T08:42:34.166 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-03-21T08:42:34.166 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-03-21T08:42:34.166 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-03-21T08:42:34.166 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-03-21T08:42:34.167 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-03-21T08:42:34.168 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-03-21T08:42:34.169 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-03-21T08:42:34.169 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-03-21T08:42:34.169 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-03-21T08:42:34.169 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-03-21T08:42:34.169 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-03-21T08:42:34.169 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 107, 2024-03-21T08:42:34.169 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-03-21T08:42:34.170 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 13, 2024-03-21T08:42:34.170 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 10, 2024-03-21T08:42:34.170 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 10, 2024-03-21T08:42:34.170 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-03-21T08:42:34.170 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-03-21T08:42:34.170 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-03-21T08:42:34.170 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-03-21T08:42:34.171 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-03-21T08:42:34.171 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.171 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-03-21T08:42:34.171 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-03-21T08:42:34.171 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-03-21T08:42:34.171 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-03-21T08:42:34.171 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-03-21T08:42:34.172 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-03-21T08:42:34.173 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-03-21T08:42:34.173 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-03-21T08:42:34.173 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-03-21T08:42:34.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.173 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:34.173 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:34.173 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 2812, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 67812836, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 2812, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:34.174 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2812, 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 67812836, 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-03-21T08:42:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2535, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 65691883, 2024-03-21T08:42:34.176 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2535, 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 65691883, 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.177 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-03-21T08:42:34.178 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:34.178 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-03-21T08:42:34.178 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:34.178 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:34.178 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:34.178 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2535, 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 2535, 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2535, 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 2535, 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.179 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.180 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.180 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.180 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-03-21T08:42:34.180 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:34.180 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-21T08:42:34.181 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:34.181 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:34.181 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:34.181 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:34.181 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:34.181 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:34.181 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x55da81974780": { 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:34.183 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:34.184 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.185 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:34.185 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-21T08:42:34.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:34 smithi171 ceph-mon[35388]: pgmap v334: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:34.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:34 smithi084 ceph-mon[35477]: pgmap v334: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:34.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:34 smithi003 ceph-mon[31675]: pgmap v334: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:35.104 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-21T08:42:35.104 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-21T08:42:35.104 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-21T08:42:35.105 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-21T08:42:35.105 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T08:42:35.105 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-03-21T08:42:35.105 DEBUG:tasks.check_counter:Getting stats from f 2024-03-21T08:42:35.105 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-21T08:42:35.544 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:42:36.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:36 smithi084 ceph-mon[35477]: pgmap v335: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:36.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:36 smithi171 ceph-mon[35388]: pgmap v335: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:36.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:36 smithi003 ceph-mon[31675]: pgmap v335: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:37.851 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-21T08:42:37.851 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:37.851 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 797, 2024-03-21T08:42:37.851 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 799, 2024-03-21T08:42:37.851 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 21174557, 2024-03-21T08:42:37.852 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 253510, 2024-03-21T08:42:37.852 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-03-21T08:42:37.852 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:37.852 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.226656884, 2024-03-21T08:42:37.852 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.053872613, 2024-03-21T08:42:37.853 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.102329667, 2024-03-21T08:42:37.853 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.050282043, 2024-03-21T08:42:37.853 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:37.853 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 797, 2024-03-21T08:42:37.853 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.075938531, 2024-03-21T08:42:37.853 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000095280 2024-03-21T08:42:37.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.854 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:37.854 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.854 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.854 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.855 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:37.855 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:37.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.855 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:37.855 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1018, 2024-03-21T08:42:37.855 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 991, 2024-03-21T08:42:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 33026978, 2024-03-21T08:42:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 378523, 2024-03-21T08:42:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-21T08:42:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.353115490, 2024-03-21T08:42:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.101102642, 2024-03-21T08:42:37.856 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.207270963, 2024-03-21T08:42:37.857 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.056455772, 2024-03-21T08:42:37.857 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:37.857 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 990, 2024-03-21T08:42:37.857 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.095979537, 2024-03-21T08:42:37.857 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000096949 2024-03-21T08:42:37.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.858 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:37.858 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.858 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.858 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.859 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1678000, 2024-03-21T08:42:37.859 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 142880 2024-03-21T08:42:37.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.859 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:37.859 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 950, 2024-03-21T08:42:37.860 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1124, 2024-03-21T08:42:37.860 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 17086000, 2024-03-21T08:42:37.860 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 697968, 2024-03-21T08:42:37.860 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-03-21T08:42:37.861 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-03-21T08:42:37.861 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.288233589, 2024-03-21T08:42:37.861 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.086513350, 2024-03-21T08:42:37.861 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.116038758, 2024-03-21T08:42:37.861 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.058412432, 2024-03-21T08:42:37.861 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:37.861 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1122, 2024-03-21T08:42:37.862 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.133391198, 2024-03-21T08:42:37.862 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000118886 2024-03-21T08:42:37.862 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.862 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:37.862 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.863 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.863 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.863 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T08:42:37.863 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 397776 2024-03-21T08:42:37.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.864 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-21T08:42:37.864 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-03-21T08:42:37.864 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-21T08:42:37.864 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.864 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-03-21T08:42:37.864 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4181, 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.232104636, 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000055514 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-03-21T08:42:37.865 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-21T08:42:37.866 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-21T08:42:37.866 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.866 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.866 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.866 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.867 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.867 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-03-21T08:42:37.867 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-03-21T08:42:37.867 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-03-21T08:42:37.867 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-03-21T08:42:37.867 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.867 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.868 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.868 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.868 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-03-21T08:42:37.868 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-03-21T08:42:37.868 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-03-21T08:42:37.868 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-03-21T08:42:37.868 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-03-21T08:42:37.869 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-03-21T08:42:37.869 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-03-21T08:42:37.869 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 1468, 2024-03-21T08:42:37.869 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 30, 2024-03-21T08:42:37.869 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 1438, 2024-03-21T08:42:37.869 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-03-21T08:42:37.870 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 1424, 2024-03-21T08:42:37.870 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-03-21T08:42:37.870 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-03-21T08:42:37.870 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-03-21T08:42:37.870 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 10, 2024-03-21T08:42:37.870 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-03-21T08:42:37.870 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-03-21T08:42:37.871 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-03-21T08:42:37.871 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-03-21T08:42:37.871 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-03-21T08:42:37.871 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-03-21T08:42:37.871 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-03-21T08:42:37.871 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-03-21T08:42:37.871 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-03-21T08:42:37.872 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-03-21T08:42:37.872 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-03-21T08:42:37.872 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-03-21T08:42:37.872 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-03-21T08:42:37.872 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-03-21T08:42:37.872 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-03-21T08:42:37.872 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-03-21T08:42:37.873 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-03-21T08:42:37.873 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-03-21T08:42:37.873 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-03-21T08:42:37.873 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T08:42:37.873 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-03-21T08:42:37.873 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T08:42:37.874 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-03-21T08:42:37.874 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-03-21T08:42:37.874 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T08:42:37.874 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T08:42:37.874 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-03-21T08:42:37.874 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-03-21T08:42:37.874 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T08:42:37.875 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-03-21T08:42:37.875 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T08:42:37.875 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T08:42:37.875 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T08:42:37.875 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-03-21T08:42:37.875 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-03-21T08:42:37.875 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-03-21T08:42:37.876 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-03-21T08:42:37.876 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.876 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-03-21T08:42:37.876 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-03-21T08:42:37.877 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-03-21T08:42:37.877 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-03-21T08:42:37.877 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-03-21T08:42:37.877 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-03-21T08:42:37.877 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-03-21T08:42:37.877 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-03-21T08:42:37.878 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-03-21T08:42:37.878 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-03-21T08:42:37.878 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-03-21T08:42:37.878 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-03-21T08:42:37.878 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-03-21T08:42:37.878 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-03-21T08:42:37.879 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-03-21T08:42:37.879 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-03-21T08:42:37.879 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-03-21T08:42:37.879 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-03-21T08:42:37.879 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-03-21T08:42:37.880 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-03-21T08:42:37.881 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-03-21T08:42:37.881 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-03-21T08:42:37.881 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-03-21T08:42:37.881 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 104409, 2024-03-21T08:42:37.882 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-03-21T08:42:37.882 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-03-21T08:42:37.882 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-03-21T08:42:37.882 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-03-21T08:42:37.882 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-03-21T08:42:37.882 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 115, 2024-03-21T08:42:37.882 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 10, 2024-03-21T08:42:37.883 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-03-21T08:42:37.883 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-03-21T08:42:37.883 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-03-21T08:42:37.883 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 73102574, 2024-03-21T08:42:37.883 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 73093241, 2024-03-21T08:42:37.884 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-03-21T08:42:37.884 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.884 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.884 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.884 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 104409 2024-03-21T08:42:37.885 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.885 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-03-21T08:42:37.885 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 312, 2024-03-21T08:42:37.885 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 4281, 2024-03-21T08:42:37.885 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 3969, 2024-03-21T08:42:37.886 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 39, 2024-03-21T08:42:37.886 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 155, 2024-03-21T08:42:37.886 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 116, 2024-03-21T08:42:37.886 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 1468, 2024-03-21T08:42:37.886 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 4822, 2024-03-21T08:42:37.887 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 3354, 2024-03-21T08:42:37.887 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-03-21T08:42:37.887 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-03-21T08:42:37.887 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-03-21T08:42:37.887 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 61104, 2024-03-21T08:42:37.888 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215420 2024-03-21T08:42:37.888 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.888 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-03-21T08:42:37.888 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-03-21T08:42:37.888 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-03-21T08:42:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-03-21T08:42:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-03-21T08:42:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-03-21T08:42:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-03-21T08:42:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.890 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-03-21T08:42:37.890 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.890 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.890 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.890 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.891 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-03-21T08:42:37.891 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.891 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.891 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.891 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-03-21T08:42:37.891 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.892 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.892 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.892 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.892 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-03-21T08:42:37.892 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.892 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.893 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.893 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.893 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-03-21T08:42:37.893 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.893 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.894 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-03-21T08:42:37.895 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.895 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.895 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.895 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.895 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-03-21T08:42:37.895 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-03-21T08:42:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.896 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.897 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-03-21T08:42:37.897 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.897 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.897 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.897 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.897 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-03-21T08:42:37.898 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.899 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.899 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.899 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.899 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-03-21T08:42:37.899 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.899 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.900 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-03-21T08:42:37.901 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.901 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.901 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.901 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.901 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-03-21T08:42:37.901 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.902 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.902 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.902 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.902 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-03-21T08:42:37.902 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.902 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.902 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.903 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-03-21T08:42:37.903 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.903 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.903 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.903 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.903 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-03-21T08:42:37.904 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.904 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.904 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.904 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.904 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-03-21T08:42:37.904 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.904 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.905 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.905 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-03-21T08:42:37.905 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.905 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.905 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.905 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.906 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-03-21T08:42:37.906 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.906 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.906 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.906 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.906 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-03-21T08:42:37.907 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.907 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.907 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.907 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.907 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-03-21T08:42:37.907 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.907 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.908 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-03-21T08:42:37.909 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.909 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.909 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.909 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.909 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-03-21T08:42:37.909 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.910 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.910 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.910 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.910 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-03-21T08:42:37.910 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.910 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.911 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.911 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-03-21T08:42:37.911 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.911 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.911 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.911 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.912 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-03-21T08:42:37.912 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-03-21T08:42:37.912 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-03-21T08:42:37.912 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.912 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.913 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.913 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.913 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.913 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-03-21T08:42:37.913 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 1, 2024-03-21T08:42:37.913 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 3, 2024-03-21T08:42:37.913 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 2, 2024-03-21T08:42:37.914 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 3, 2024-03-21T08:42:37.914 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-03-21T08:42:37.914 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-03-21T08:42:37.914 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-03-21T08:42:37.914 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 5275, 2024-03-21T08:42:37.914 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-03-21T08:42:37.915 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.915 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-21T08:42:37.915 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:37.915 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:37.915 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:37.915 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:37.915 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:37.916 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:37.916 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:37.916 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:37.916 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:37.916 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:37.917 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:37.917 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:37.917 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:37.917 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:37.917 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:37.917 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:37.918 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:37.918 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:37.918 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:37.918 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:37.918 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:37.919 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:37.920 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:37.920 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:37.920 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:37.920 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-21T08:42:37.920 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 215201, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 67, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:37.921 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-21T08:42:37.922 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-21T08:42:37.922 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:37.922 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:37.922 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:37.922 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:37.923 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 10088, 2024-03-21T08:42:37.923 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 194, 2024-03-21T08:42:37.923 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:37.923 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:37.923 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:37.924 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-21T08:42:37.924 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 1757946, 2024-03-21T08:42:37.924 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 9408, 2024-03-21T08:42:37.924 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:37.924 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-21T08:42:37.924 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:37.925 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-21T08:42:37.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.925 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-03-21T08:42:37.925 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-03-21T08:42:37.925 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-03-21T08:42:37.925 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 2482, 2024-03-21T08:42:37.925 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-03-21T08:42:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 2, 2024-03-21T08:42:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 2482, 2024-03-21T08:42:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-03-21T08:42:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2482, 2024-03-21T08:42:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3.638928355, 2024-03-21T08:42:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001466127 2024-03-21T08:42:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2482, 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2483 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 2482, 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 2482, 2024-03-21T08:42:37.927 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 1475, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1005, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-03-21T08:42:37.928 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-03-21T08:42:37.929 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-03-21T08:42:37.929 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-03-21T08:42:37.929 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-03-21T08:42:37.929 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-03-21T08:42:37.929 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-03-21T08:42:37.929 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-03-21T08:42:37.929 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-03-21T08:42:37.930 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-03-21T08:42:37.930 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-03-21T08:42:37.930 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-03-21T08:42:37.930 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-03-21T08:42:37.930 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-03-21T08:42:37.930 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-03-21T08:42:37.930 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-03-21T08:42:37.931 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-03-21T08:42:37.932 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-03-21T08:42:37.933 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-03-21T08:42:37.933 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-03-21T08:42:37.933 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-03-21T08:42:37.933 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-03-21T08:42:37.933 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-03-21T08:42:37.933 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 107, 2024-03-21T08:42:37.933 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 13, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 8, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 8, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-03-21T08:42:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-03-21T08:42:37.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.935 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-03-21T08:42:37.935 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-03-21T08:42:37.935 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-03-21T08:42:37.935 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-03-21T08:42:37.935 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-03-21T08:42:37.935 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-03-21T08:42:37.936 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-03-21T08:42:37.937 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-03-21T08:42:37.937 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-03-21T08:42:37.937 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-03-21T08:42:37.937 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.937 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:37.937 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:37.937 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-21T08:42:37.938 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:37.938 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 2761, 2024-03-21T08:42:37.938 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 71022002, 2024-03-21T08:42:37.938 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:37.938 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 2761, 2024-03-21T08:42:37.938 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:37.938 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2761, 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 71022002, 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.939 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.940 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-03-21T08:42:37.940 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:37.940 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-21T08:42:37.940 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:37.940 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:37.940 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:37.940 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:37.941 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:37.941 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2482, 2024-03-21T08:42:37.941 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 68908275, 2024-03-21T08:42:37.941 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2482, 2024-03-21T08:42:37.941 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 68908275, 2024-03-21T08:42:37.941 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:37.942 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.942 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.942 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.942 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.942 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-03-21T08:42:37.943 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:37.943 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-03-21T08:42:37.943 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:37.943 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:37.943 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:37.943 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:37.943 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:37.944 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2482, 2024-03-21T08:42:37.944 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 2482, 2024-03-21T08:42:37.944 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2482, 2024-03-21T08:42:37.944 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 2482, 2024-03-21T08:42:37.944 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:37.944 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.944 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.945 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.945 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.945 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.945 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-03-21T08:42:37.945 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:37.945 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-21T08:42:37.945 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:37.946 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:37.946 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:37.946 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:37.946 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:37.946 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:37.947 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:37.947 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-21T08:42:37.947 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-21T08:42:37.947 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:37.947 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.947 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.947 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.948 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.948 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:37.948 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x55a3e94aa780": { 2024-03-21T08:42:37.948 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:37.948 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-21T08:42:37.948 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:37.949 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:37.949 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:37.949 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:37.949 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:37.949 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:37.949 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:37.950 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-21T08:42:37.950 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-21T08:42:37.950 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:37.950 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:37.950 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:37.950 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:37.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.951 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:37.951 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-21T08:42:38.813 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:38 smithi171 ceph-mon[35388]: pgmap v336: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:38.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:38 smithi084 ceph-mon[35477]: pgmap v336: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:38.893 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-21T08:42:38.893 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-21T08:42:38.893 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-21T08:42:38.893 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-21T08:42:38.893 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T08:42:38.893 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-03-21T08:42:38.894 DEBUG:tasks.check_counter:Getting stats from i 2024-03-21T08:42:38.894 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-21T08:42:38.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:38 smithi003 ceph-mon[31675]: pgmap v336: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:39.341 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:42:40.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:40 smithi171 ceph-mon[35388]: pgmap v337: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:40.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:40 smithi084 ceph-mon[35477]: pgmap v337: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:40.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:40 smithi003 ceph-mon[31675]: pgmap v337: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:41.554 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-21T08:42:41.555 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:41.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1412, 2024-03-21T08:42:41.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1414, 2024-03-21T08:42:41.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 25593009, 2024-03-21T08:42:41.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 449026, 2024-03-21T08:42:41.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-03-21T08:42:41.555 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.375283814, 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.106664788, 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.148464201, 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.086784818, 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1412, 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.139994115, 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000099145 2024-03-21T08:42:41.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.557 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 500, 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 672, 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 21099802, 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 556182, 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.197806906, 2024-03-21T08:42:41.558 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.053426508, 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.094107495, 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.034926906, 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 670, 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.081574671, 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000121753 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 399024 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1095, 2024-03-21T08:42:41.560 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1068, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 20766825, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 402077, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.356100858, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.115255537, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.187128432, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.060143932, 2024-03-21T08:42:41.561 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1067, 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.106768113, 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000100063 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.562 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1677776, 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 141952 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-03-21T08:42:41.563 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4544, 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.346197446, 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000076187 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-03-21T08:42:41.564 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-03-21T08:42:41.565 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-03-21T08:42:41.566 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 1204, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 802, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 402, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 47, 2024-03-21T08:42:41.567 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 21, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-03-21T08:42:41.568 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-03-21T08:42:41.569 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 3452, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 26526745, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 6, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T08:42:41.570 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T08:42:41.571 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-03-21T08:42:41.572 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-03-21T08:42:41.573 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T08:42:41.574 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-03-21T08:42:41.575 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-03-21T08:42:41.576 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-03-21T08:42:41.576 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-03-21T08:42:41.576 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 16258, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 2, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 29, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-03-21T08:42:41.577 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 68743249, 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 69213030, 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 69212894, 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.578 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 16258 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 177, 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 5686, 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 5509, 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 57, 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 307, 2024-03-21T08:42:41.579 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 250, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 1204, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 6219, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 5015, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 60684, 2024-03-21T08:42:41.580 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215420 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-03-21T08:42:41.581 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.583 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-03-21T08:42:41.584 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.585 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.586 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.587 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.588 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.588 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.588 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-03-21T08:42:41.588 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.589 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.589 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.589 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.589 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-03-21T08:42:41.589 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.589 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.590 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.591 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.592 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.593 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.594 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.595 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.596 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.597 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.598 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-03-21T08:42:41.599 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 5, 2024-03-21T08:42:41.600 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 3, 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 3, 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 2965, 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-21T08:42:41.601 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:41.602 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:41.603 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:41.604 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 223282, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 71, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:41.606 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 10088, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 194, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:41.607 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 1355462, 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 6839, 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-03-21T08:42:41.608 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 2724, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 26, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 4, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 2724, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2724, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3.392933227, 2024-03-21T08:42:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001245570 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2724, 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2725 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 2724, 2024-03-21T08:42:41.610 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 2724, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 1616, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1105, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-03-21T08:42:41.611 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 2, 2024-03-21T08:42:41.612 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-03-21T08:42:41.613 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-03-21T08:42:41.614 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-03-21T08:42:41.615 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 107, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 13, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 9, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 9, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-03-21T08:42:41.616 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.617 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-03-21T08:42:41.618 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 3003, 2024-03-21T08:42:41.619 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 67172335, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 3003, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3003, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 67172335, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.620 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:41.621 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2724, 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 65020470, 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2724, 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 65020470, 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:41.622 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:41.623 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2724, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 2724, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2724, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 2724, 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:41.624 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:41.625 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:41.626 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x556295664780": { 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-21T08:42:41.627 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-21T08:42:41.628 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:41.629 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:41.629 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:41.629 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:41.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.629 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:41.629 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-21T08:42:42.567 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-21T08:42:42.567 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-21T08:42:42.567 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-21T08:42:42.567 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-21T08:42:42.567 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T08:42:42.567 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-03-21T08:42:42.567 DEBUG:tasks.check_counter:Getting stats from l 2024-03-21T08:42:42.568 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-21T08:42:42.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:42 smithi084 ceph-mon[35477]: pgmap v338: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:43.084 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:42 smithi003 ceph-mon[31675]: pgmap v338: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:43.084 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:42 smithi171 ceph-mon[35388]: pgmap v338: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:43.084 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:42:44.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:44 smithi171 ceph-mon[35388]: pgmap v339: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:44.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:44 smithi084 ceph-mon[35477]: pgmap v339: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:44.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:44 smithi003 ceph-mon[31675]: pgmap v339: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:45.319 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-21T08:42:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T08:42:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 251, 2024-03-21T08:42:45.319 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 240, 2024-03-21T08:42:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1588832, 2024-03-21T08:42:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 144288, 2024-03-21T08:42:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-03-21T08:42:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-03-21T08:42:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.116514656, 2024-03-21T08:42:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.036303462, 2024-03-21T08:42:45.320 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.093104803, 2024-03-21T08:42:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.011792630, 2024-03-21T08:42:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 239, 2024-03-21T08:42:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.024005583, 2024-03-21T08:42:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000100441 2024-03-21T08:42:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.321 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1588832, 2024-03-21T08:42:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 144288 2024-03-21T08:42:45.322 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 6, 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 185, 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3264, 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 124448, 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-03-21T08:42:45.323 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.025842926, 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.015836086, 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.004084890, 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 183, 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.038045682, 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000207899 2024-03-21T08:42:45.324 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T08:42:45.325 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 124448 2024-03-21T08:42:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T08:42:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-03-21T08:42:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-03-21T08:42:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-03-21T08:42:45.326 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-03-21T08:42:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-03-21T08:42:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-03-21T08:42:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000719371, 2024-03-21T08:42:45.327 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T08:42:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000592991, 2024-03-21T08:42:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T08:42:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T08:42:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:45.328 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T08:42:45.329 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T08:42:45.330 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T08:42:45.331 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T08:42:45.332 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T08:42:45.333 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-21T08:42:45.334 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148744, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 30, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-21T08:42:45.335 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-03-21T08:42:45.336 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-21T08:42:45.337 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 253, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1553444, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 253, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 253, 2024-03-21T08:42:45.338 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1553444, 2024-03-21T08:42:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-21T08:42:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-21T08:42:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-21T08:42:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-21T08:42:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:45.339 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-21T08:42:45.339 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-21T08:42:46.223 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-21T08:42:46.223 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-21T08:42:46.223 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-21T08:42:46.223 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-21T08:42:46.223 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-21T08:42:46.223 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-03-21T08:42:46.224 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-21T08:42:46.235 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-21T08:42:46.235 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:42:46.235 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T08:42:46.296 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:42:46.296 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T08:42:46.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:42:46.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.587+0000 7fee12800640 1 -- 172.21.15.3:0/3272538957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c102800 msgr2=0x7fee0c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.587+0000 7fee12800640 1 --2- 172.21.15.3:0/3272538957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c102800 0x7fee0c102be0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009900 tx=0x7fedfc02bcd0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.588+0000 7fee12800640 1 -- 172.21.15.3:0/3272538957 shutdown_connections 2024-03-21T08:42:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.588+0000 7fee12800640 1 --2- 172.21.15.3:0/3272538957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c10bc20 0x7fee0c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.588+0000 7fee12800640 1 --2- 172.21.15.3:0/3272538957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee0c103120 0x7fee0c10b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.588+0000 7fee12800640 1 --2- 172.21.15.3:0/3272538957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c102800 0x7fee0c102be0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.588+0000 7fee12800640 1 -- 172.21.15.3:0/3272538957 >> 172.21.15.3:0/3272538957 conn(0x7fee0c0fc020 msgr2=0x7fee0c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.588+0000 7fee12800640 1 -- 172.21.15.3:0/3272538957 shutdown_connections 2024-03-21T08:42:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.588+0000 7fee12800640 1 -- 172.21.15.3:0/3272538957 wait complete. 2024-03-21T08:42:46.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 Processor -- start 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 -- start start 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c102800 0x7fee0c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c103120 0x7fee0c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee0c10bc20 0x7fee0c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c1118d0 con 0x7fee0c103120 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c111750 con 0x7fee0c102800 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee12800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c111a50 con 0x7fee0c10bc20 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.589+0000 7fee03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c103120 0x7fee0c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:46.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c103120 0x7fee0c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36344/0 (socket says 172.21.15.3:36344) 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 -- 172.21.15.3:0/3148535416 learned_addr learned my addr 172.21.15.3:0/3148535416 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee0b400640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c102800 0x7fee0c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee0be00640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee0c10bc20 0x7fee0c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 -- 172.21.15.3:0/3148535416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee0c10bc20 msgr2=0x7fee0c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee0c10bc20 0x7fee0c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 -- 172.21.15.3:0/3148535416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c102800 msgr2=0x7fee0c19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c102800 0x7fee0c19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 -- 172.21.15.3:0/3148535416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c19f070 con 0x7fee0c103120 2024-03-21T08:42:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee0b400640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c102800 0x7fee0c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:42:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.590+0000 7fee03e00640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c103120 0x7fee0c19a610 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fedf400d750 tx=0x7fedf400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.591+0000 7fee08c00640 1 -- 172.21.15.3:0/3148535416 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4017070 con 0x7fee0c103120 2024-03-21T08:42:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.591+0000 7fee08c00640 1 -- 172.21.15.3:0/3148535416 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fedf4005020 con 0x7fee0c103120 2024-03-21T08:42:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.591+0000 7fee08c00640 1 -- 172.21.15.3:0/3148535416 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4004c60 con 0x7fee0c103120 2024-03-21T08:42:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.591+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c19f360 con 0x7fee0c103120 2024-03-21T08:42:46.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.591+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c1a6bf0 con 0x7fee0c103120 2024-03-21T08:42:46.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.592+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c06a600 con 0x7fee0c103120 2024-03-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.596+0000 7fee08c00640 1 -- 172.21.15.3:0/3148535416 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fedf4009a30 con 0x7fee0c103120 2024-03-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.597+0000 7fee08c00640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3110 0x7fedf00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.597+0000 7fee08c00640 1 -- 172.21.15.3:0/3148535416 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7fedf40cfa50 con 0x7fee0c103120 2024-03-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.597+0000 7fee0b400640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3110 0x7fedf00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.598+0000 7fee0b400640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3110 0x7fedf00a55d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005ab0 tx=0x7fedfc005a20 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.600+0000 7fee08c00640 1 -- 172.21.15.3:0/3148535416 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fedf409c0e0 con 0x7fee0c103120 2024-03-21T08:42:46.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:46 smithi003 ceph-mon[31675]: pgmap v340: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.810+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee0c19b3d0 con 0x7fee0c103120 2024-03-21T08:42:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.815+0000 7fee08c00640 1 -- 172.21.15.3:0/3148535416 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25436 (secure 0 0 0) 0x7fedf409b9e0 con 0x7fee0c103120 2024-03-21T08:42:46.816 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:42:46.816 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":107,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:42:17.611825+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3880262252":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/62217793":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/1599875703":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/1520535032":"2024-03-22T08:31:50.557523+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1115678781":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6801/1520535032":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:42:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3110 msgr2=0x7fedf00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3110 0x7fedf00a55d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005ab0 tx=0x7fedfc005a20 comp rx=0 tx=0).stop 2024-03-21T08:42:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c103120 msgr2=0x7fee0c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c103120 0x7fee0c19a610 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fedf400d750 tx=0x7fedf400dc10 comp rx=0 tx=0).stop 2024-03-21T08:42:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 shutdown_connections 2024-03-21T08:42:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3110 0x7fedf00a55d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee0c10bc20 0x7fee0c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0c103120 0x7fee0c19a610 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 --2- 172.21.15.3:0/3148535416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c102800 0x7fee0c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.818+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 >> 172.21.15.3:0/3148535416 conn(0x7fee0c0fc020 msgr2=0x7fee0c1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.819+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 shutdown_connections 2024-03-21T08:42:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:46.819+0000 7fee12800640 1 -- 172.21.15.3:0/3148535416 wait complete. 2024-03-21T08:42:46.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:42:46.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:42:46.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:46 smithi084 ceph-mon[35477]: pgmap v340: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:46.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:46 smithi171 ceph-mon[35388]: pgmap v340: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 -- 172.21.15.3:0/4252750709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc102630 msgr2=0x7f67cc102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 --2- 172.21.15.3:0/4252750709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc102630 0x7f67cc102a10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009900 tx=0x7f67bc02bcd0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 -- 172.21.15.3:0/4252750709 shutdown_connections 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 --2- 172.21.15.3:0/4252750709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67cc10ba50 0x7f67cc10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 --2- 172.21.15.3:0/4252750709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67cc102f50 0x7f67cc10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 --2- 172.21.15.3:0/4252750709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc102630 0x7f67cc102a10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 -- 172.21.15.3:0/4252750709 >> 172.21.15.3:0/4252750709 conn(0x7f67cc077130 msgr2=0x7f67cc0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 -- 172.21.15.3:0/4252750709 shutdown_connections 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.045+0000 7f67d1200640 1 -- 172.21.15.3:0/4252750709 wait complete. 2024-03-21T08:42:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.046+0000 7f67d1200640 1 Processor -- start 2024-03-21T08:42:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.046+0000 7f67d1200640 1 -- start start 2024-03-21T08:42:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.046+0000 7f67d1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67cc102f50 0x7f67cc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67d1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc10ba50 0x7f67cc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67d1200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67cc19c760 0x7f67cc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67d1200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc111720 con 0x7f67cc10ba50 2024-03-21T08:42:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67d1200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1115a0 con 0x7f67cc102f50 2024-03-21T08:42:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67d1200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1118a0 con 0x7f67cc19c760 2024-03-21T08:42:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc10ba50 0x7f67cc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc10ba50 0x7f67cc1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36356/0 (socket says 172.21.15.3:36356) 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 -- 172.21.15.3:0/3434719364 learned_addr learned my addr 172.21.15.3:0/3434719364 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67cb400640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67cc102f50 0x7f67cc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67cbe00640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67cc19c760 0x7f67cc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 -- 172.21.15.3:0/3434719364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67cc19c760 msgr2=0x7f67cc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67cc19c760 0x7f67cc19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 -- 172.21.15.3:0/3434719364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67cc102f50 msgr2=0x7f67cc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67cc102f50 0x7f67cc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67caa00640 1 -- 172.21.15.3:0/3434719364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc19d250 con 0x7f67cc10ba50 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67cbe00640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67cc19c760 0x7f67cc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.047+0000 7f67cb400640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67cc102f50 0x7f67cc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.048+0000 7f67caa00640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc10ba50 0x7f67cc1983e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f67b800e9f0 tx=0x7f67b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.048+0000 7f67c3e00640 1 -- 172.21.15.3:0/3434719364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800cd80 con 0x7f67cc10ba50 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.048+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc19d540 con 0x7f67cc10ba50 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.048+0000 7f67c3e00640 1 -- 172.21.15.3:0/3434719364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f67b80108e0 con 0x7f67cc10ba50 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.048+0000 7f67c3e00640 1 -- 172.21.15.3:0/3434719364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8018920 con 0x7f67cc10ba50 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.048+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc1a4d30 con 0x7f67cc10ba50 2024-03-21T08:42:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.049+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc04f400 con 0x7f67cc10ba50 2024-03-21T08:42:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.055+0000 7f67c3e00640 1 -- 172.21.15.3:0/3434719364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f67b8010a90 con 0x7f67cc10ba50 2024-03-21T08:42:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.056+0000 7f67c3e00640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f67b00a3090 0x7f67b00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.056+0000 7f67c3e00640 1 -- 172.21.15.3:0/3434719364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f67b80d05b0 con 0x7f67cc10ba50 2024-03-21T08:42:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.057+0000 7f67cb400640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f67b00a3090 0x7f67b00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.057+0000 7f67c3e00640 1 -- 172.21.15.3:0/3434719364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f67b809cbc0 con 0x7f67cc10ba50 2024-03-21T08:42:47.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.057+0000 7f67cb400640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f67b00a3090 0x7f67b00a5550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f67bc036040 tx=0x7f67bc005a20 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.259+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67cc02d070 con 0x7f67cc10ba50 2024-03-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.260+0000 7f67c3e00640 1 -- 172.21.15.3:0/3434719364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67b809c4c0 con 0x7f67cc10ba50 2024-03-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f67b00a3090 msgr2=0x7f67b00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f67b00a3090 0x7f67b00a5550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f67bc036040 tx=0x7f67bc005a20 comp rx=0 tx=0).stop 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc10ba50 msgr2=0x7f67cc1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc10ba50 0x7f67cc1983e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f67b800e9f0 tx=0x7f67b800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 shutdown_connections 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f67b00a3090 0x7f67b00a5550 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67cc19c760 0x7f67cc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc10ba50 0x7f67cc1983e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 --2- 172.21.15.3:0/3434719364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67cc102f50 0x7f67cc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 >> 172.21.15.3:0/3434719364 conn(0x7f67cc077130 msgr2=0x7f67cc0fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 shutdown_connections 2024-03-21T08:42:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.263+0000 7f67d1200640 1 -- 172.21.15.3:0/3434719364 wait complete. 2024-03-21T08:42:47.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:42:47.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 -- 172.21.15.3:0/1027065729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100420 msgr2=0x7fee08100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1027065729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100420 0x7fee08100800 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009900 tx=0x7fedfc02bcd0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 -- 172.21.15.3:0/1027065729 shutdown_connections 2024-03-21T08:42:47.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1027065729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee08105820 0x7fee08105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1027065729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100d40 0x7fee08105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1027065729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100420 0x7fee08100800 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 -- 172.21.15.3:0/1027065729 >> 172.21.15.3:0/1027065729 conn(0x7fee08077130 msgr2=0x7fee080fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:47.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 -- 172.21.15.3:0/1027065729 shutdown_connections 2024-03-21T08:42:47.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.483+0000 7fee0ec00640 1 -- 172.21.15.3:0/1027065729 wait complete. 2024-03-21T08:42:47.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.484+0000 7fee0ec00640 1 Processor -- start 2024-03-21T08:42:47.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.484+0000 7fee0ec00640 1 -- start start 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ec00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100420 0x7fee0819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100d40 0x7fee0819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ec00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee08105820 0x7fee081a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08117d30 con 0x7fee08100d40 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ec00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08117bb0 con 0x7fee08100420 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ec00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08117eb0 con 0x7fee08105820 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100420 0x7fee0819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100d40 0x7fee0819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100d40 0x7fee0819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36380/0 (socket says 172.21.15.3:36380) 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0ce00640 1 -- 172.21.15.3:0/1069548162 learned_addr learned my addr 172.21.15.3:0/1069548162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.485+0000 7fee0d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100420 0x7fee0819e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.3:45148/0 (socket says 172.21.15.3:45148) 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0e200640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee08105820 0x7fee081a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0d800640 1 -- 172.21.15.3:0/1069548162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee08105820 msgr2=0x7fee081a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0d800640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee08105820 0x7fee081a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0d800640 1 -- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100d40 msgr2=0x7fee0819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0d800640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100d40 0x7fee0819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0d800640 1 -- 172.21.15.3:0/1069548162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee081a3240 con 0x7fee08100420 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0ce00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100d40 0x7fee0819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.486+0000 7fee0d800640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100420 0x7fee0819e250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedfc02c210 tx=0x7fedfc0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.487+0000 7fee06000640 1 -- 172.21.15.3:0/1069548162 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc043020 con 0x7fee08100420 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.487+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee081a34d0 con 0x7fee08100420 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.487+0000 7fee06000640 1 -- 172.21.15.3:0/1069548162 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fedfc02dc20 con 0x7fee08100420 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.487+0000 7fee06000640 1 -- 172.21.15.3:0/1069548162 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0039b0 con 0x7fee08100420 2024-03-21T08:42:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.487+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee081aadc0 con 0x7fee08100420 2024-03-21T08:42:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.488+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0804f400 con 0x7fee08100420 2024-03-21T08:42:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.491+0000 7fee06000640 1 -- 172.21.15.3:0/1069548162 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fedfc046020 con 0x7fee08100420 2024-03-21T08:42:47.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.492+0000 7fee06000640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3090 0x7fedf00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.493+0000 7fee06000640 1 -- 172.21.15.3:0/1069548162 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7fedfc0f21f0 con 0x7fee08100420 2024-03-21T08:42:47.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.493+0000 7fee0ce00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3090 0x7fedf00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.493+0000 7fee0ce00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3090 0x7fedf00a5550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fedf40095f0 tx=0x7fedf4005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:47.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.506+0000 7fee06000640 1 -- 172.21.15.3:0/1069548162 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fedfc04d020 con 0x7fee08100420 2024-03-21T08:42:47.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.707+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee081aaff0 con 0x7fee08100420 2024-03-21T08:42:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.708+0000 7fee06000640 1 -- 172.21.15.3:0/1069548162 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7fedfc0be910 con 0x7fee08100420 2024-03-21T08:42:47.708 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3090 msgr2=0x7fedf00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3090 0x7fedf00a5550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fedf40095f0 tx=0x7fedf4005f90 comp rx=0 tx=0).stop 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100420 msgr2=0x7fee0819e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100420 0x7fee0819e250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedfc02c210 tx=0x7fedfc0045e0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 shutdown_connections 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fedf00a3090 0x7fedf00a5550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee08105820 0x7fee081a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee08100d40 0x7fee0819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 --2- 172.21.15.3:0/1069548162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee08100420 0x7fee0819e250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 >> 172.21.15.3:0/1069548162 conn(0x7fee08077130 msgr2=0x7fee081031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 shutdown_connections 2024-03-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.711+0000 7fee0ec00640 1 -- 172.21.15.3:0/1069548162 wait complete. 2024-03-21T08:42:47.730 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:42:47.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:47 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3148535416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:42:47.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:47 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3434719364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:42:47.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:47 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3148535416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:42:47.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:47 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3434719364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:42:47.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:47 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3148535416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:42:47.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:47 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3434719364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:42:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.932+0000 7fcd7fe00640 1 -- 172.21.15.3:0/4277822545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80107500 msgr2=0x7fcd801078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.932+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/4277822545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80107500 0x7fcd801078e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009930 tx=0x7fcd7002bc30 comp rx=0 tx=0).stop 2024-03-21T08:42:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.932+0000 7fcd7fe00640 1 -- 172.21.15.3:0/4277822545 shutdown_connections 2024-03-21T08:42:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.932+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/4277822545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd80101e20 0x7fcd8010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.932+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/4277822545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd80101460 0x7fcd801018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.932+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/4277822545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80107500 0x7fcd801078e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.932+0000 7fcd7fe00640 1 -- 172.21.15.3:0/4277822545 >> 172.21.15.3:0/4277822545 conn(0x7fcd800fcff0 msgr2=0x7fcd800ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.933+0000 7fcd7fe00640 1 -- 172.21.15.3:0/4277822545 shutdown_connections 2024-03-21T08:42:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.933+0000 7fcd7fe00640 1 -- 172.21.15.3:0/4277822545 wait complete. 2024-03-21T08:42:47.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.933+0000 7fcd7fe00640 1 Processor -- start 2024-03-21T08:42:47.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7fe00640 1 -- start start 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80101460 0x7fcd800739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd80101e20 0x7fcd80073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd80107500 0x7fcd80074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80111730 con 0x7fcd80101460 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd801115b0 con 0x7fcd80107500 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd801118b0 con 0x7fcd80101e20 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80101460 0x7fcd800739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80101460 0x7fcd800739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36402/0 (socket says 172.21.15.3:36402) 2024-03-21T08:42:47.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7ea00640 1 -- 172.21.15.3:0/2450463660 learned_addr learned my addr 172.21.15.3:0/2450463660 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.934+0000 7fcd7f400640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd80107500 0x7fcd80074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd76000640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd80101e20 0x7fcd80073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7ea00640 1 -- 172.21.15.3:0/2450463660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd80101e20 msgr2=0x7fcd80073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7ea00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd80101e20 0x7fcd80073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7ea00640 1 -- 172.21.15.3:0/2450463660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd80107500 msgr2=0x7fcd80074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7ea00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd80107500 0x7fcd80074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7ea00640 1 -- 172.21.15.3:0/2450463660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd801a6170 con 0x7fcd80101460 2024-03-21T08:42:47.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7ea00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80101460 0x7fcd800739a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005940 tx=0x7fcd7002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd77e00640 1 -- 172.21.15.3:0/2450463660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70043020 con 0x7fcd80101460 2024-03-21T08:42:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd801a6400 con 0x7fcd80101460 2024-03-21T08:42:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd77e00640 1 -- 172.21.15.3:0/2450463660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcd7002cc80 con 0x7fcd80101460 2024-03-21T08:42:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd77e00640 1 -- 172.21.15.3:0/2450463660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70003c60 con 0x7fcd80101460 2024-03-21T08:42:47.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.935+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd801a6950 con 0x7fcd80101460 2024-03-21T08:42:47.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.937+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8004f400 con 0x7fcd80101460 2024-03-21T08:42:47.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.939+0000 7fcd77e00640 1 -- 172.21.15.3:0/2450463660 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fcd70046020 con 0x7fcd80101460 2024-03-21T08:42:47.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.940+0000 7fcd77e00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fcd640a3090 0x7fcd640a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:47.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.940+0000 7fcd77e00640 1 -- 172.21.15.3:0/2450463660 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7fcd700f1f30 con 0x7fcd80101460 2024-03-21T08:42:47.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.940+0000 7fcd76000640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fcd640a3090 0x7fcd640a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:47.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.941+0000 7fcd76000640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fcd640a3090 0x7fcd640a5550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005e10 tx=0x7fcd68005da0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:47.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:47.948+0000 7fcd77e00640 1 -- 172.21.15.3:0/2450463660 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcd7004d020 con 0x7fcd80101460 2024-03-21T08:42:48.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.147+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd801099d0 con 0x7fcd80101460 2024-03-21T08:42:48.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.147+0000 7fcd77e00640 1 -- 172.21.15.3:0/2450463660 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd700be6b0 con 0x7fcd80101460 2024-03-21T08:42:48.149 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:42:48.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fcd640a3090 msgr2=0x7fcd640a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fcd640a3090 0x7fcd640a5550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005e10 tx=0x7fcd68005da0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80101460 msgr2=0x7fcd800739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80101460 0x7fcd800739a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005940 tx=0x7fcd7002ca90 comp rx=0 tx=0).stop 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 shutdown_connections 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd80107500 0x7fcd80074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fcd640a3090 0x7fcd640a5550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd80101e20 0x7fcd80073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 --2- 172.21.15.3:0/2450463660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd80101460 0x7fcd800739a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 >> 172.21.15.3:0/2450463660 conn(0x7fcd800fcff0 msgr2=0x7fcd800fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:48.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 shutdown_connections 2024-03-21T08:42:48.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.151+0000 7fcd7fe00640 1 -- 172.21.15.3:0/2450463660 wait complete. 2024-03-21T08:42:48.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:42:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 -- 172.21.15.3:0/2579133432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101630 msgr2=0x7f6910101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 --2- 172.21.15.3:0/2579133432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101630 0x7f6910101ab0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009930 tx=0x7f68fc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:42:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 -- 172.21.15.3:0/2579133432 shutdown_connections 2024-03-21T08:42:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 --2- 172.21.15.3:0/2579133432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6910101ff0 0x7f691010e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 --2- 172.21.15.3:0/2579133432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101630 0x7f6910101ab0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 --2- 172.21.15.3:0/2579133432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6910107640 0x7f6910107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 -- 172.21.15.3:0/2579133432 >> 172.21.15.3:0/2579133432 conn(0x7f69100fd140 msgr2=0x7f69100ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 -- 172.21.15.3:0/2579133432 shutdown_connections 2024-03-21T08:42:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.395+0000 7f6918600640 1 -- 172.21.15.3:0/2579133432 wait complete. 2024-03-21T08:42:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.396+0000 7f6918600640 1 Processor -- start 2024-03-21T08:42:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.396+0000 7f6918600640 1 -- start start 2024-03-21T08:42:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6918600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6910101630 0x7f691019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6918600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101ff0 0x7f691019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6918600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6910107640 0x7f691019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6918600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69101118c0 con 0x7f6910101ff0 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6918600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910111740 con 0x7f6910107640 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6918600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910111a40 con 0x7f6910101630 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6915200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101ff0 0x7f691019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6915200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101ff0 0x7f691019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36428/0 (socket says 172.21.15.3:36428) 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6915200640 1 -- 172.21.15.3:0/1902037196 learned_addr learned my addr 172.21.15.3:0/1902037196 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6916600640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6910107640 0x7f691019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.397+0000 7f6915c00640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6910101630 0x7f691019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6915200640 1 -- 172.21.15.3:0/1902037196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6910101630 msgr2=0x7f691019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6915200640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6910101630 0x7f691019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6915200640 1 -- 172.21.15.3:0/1902037196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6910107640 msgr2=0x7f691019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6915200640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6910107640 0x7f691019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6915200640 1 -- 172.21.15.3:0/1902037196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691019f090 con 0x7f6910101ff0 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6915c00640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6910101630 0x7f691019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:42:48.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6915200640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101ff0 0x7f691019a5e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009230 tx=0x7f68fc02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f690e000640 1 -- 172.21.15.3:0/1902037196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc043020 con 0x7f6910101ff0 2024-03-21T08:42:48.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691019f320 con 0x7f6910101ff0 2024-03-21T08:42:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.398+0000 7f690e000640 1 -- 172.21.15.3:0/1902037196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68fc02caf0 con 0x7f6910101ff0 2024-03-21T08:42:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.399+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69101a6c10 con 0x7f6910101ff0 2024-03-21T08:42:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.399+0000 7f690e000640 1 -- 172.21.15.3:0/1902037196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc003a80 con 0x7f6910101ff0 2024-03-21T08:42:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.400+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6910068970 con 0x7f6910101ff0 2024-03-21T08:42:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.402+0000 7f690e000640 1 -- 172.21.15.3:0/1902037196 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f68fc046020 con 0x7f6910101ff0 2024-03-21T08:42:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.403+0000 7f690e000640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f68f80a3090 0x7f68f80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.403+0000 7f690e000640 1 -- 172.21.15.3:0/1902037196 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f68fc0f1ff0 con 0x7f6910101ff0 2024-03-21T08:42:48.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.404+0000 7f6915c00640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f68f80a3090 0x7f68f80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.404+0000 7f6915c00640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f68f80a3090 0x7f68f80a5550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6904009ba0 tx=0x7f6904005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:48.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.406+0000 7f690e000640 1 -- 172.21.15.3:0/1902037196 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f68fc04d020 con 0x7f6910101ff0 2024-03-21T08:42:48.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.609+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f69101a6e40 con 0x7f6910101ff0 2024-03-21T08:42:48.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.610+0000 7f690e000640 1 -- 172.21.15.3:0/1902037196 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) v1 ==== 103+0+11 (secure 0 0 0) 0x7f68fc0be770 con 0x7f6910101ff0 2024-03-21T08:42:48.610 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.612+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f68f80a3090 msgr2=0x7f68f80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.612+0000 7f6918600640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f68f80a3090 0x7f68f80a5550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6904009ba0 tx=0x7f6904005f90 comp rx=0 tx=0).stop 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.612+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101ff0 msgr2=0x7f691019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.612+0000 7f6918600640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101ff0 0x7f691019a5e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009230 tx=0x7f68fc02c710 comp rx=0 tx=0).stop 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 shutdown_connections 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6910107640 0x7f691019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6910101ff0 0x7f691019a5e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f68f80a3090 0x7f68f80a5550 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 --2- 172.21.15.3:0/1902037196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6910101630 0x7f691019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 >> 172.21.15.3:0/1902037196 conn(0x7f69100fd140 msgr2=0x7f691010cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:48.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 shutdown_connections 2024-03-21T08:42:48.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.613+0000 7f6918600640 1 -- 172.21.15.3:0/1902037196 wait complete. 2024-03-21T08:42:48.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T08:42:48.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:48 smithi003 ceph-mon[31675]: pgmap v341: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:48.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:48 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1069548162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:42:48.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:48 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2450463660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:42:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.863+0000 7f2fdf200640 1 -- 172.21.15.3:0/2216270043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8103680 msgr2=0x7f2fd8103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.863+0000 7f2fdf200640 1 --2- 172.21.15.3:0/2216270043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8103680 0x7f2fd8103b00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009930 tx=0x7f2fc802bc30 comp rx=0 tx=0).stop 2024-03-21T08:42:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.864+0000 7f2fdf200640 1 -- 172.21.15.3:0/2216270043 shutdown_connections 2024-03-21T08:42:48.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.864+0000 7f2fdf200640 1 --2- 172.21.15.3:0/2216270043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fd8104040 0x7f2fd8110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.864+0000 7f2fdf200640 1 --2- 172.21.15.3:0/2216270043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8103680 0x7f2fd8103b00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.864+0000 7f2fdf200640 1 --2- 172.21.15.3:0/2216270043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8109690 0x7f2fd8109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.864+0000 7f2fdf200640 1 -- 172.21.15.3:0/2216270043 >> 172.21.15.3:0/2216270043 conn(0x7f2fd80ff110 msgr2=0x7f2fd8101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.864+0000 7f2fdf200640 1 -- 172.21.15.3:0/2216270043 shutdown_connections 2024-03-21T08:42:48.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.864+0000 7f2fdf200640 1 -- 172.21.15.3:0/2216270043 wait complete. 2024-03-21T08:42:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.865+0000 7f2fdf200640 1 Processor -- start 2024-03-21T08:42:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.865+0000 7f2fdf200640 1 -- start start 2024-03-21T08:42:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fd8103680 0x7f2fd8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8104040 0x7f2fd819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8109690 0x7f2fd819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdf200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8113950 con 0x7f2fd8104040 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdf200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd81137d0 con 0x7f2fd8109690 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdf200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8113ad0 con 0x7f2fd8103680 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8104040 0x7f2fd819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8104040 0x7f2fd819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36446/0 (socket says 172.21.15.3:36446) 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdd400640 1 -- 172.21.15.3:0/764851906 learned_addr learned my addr 172.21.15.3:0/764851906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fde800640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8109690 0x7f2fd819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.866+0000 7f2fdde00640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fd8103680 0x7f2fd8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fdd400640 1 -- 172.21.15.3:0/764851906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fd8103680 msgr2=0x7f2fd8199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fdd400640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fd8103680 0x7f2fd8199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fdd400640 1 -- 172.21.15.3:0/764851906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8109690 msgr2=0x7f2fd819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fdd400640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8109690 0x7f2fd819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fdd400640 1 -- 172.21.15.3:0/764851906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd819eeb0 con 0x7f2fd8104040 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fdd400640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8104040 0x7f2fd819a400 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009230 tx=0x7f2fc802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fcea00640 1 -- 172.21.15.3:0/764851906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8043020 con 0x7f2fd8104040 2024-03-21T08:42:48.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fcea00640 1 -- 172.21.15.3:0/764851906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fc802caf0 con 0x7f2fd8104040 2024-03-21T08:42:48.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd819f140 con 0x7f2fd8104040 2024-03-21T08:42:48.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.867+0000 7f2fcea00640 1 -- 172.21.15.3:0/764851906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8003ae0 con 0x7f2fd8104040 2024-03-21T08:42:48.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.868+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd81a6a30 con 0x7f2fd8104040 2024-03-21T08:42:48.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.869+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd8104dc0 con 0x7f2fd8104040 2024-03-21T08:42:48.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.871+0000 7f2fcea00640 1 -- 172.21.15.3:0/764851906 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f2fc8046020 con 0x7f2fd8104040 2024-03-21T08:42:48.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.872+0000 7f2fcea00640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2fc40a3090 0x7f2fc40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:48.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.872+0000 7f2fcea00640 1 -- 172.21.15.3:0/764851906 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f2fc80f1c80 con 0x7f2fd8104040 2024-03-21T08:42:48.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.872+0000 7f2fdde00640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2fc40a3090 0x7f2fc40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:48.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.873+0000 7f2fdde00640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2fc40a3090 0x7f2fc40a5550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0005fd0 tx=0x7f2fd0009500 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:48.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:48 smithi084 ceph-mon[35477]: pgmap v341: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:48.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:48 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1069548162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:42:48.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:48 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2450463660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:48.876+0000 7f2fcea00640 1 -- 172.21.15.3:0/764851906 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2fc804d020 con 0x7f2fd8104040 2024-03-21T08:42:48.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:48 smithi171 ceph-mon[35388]: pgmap v341: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:48.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:48 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1069548162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:42:48.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:48 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2450463660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:42:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.095+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2fd810bbf0 con 0x7f2fd8104040 2024-03-21T08:42:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.096+0000 7f2fcea00640 1 -- 172.21.15.3:0/764851906 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) v1 ==== 81+0+927 (secure 0 0 0) 0x7f2fc80be3a0 con 0x7f2fd8104040 2024-03-21T08:42:49.097 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3880262252 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:49.097 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/62217793 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:49.097 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1599875703 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:49.097 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2117697938 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2293101955 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1520535032 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1115678781 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:49.098 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/404577225 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:49.099 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1520535032 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:49.099 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1597279144 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:49.099 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4112229601 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:49.099 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2051354447 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:listed 16 entries 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2fc40a3090 msgr2=0x7f2fc40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2fc40a3090 0x7f2fc40a5550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0005fd0 tx=0x7f2fd0009500 comp rx=0 tx=0).stop 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8104040 msgr2=0x7f2fd819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8104040 0x7f2fd819a400 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009230 tx=0x7f2fc802c710 comp rx=0 tx=0).stop 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 shutdown_connections 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8109690 0x7f2fd819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8104040 0x7f2fd819a400 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f2fc40a3090 0x7f2fc40a5550 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 --2- 172.21.15.3:0/764851906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fd8103680 0x7f2fd8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 >> 172.21.15.3:0/764851906 conn(0x7f2fd80ff110 msgr2=0x7f2fd80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 shutdown_connections 2024-03-21T08:42:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.099+0000 7f2fdf200640 1 -- 172.21.15.3:0/764851906 wait complete. 2024-03-21T08:42:49.118 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-03-21T08:42:49.119 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T08:42:49.170 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:42:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 -- 172.21.15.3:0/1671424308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8108800 msgr2=0x7fe9f810c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 --2- 172.21.15.3:0/1671424308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8108800 0x7fe9f810c7c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009900 tx=0x7fe9e402bc30 comp rx=0 tx=0).stop 2024-03-21T08:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 -- 172.21.15.3:0/1671424308 shutdown_connections 2024-03-21T08:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 --2- 172.21.15.3:0/1671424308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9f810cea0 0x7fe9f8114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 --2- 172.21.15.3:0/1671424308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8108800 0x7fe9f810c7c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 --2- 172.21.15.3:0/1671424308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f8107e50 0x7fe9f8108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 -- 172.21.15.3:0/1671424308 >> 172.21.15.3:0/1671424308 conn(0x7fe9f8077470 msgr2=0x7fe9f81064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:49.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 -- 172.21.15.3:0/1671424308 shutdown_connections 2024-03-21T08:42:49.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.439+0000 7fe9fde00640 1 -- 172.21.15.3:0/1671424308 wait complete. 2024-03-21T08:42:49.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.440+0000 7fe9fde00640 1 Processor -- start 2024-03-21T08:42:49.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.440+0000 7fe9fde00640 1 -- start start 2024-03-21T08:42:49.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8107e50 0x7fe9f819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fde00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9f810cea0 0x7fe9f819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f81a2c50 0x7fe9f81a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fde00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8117e10 con 0x7fe9f8107e50 2024-03-21T08:42:49.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fde00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8117c90 con 0x7fe9f81a2c50 2024-03-21T08:42:49.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fde00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8117f90 con 0x7fe9f810cea0 2024-03-21T08:42:49.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8107e50 0x7fe9f819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8107e50 0x7fe9f819e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36456/0 (socket says 172.21.15.3:36456) 2024-03-21T08:42:49.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fca00640 1 -- 172.21.15.3:0/3230104215 learned_addr learned my addr 172.21.15.3:0/3230104215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:49.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9fd400640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f81a2c50 0x7fe9f81a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.441+0000 7fe9f7e00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9f810cea0 0x7fe9f819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fca00640 1 -- 172.21.15.3:0/3230104215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9f810cea0 msgr2=0x7fe9f819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fca00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9f810cea0 0x7fe9f819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fca00640 1 -- 172.21.15.3:0/3230104215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f81a2c50 msgr2=0x7fe9f81a3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fca00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f81a2c50 0x7fe9f81a3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fca00640 1 -- 172.21.15.3:0/3230104215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f81a3740 con 0x7fe9f8107e50 2024-03-21T08:42:49.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fca00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8107e50 0x7fe9f819e390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00d950 tx=0x7fe9ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:49.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9f5600640 1 -- 172.21.15.3:0/3230104215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec017070 con 0x7fe9f8107e50 2024-03-21T08:42:49.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9f5600640 1 -- 172.21.15.3:0/3230104215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe9ec009c30 con 0x7fe9f8107e50 2024-03-21T08:42:49.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9f5600640 1 -- 172.21.15.3:0/3230104215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00f620 con 0x7fe9f8107e50 2024-03-21T08:42:49.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f81a3a30 con 0x7fe9f8107e50 2024-03-21T08:42:49.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.442+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f81ab1e0 con 0x7fe9f8107e50 2024-03-21T08:42:49.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.444+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f804f470 con 0x7fe9f8107e50 2024-03-21T08:42:49.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.445+0000 7fe9f5600640 1 -- 172.21.15.3:0/3230104215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fe9ec0109a0 con 0x7fe9f8107e50 2024-03-21T08:42:49.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.447+0000 7fe9f5600640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9e00a3100 0x7fe9e00a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.448+0000 7fe9f5600640 1 -- 172.21.15.3:0/3230104215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7fe9ec0cf900 con 0x7fe9f8107e50 2024-03-21T08:42:49.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.448+0000 7fe9f7e00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9e00a3100 0x7fe9e00a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.449+0000 7fe9f7e00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9e00a3100 0x7fe9e00a55c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9e402c170 tx=0x7fe9e4000bc0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:49.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.452+0000 7fe9f5600640 1 -- 172.21.15.3:0/3230104215 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe9ec009de0 con 0x7fe9f8107e50 2024-03-21T08:42:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.659+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9f8108800 con 0x7fe9f8107e50 2024-03-21T08:42:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.663+0000 7fe9f5600640 1 -- 172.21.15.3:0/3230104215 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25436 (secure 0 0 0) 0x7fe9ec09bf10 con 0x7fe9f8107e50 2024-03-21T08:42:49.665 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-03-21T08:42:49.665 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":107,"fsid":"544fc91e-e75b-11ee-95cd-87774f69a715","created":"2024-03-21T08:17:50.596819+0000","modified":"2024-03-21T08:42:17.611825+0000","last_up_change":"2024-03-21T08:27:29.124361+0000","last_in_change":"2024-03-21T08:27:04.875015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:22:17.881021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:28:34.541804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:28:35.622299+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T08:28:39.211474+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94762ab2-9404-4097-911f-0f6a15f4788a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6803","nonce":3803233195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6805","nonce":3803233195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6809","nonce":3803233195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3803233195},{"type":"v1","addr":"172.21.15.3:6807","nonce":3803233195}]},"public_addr":"172.21.15.3:6803/3803233195","cluster_addr":"172.21.15.3:6805/3803233195","heartbeat_back_addr":"172.21.15.3:6809/3803233195","heartbeat_front_addr":"172.21.15.3:6807/3803233195","state":["exists","up"]},{"osd":1,"uuid":"4ae4ad7a-eb28-42c8-a31a-26241b629dae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6801","nonce":2209911029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6803","nonce":2209911029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6807","nonce":2209911029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2209911029},{"type":"v1","addr":"172.21.15.84:6805","nonce":2209911029}]},"public_addr":"172.21.15.84:6801/2209911029","cluster_addr":"172.21.15.84:6803/2209911029","heartbeat_back_addr":"172.21.15.84:6807/2209911029","heartbeat_front_addr":"172.21.15.84:6805/2209911029","state":["exists","up"]},{"osd":2,"uuid":"72b1933d-fa5d-423e-9711-162d319d2c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6801","nonce":2357635272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6803","nonce":2357635272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6807","nonce":2357635272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2357635272},{"type":"v1","addr":"172.21.15.171:6805","nonce":2357635272}]},"public_addr":"172.21.15.171:6801/2357635272","cluster_addr":"172.21.15.171:6803/2357635272","heartbeat_back_addr":"172.21.15.171:6807/2357635272","heartbeat_front_addr":"172.21.15.171:6805/2357635272","state":["exists","up"]},{"osd":3,"uuid":"097d03f7-da4c-4c3e-b98d-1da6de58611f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6811","nonce":674164001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6813","nonce":674164001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6817","nonce":674164001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":674164001},{"type":"v1","addr":"172.21.15.3:6815","nonce":674164001}]},"public_addr":"172.21.15.3:6811/674164001","cluster_addr":"172.21.15.3:6813/674164001","heartbeat_back_addr":"172.21.15.3:6817/674164001","heartbeat_front_addr":"172.21.15.3:6815/674164001","state":["exists","up"]},{"osd":4,"uuid":"73fe75be-a893-4d19-b3ec-9d75c169f300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6809","nonce":2287981020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6811","nonce":2287981020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6815","nonce":2287981020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2287981020},{"type":"v1","addr":"172.21.15.84:6813","nonce":2287981020}]},"public_addr":"172.21.15.84:6809/2287981020","cluster_addr":"172.21.15.84:6811/2287981020","heartbeat_back_addr":"172.21.15.84:6815/2287981020","heartbeat_front_addr":"172.21.15.84:6813/2287981020","state":["exists","up"]},{"osd":5,"uuid":"51c9f7b7-be7e-46db-b99c-ea76625620dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6809","nonce":246158153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6811","nonce":246158153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6815","nonce":246158153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":246158153},{"type":"v1","addr":"172.21.15.171:6813","nonce":246158153}]},"public_addr":"172.21.15.171:6809/246158153","cluster_addr":"172.21.15.171:6811/246158153","heartbeat_back_addr":"172.21.15.171:6815/246158153","heartbeat_front_addr":"172.21.15.171:6813/246158153","state":["exists","up"]},{"osd":6,"uuid":"68131baf-10d4-4e02-a3cc-b7ec95fe3708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6819","nonce":3180742022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6821","nonce":3180742022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6825","nonce":3180742022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3180742022},{"type":"v1","addr":"172.21.15.3:6823","nonce":3180742022}]},"public_addr":"172.21.15.3:6819/3180742022","cluster_addr":"172.21.15.3:6821/3180742022","heartbeat_back_addr":"172.21.15.3:6825/3180742022","heartbeat_front_addr":"172.21.15.3:6823/3180742022","state":["exists","up"]},{"osd":7,"uuid":"06b9b8c1-ca04-4499-96d1-79e0e0f316d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6817","nonce":3012309737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6819","nonce":3012309737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6823","nonce":3012309737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":3012309737},{"type":"v1","addr":"172.21.15.84:6821","nonce":3012309737}]},"public_addr":"172.21.15.84:6817/3012309737","cluster_addr":"172.21.15.84:6819/3012309737","heartbeat_back_addr":"172.21.15.84:6823/3012309737","heartbeat_front_addr":"172.21.15.84:6821/3012309737","state":["exists","up"]},{"osd":8,"uuid":"e75bbebb-99e2-47c0-8e5a-0842a58c1092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6817","nonce":1162829385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6819","nonce":1162829385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6823","nonce":1162829385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1162829385},{"type":"v1","addr":"172.21.15.171:6821","nonce":1162829385}]},"public_addr":"172.21.15.171:6817/1162829385","cluster_addr":"172.21.15.171:6819/1162829385","heartbeat_back_addr":"172.21.15.171:6823/1162829385","heartbeat_front_addr":"172.21.15.171:6821/1162829385","state":["exists","up"]},{"osd":9,"uuid":"5bad569e-33cd-4ec5-a11f-66e7b60cc9ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6827","nonce":1803156787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6829","nonce":1803156787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6833","nonce":1803156787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1803156787},{"type":"v1","addr":"172.21.15.3:6831","nonce":1803156787}]},"public_addr":"172.21.15.3:6827/1803156787","cluster_addr":"172.21.15.3:6829/1803156787","heartbeat_back_addr":"172.21.15.3:6833/1803156787","heartbeat_front_addr":"172.21.15.3:6831/1803156787","state":["exists","up"]},{"osd":10,"uuid":"960b5226-4fe7-42b0-bba6-193c2425e0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6825","nonce":3984184621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6827","nonce":3984184621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6831","nonce":3984184621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3984184621},{"type":"v1","addr":"172.21.15.84:6829","nonce":3984184621}]},"public_addr":"172.21.15.84:6825/3984184621","cluster_addr":"172.21.15.84:6827/3984184621","heartbeat_back_addr":"172.21.15.84:6831/3984184621","heartbeat_front_addr":"172.21.15.84:6829/3984184621","state":["exists","up"]},{"osd":11,"uuid":"553edf28-7b5a-48cf-8ab4-c003e219e1b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6825","nonce":2511529180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6827","nonce":2511529180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6831","nonce":2511529180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2511529180},{"type":"v1","addr":"172.21.15.171:6829","nonce":2511529180}]},"public_addr":"172.21.15.171:6825/2511529180","cluster_addr":"172.21.15.171:6827/2511529180","heartbeat_back_addr":"172.21.15.171:6831/2511529180","heartbeat_front_addr":"172.21.15.171:6829/2511529180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:08.520857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:21:40.629581+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:13.152561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:22:48.458441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:22.411325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:23:56.072026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:24:31.562967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:06.178647+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:25:40.965721+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:16.571326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:26:52.372879+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:27:27.953419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":5,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/3880262252":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/62217793":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/1599875703":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/2117697938":"2024-03-22T08:19:05.157913+0000","172.21.15.3:0/2293101955":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/1520535032":"2024-03-22T08:31:50.557523+0000","172.21.15.3:6801/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6800/2168061035":"2024-03-22T08:19:05.157913+0000","172.21.15.3:6801/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6800/1378601347":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/1115678781":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/404577225":"2024-03-22T08:18:30.424775+0000","172.21.15.3:6801/1520535032":"2024-03-22T08:31:50.557523+0000","172.21.15.3:0/1597279144":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/4112229601":"2024-03-22T08:18:30.424775+0000","172.21.15.3:0/2051354447":"2024-03-22T08:19:05.157913+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:42:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9e00a3100 msgr2=0x7fe9e00a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9e00a3100 0x7fe9e00a55c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9e402c170 tx=0x7fe9e4000bc0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8107e50 msgr2=0x7fe9f819e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8107e50 0x7fe9f819e390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00d950 tx=0x7fe9ec00de10 comp rx=0 tx=0).stop 2024-03-21T08:42:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 shutdown_connections 2024-03-21T08:42:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9f81a2c50 0x7fe9f81a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9e00a3100 0x7fe9e00a55c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9f810cea0 0x7fe9f819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 --2- 172.21.15.3:0/3230104215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8107e50 0x7fe9f819e390 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.666+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 >> 172.21.15.3:0/3230104215 conn(0x7fe9f8077470 msgr2=0x7fe9f80790f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.667+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 shutdown_connections 2024-03-21T08:42:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.667+0000 7fe9fde00640 1 -- 172.21.15.3:0/3230104215 wait complete. 2024-03-21T08:42:49.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:22:17.881021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:28:34.541804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:28:35.622299+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T08:28:39.211474+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T08:42:49.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:42:49.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:49 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1902037196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:42:49.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:49 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/764851906' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:42:49.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:49 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1902037196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:42:49.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:49 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/764851906' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:42:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.917+0000 7fe976400640 1 -- 172.21.15.3:0/2060120482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe970107aa0 msgr2=0x7fe970107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.917+0000 7fe976400640 1 --2- 172.21.15.3:0/2060120482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe970107aa0 0x7fe970107f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009900 tx=0x7fe95c02bcc0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.917+0000 7fe976400640 1 -- 172.21.15.3:0/2060120482 shutdown_connections 2024-03-21T08:42:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.917+0000 7fe976400640 1 --2- 172.21.15.3:0/2060120482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe970108440 0x7fe970114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.917+0000 7fe976400640 1 --2- 172.21.15.3:0/2060120482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe970107aa0 0x7fe970107f00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.917+0000 7fe976400640 1 --2- 172.21.15.3:0/2060120482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe97010c980 0x7fe97010cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.917+0000 7fe976400640 1 -- 172.21.15.3:0/2060120482 >> 172.21.15.3:0/2060120482 conn(0x7fe9700772a0 msgr2=0x7fe9700796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.918+0000 7fe976400640 1 -- 172.21.15.3:0/2060120482 shutdown_connections 2024-03-21T08:42:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.918+0000 7fe976400640 1 -- 172.21.15.3:0/2060120482 wait complete. 2024-03-21T08:42:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 Processor -- start 2024-03-21T08:42:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 -- start start 2024-03-21T08:42:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe970107aa0 0x7fe97019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe970108440 0x7fe97019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97010c980 0x7fe9701a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970117cf0 con 0x7fe970108440 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970117b70 con 0x7fe97010c980 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe976400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970117e70 con 0x7fe970107aa0 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.919+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe970108440 0x7fe97019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe970108440 0x7fe97019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39970/0 (socket says 172.21.15.3:39970) 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 -- 172.21.15.3:0/2256976337 learned_addr learned my addr 172.21.15.3:0/2256976337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe975a00640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97010c980 0x7fe9701a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe975000640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe970107aa0 0x7fe97019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 -- 172.21.15.3:0/2256976337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe970107aa0 msgr2=0x7fe97019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe970107aa0 0x7fe97019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 -- 172.21.15.3:0/2256976337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97010c980 msgr2=0x7fe9701a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97010c980 0x7fe9701a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 -- 172.21.15.3:0/2256976337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9701a32a0 con 0x7fe970108440 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.920+0000 7fe96fe00640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe970108440 0x7fe97019e7f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe95c005940 tx=0x7fe95c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.921+0000 7fe96d600640 1 -- 172.21.15.3:0/2256976337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c043020 con 0x7fe970108440 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.921+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9701a3530 con 0x7fe970108440 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.921+0000 7fe96d600640 1 -- 172.21.15.3:0/2256976337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe95c02db90 con 0x7fe970108440 2024-03-21T08:42:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.921+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9701aae20 con 0x7fe970108440 2024-03-21T08:42:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.921+0000 7fe96d600640 1 -- 172.21.15.3:0/2256976337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0039b0 con 0x7fe970108440 2024-03-21T08:42:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.922+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97004f400 con 0x7fe970108440 2024-03-21T08:42:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.924+0000 7fe96d600640 1 -- 172.21.15.3:0/2256976337 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7fe95c046020 con 0x7fe970108440 2024-03-21T08:42:49.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:49 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1902037196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:42:49.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:49 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/764851906' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:42:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.926+0000 7fe96d600640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9580a3090 0x7fe9580a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.926+0000 7fe96d600640 1 -- 172.21.15.3:0/2256976337 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7fe95c0f2220 con 0x7fe970108440 2024-03-21T08:42:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.926+0000 7fe975000640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9580a3090 0x7fe9580a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.927+0000 7fe975000640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9580a3090 0x7fe9580a5550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe964009b50 tx=0x7fe964005f90 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:49.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:49.929+0000 7fe96d600640 1 -- 172.21.15.3:0/2256976337 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe95c04d020 con 0x7fe970108440 2024-03-21T08:42:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.134+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe97010ff50 con 0x7fe970108440 2024-03-21T08:42:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.135+0000 7fe96d600640 1 -- 172.21.15.3:0/2256976337 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe95c0be940 con 0x7fe970108440 2024-03-21T08:42:50.135 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-03-21T08:42:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9580a3090 msgr2=0x7fe9580a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9580a3090 0x7fe9580a5550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe964009b50 tx=0x7fe964005f90 comp rx=0 tx=0).stop 2024-03-21T08:42:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe970108440 msgr2=0x7fe97019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe970108440 0x7fe97019e7f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe95c005940 tx=0x7fe95c0045e0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 shutdown_connections 2024-03-21T08:42:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe97010c980 0x7fe9701a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe970108440 0x7fe97019e7f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7fe9580a3090 0x7fe9580a5550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 --2- 172.21.15.3:0/2256976337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe970107aa0 0x7fe97019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 >> 172.21.15.3:0/2256976337 conn(0x7fe9700772a0 msgr2=0x7fe970077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:50.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 shutdown_connections 2024-03-21T08:42:50.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.138+0000 7fe976400640 1 -- 172.21.15.3:0/2256976337 wait complete. 2024-03-21T08:42:50.159 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:42:50.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.390+0000 7f772f000640 1 -- 172.21.15.3:0/97526965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7728100400 msgr2=0x7f772810dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.390+0000 7f772f000640 1 --2- 172.21.15.3:0/97526965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7728100400 0x7f772810dae0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7710009930 tx=0x7f771002bc30 comp rx=0 tx=0).stop 2024-03-21T08:42:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.390+0000 7f772f000640 1 -- 172.21.15.3:0/97526965 shutdown_connections 2024-03-21T08:42:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.390+0000 7f772f000640 1 --2- 172.21.15.3:0/97526965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772810e020 0x7f7728110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.390+0000 7f772f000640 1 --2- 172.21.15.3:0/97526965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7728100400 0x7f772810dae0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.390+0000 7f772f000640 1 --2- 172.21.15.3:0/97526965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77280ffae0 0x7f77280ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.390+0000 7f772f000640 1 -- 172.21.15.3:0/97526965 >> 172.21.15.3:0/97526965 conn(0x7f77280753e0 msgr2=0x7f77280757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:50.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.391+0000 7f772f000640 1 -- 172.21.15.3:0/97526965 shutdown_connections 2024-03-21T08:42:50.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.391+0000 7f772f000640 1 -- 172.21.15.3:0/97526965 wait complete. 2024-03-21T08:42:50.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 Processor -- start 2024-03-21T08:42:50.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 -- start start 2024-03-21T08:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77280ffae0 0x7f772819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7728100400 0x7f772819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f772810e020 0x7f772819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728113ab0 con 0x7f772810e020 2024-03-21T08:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728113930 con 0x7f77280ffae0 2024-03-21T08:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772f000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728113c30 con 0x7f7728100400 2024-03-21T08:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.392+0000 7f772d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f772810e020 0x7f772819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f772810e020 0x7f772819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39978/0 (socket says 172.21.15.3:39978) 2024-03-21T08:42:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 -- 172.21.15.3:0/548629474 learned_addr learned my addr 172.21.15.3:0/548629474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f7727e00640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77280ffae0 0x7f772819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f771fe00640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7728100400 0x7f772819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 -- 172.21.15.3:0/548629474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7728100400 msgr2=0x7f772819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7728100400 0x7f772819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 -- 172.21.15.3:0/548629474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77280ffae0 msgr2=0x7f772819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77280ffae0 0x7f772819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 -- 172.21.15.3:0/548629474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772819f030 con 0x7f772810e020 2024-03-21T08:42:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f7727e00640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77280ffae0 0x7f772819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:42:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772d000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f772810e020 0x7f772819e900 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f772000eb40 tx=0x7f772000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:50.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f7725600640 1 -- 172.21.15.3:0/548629474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720004b20 con 0x7f772810e020 2024-03-21T08:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772819f320 con 0x7f772810e020 2024-03-21T08:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.393+0000 7f7725600640 1 -- 172.21.15.3:0/548629474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7720004cc0 con 0x7f772810e020 2024-03-21T08:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.394+0000 7f7725600640 1 -- 172.21.15.3:0/548629474 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200107b0 con 0x7f772810e020 2024-03-21T08:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.394+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77281a6bb0 con 0x7f772810e020 2024-03-21T08:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.395+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7728068970 con 0x7f772810e020 2024-03-21T08:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.397+0000 7f7725600640 1 -- 172.21.15.3:0/548629474 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f7720005020 con 0x7f772810e020 2024-03-21T08:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.398+0000 7f7725600640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f770c0a3090 0x7f770c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.398+0000 7f7725600640 1 -- 172.21.15.3:0/548629474 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f77200d0560 con 0x7f772810e020 2024-03-21T08:42:50.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.398+0000 7f7727e00640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f770c0a3090 0x7f770c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.399+0000 7f7727e00640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f770c0a3090 0x7f770c0a5550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77180058c0 tx=0x7f7718005850 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:50.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.402+0000 7f7725600640 1 -- 172.21.15.3:0/548629474 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f772009cb70 con 0x7f772810e020 2024-03-21T08:42:50.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.606+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f772802d070 con 0x7f772810e020 2024-03-21T08:42:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.607+0000 7f7725600640 1 -- 172.21.15.3:0/548629474 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f772009c470 con 0x7f772810e020 2024-03-21T08:42:50.608 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:42:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f770c0a3090 msgr2=0x7f770c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f770c0a3090 0x7f770c0a5550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77180058c0 tx=0x7f7718005850 comp rx=0 tx=0).stop 2024-03-21T08:42:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f772810e020 msgr2=0x7f772819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f772810e020 0x7f772819e900 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f772000eb40 tx=0x7f772000c2d0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 shutdown_connections 2024-03-21T08:42:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f772810e020 0x7f772819e900 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f770c0a3090 0x7f770c0a5550 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7728100400 0x7f772819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 --2- 172.21.15.3:0/548629474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77280ffae0 0x7f772819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 >> 172.21.15.3:0/548629474 conn(0x7f77280753e0 msgr2=0x7f772810e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 shutdown_connections 2024-03-21T08:42:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.610+0000 7f772f000640 1 -- 172.21.15.3:0/548629474 wait complete. 2024-03-21T08:42:50.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:42:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 -- 172.21.15.3:0/4263939170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 msgr2=0x7f3174103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 --2- 172.21.15.3:0/4263939170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f3174103cc0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f315c009930 tx=0x7f315c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:42:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 -- 172.21.15.3:0/4263939170 shutdown_connections 2024-03-21T08:42:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 --2- 172.21.15.3:0/4263939170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3174104200 0x7f3174110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 --2- 172.21.15.3:0/4263939170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f3174103cc0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 --2- 172.21.15.3:0/4263939170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3174109850 0x7f3174109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 -- 172.21.15.3:0/4263939170 >> 172.21.15.3:0/4263939170 conn(0x7f31740ff2f0 msgr2=0x7f3174101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 -- 172.21.15.3:0/4263939170 shutdown_connections 2024-03-21T08:42:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.846+0000 7f317a000640 1 -- 172.21.15.3:0/4263939170 wait complete. 2024-03-21T08:42:50.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.847+0000 7f317a000640 1 Processor -- start 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f317a000640 1 -- start start 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f317a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f317419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f317a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3174104200 0x7f317419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f317a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3174109850 0x7f317419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f317a000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3174113ad0 con 0x7f3174103840 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f317a000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3174113950 con 0x7f3174104200 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f317a000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3174113c50 con 0x7f3174109850 2024-03-21T08:42:50.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f3173400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f317419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f3173400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f317419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39998/0 (socket says 172.21.15.3:39998) 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f3173400640 1 -- 172.21.15.3:0/645552250 learned_addr learned my addr 172.21.15.3:0/645552250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.848+0000 7f316aa00640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3174104200 0x7f317419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3173400640 1 -- 172.21.15.3:0/645552250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3174109850 msgr2=0x7f317419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3173e00640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3174109850 0x7f317419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3173400640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3174109850 0x7f317419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3173400640 1 -- 172.21.15.3:0/645552250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3174104200 msgr2=0x7f317419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3173400640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3174104200 0x7f317419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3173400640 1 -- 172.21.15.3:0/645552250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317419f0b0 con 0x7f3174103840 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f316aa00640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3174104200 0x7f317419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3173400640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f317419a0c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f316400d950 tx=0x7f316400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3170c00640 1 -- 172.21.15.3:0/645552250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164017070 con 0x7f3174103840 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3170c00640 1 -- 172.21.15.3:0/645552250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3164009c30 con 0x7f3174103840 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317419f3a0 con 0x7f3174103840 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f3170c00640 1 -- 172.21.15.3:0/645552250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400f620 con 0x7f3174103840 2024-03-21T08:42:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.849+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31741a6c30 con 0x7f3174103840 2024-03-21T08:42:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.851+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3174068970 con 0x7f3174103840 2024-03-21T08:42:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.853+0000 7f3170c00640 1 -- 172.21.15.3:0/645552250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f31640109a0 con 0x7f3174103840 2024-03-21T08:42:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.854+0000 7f3170c00640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f31580a3110 0x7f31580a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.854+0000 7f3170c00640 1 -- 172.21.15.3:0/645552250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f31640cf8b0 con 0x7f3174103840 2024-03-21T08:42:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.854+0000 7f316aa00640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f31580a3110 0x7f31580a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.855+0000 7f316aa00640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f31580a3110 0x7f31580a55d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f315c006010 tx=0x7f315c009610 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:50.859+0000 7f3170c00640 1 -- 172.21.15.3:0/645552250 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3164009de0 con 0x7f3174103840 2024-03-21T08:42:50.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:50 smithi084 ceph-mon[35477]: pgmap v342: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:50.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:50 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/3230104215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:42:50.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:50 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2256976337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:42:50.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:50 smithi171 ceph-mon[35388]: pgmap v342: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:50.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:50 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/3230104215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:42:50.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:50 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2256976337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:42:50.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:50 smithi003 ceph-mon[31675]: pgmap v342: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:50.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:50 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/3230104215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:42:50.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:50 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2256976337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.064+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f317402d070 con 0x7f3174103840 2024-03-21T08:42:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.065+0000 7f3170c00640 1 -- 172.21.15.3:0/645552250 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f316409bec0 con 0x7f3174103840 2024-03-21T08:42:51.066 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-03-21T08:42:51.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f31580a3110 msgr2=0x7f31580a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f31580a3110 0x7f31580a55d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f315c006010 tx=0x7f315c009610 comp rx=0 tx=0).stop 2024-03-21T08:42:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 msgr2=0x7f317419a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f317419a0c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f316400d950 tx=0x7f316400de10 comp rx=0 tx=0).stop 2024-03-21T08:42:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 shutdown_connections 2024-03-21T08:42:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f31580a3110 0x7f31580a55d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3174109850 0x7f317419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3174104200 0x7f317419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 --2- 172.21.15.3:0/645552250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3174103840 0x7f317419a0c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 >> 172.21.15.3:0/645552250 conn(0x7f31740ff2f0 msgr2=0x7f31740ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 shutdown_connections 2024-03-21T08:42:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.068+0000 7f317a000640 1 -- 172.21.15.3:0/645552250 wait complete. 2024-03-21T08:42:51.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T08:42:51.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 -- 172.21.15.3:0/2725831681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46641074e0 msgr2=0x7f46641078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 --2- 172.21.15.3:0/2725831681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46641074e0 0x7f46641078c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4658009930 tx=0x7f465802bc20 comp rx=0 tx=0).stop 2024-03-21T08:42:51.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 -- 172.21.15.3:0/2725831681 shutdown_connections 2024-03-21T08:42:51.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 --2- 172.21.15.3:0/2725831681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664101e00 0x7f466410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 --2- 172.21.15.3:0/2725831681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4664101440 0x7f46641018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 --2- 172.21.15.3:0/2725831681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46641074e0 0x7f46641078c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 -- 172.21.15.3:0/2725831681 >> 172.21.15.3:0/2725831681 conn(0x7f46640fcf50 msgr2=0x7f46640ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:51.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 -- 172.21.15.3:0/2725831681 shutdown_connections 2024-03-21T08:42:51.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.317+0000 7f466a000640 1 -- 172.21.15.3:0/2725831681 wait complete. 2024-03-21T08:42:51.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 Processor -- start 2024-03-21T08:42:51.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 -- start start 2024-03-21T08:42:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664101440 0x7f4664199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4664101e00 0x7f466419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46641074e0 0x7f466419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46641116b0 con 0x7f4664101e00 2024-03-21T08:42:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664111530 con 0x7f46641074e0 2024-03-21T08:42:51.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.319+0000 7f466a000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664111830 con 0x7f4664101440 2024-03-21T08:42:51.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4664101e00 0x7f466419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4664101e00 0x7f466419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40016/0 (socket says 172.21.15.3:40016) 2024-03-21T08:42:51.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 -- 172.21.15.3:0/1034547845 learned_addr learned my addr 172.21.15.3:0/1034547845 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:51.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4668c00640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664101440 0x7f4664199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4669600640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46641074e0 0x7f466419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 -- 172.21.15.3:0/1034547845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664101440 msgr2=0x7f4664199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664101440 0x7f4664199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 -- 172.21.15.3:0/1034547845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46641074e0 msgr2=0x7f466419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46641074e0 0x7f466419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.320+0000 7f4663e00640 1 -- 172.21.15.3:0/1034547845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466419eed0 con 0x7f4664101e00 2024-03-21T08:42:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.321+0000 7f4663e00640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4664101e00 0x7f466419a420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f465000d950 tx=0x7f465000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.321+0000 7f4661600640 1 -- 172.21.15.3:0/1034547845 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650017070 con 0x7f4664101e00 2024-03-21T08:42:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.321+0000 7f4661600640 1 -- 172.21.15.3:0/1034547845 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4650009c30 con 0x7f4664101e00 2024-03-21T08:42:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.321+0000 7f4661600640 1 -- 172.21.15.3:0/1034547845 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465000f620 con 0x7f4664101e00 2024-03-21T08:42:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.321+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466419f1c0 con 0x7f4664101e00 2024-03-21T08:42:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.321+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46641a6a50 con 0x7f4664101e00 2024-03-21T08:42:51.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.323+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466404f400 con 0x7f4664101e00 2024-03-21T08:42:51.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.326+0000 7f4661600640 1 -- 172.21.15.3:0/1034547845 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f46500109a0 con 0x7f4664101e00 2024-03-21T08:42:51.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.328+0000 7f4661600640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f464c0a3110 0x7f464c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.328+0000 7f4661600640 1 -- 172.21.15.3:0/1034547845 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f46500cf710 con 0x7f4664101e00 2024-03-21T08:42:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.329+0000 7f4668c00640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f464c0a3110 0x7f464c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.329+0000 7f4668c00640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f464c0a3110 0x7f464c0a55d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4658005ab0 tx=0x7f4658005fd0 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.330+0000 7f4661600640 1 -- 172.21.15.3:0/1034547845 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4650009de0 con 0x7f4664101e00 2024-03-21T08:42:51.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.532+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f46641a6d50 con 0x7f4664101e00 2024-03-21T08:42:51.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.533+0000 7f4661600640 1 -- 172.21.15.3:0/1034547845 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) v1 ==== 103+0+11 (secure 0 0 0) 0x7f465009b7b0 con 0x7f4664101e00 2024-03-21T08:42:51.534 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-03-21T08:42:51.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f464c0a3110 msgr2=0x7f464c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f464c0a3110 0x7f464c0a55d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4658005ab0 tx=0x7f4658005fd0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4664101e00 msgr2=0x7f466419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4664101e00 0x7f466419a420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f465000d950 tx=0x7f465000de10 comp rx=0 tx=0).stop 2024-03-21T08:42:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 shutdown_connections 2024-03-21T08:42:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46641074e0 0x7f466419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4664101e00 0x7f466419a420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f464c0a3110 0x7f464c0a55d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 --2- 172.21.15.3:0/1034547845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4664101440 0x7f4664199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 >> 172.21.15.3:0/1034547845 conn(0x7f46640fcf50 msgr2=0x7f466410cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:51.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 shutdown_connections 2024-03-21T08:42:51.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.537+0000 7f466a000640 1 -- 172.21.15.3:0/1034547845 wait complete. 2024-03-21T08:42:51.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T08:42:51.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.788+0000 7f51e2000640 1 -- 172.21.15.3:0/2043439108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 msgr2=0x7f51dc109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.788+0000 7f51e2000640 1 --2- 172.21.15.3:0/2043439108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc109c50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009930 tx=0x7f51cc02bc20 comp rx=0 tx=0).stop 2024-03-21T08:42:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.789+0000 7f51e2000640 1 -- 172.21.15.3:0/2043439108 shutdown_connections 2024-03-21T08:42:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.789+0000 7f51e2000640 1 --2- 172.21.15.3:0/2043439108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51dc104220 0x7f51dc110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.789+0000 7f51e2000640 1 --2- 172.21.15.3:0/2043439108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc103860 0x7f51dc103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.789+0000 7f51e2000640 1 --2- 172.21.15.3:0/2043439108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc109c50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.789+0000 7f51e2000640 1 -- 172.21.15.3:0/2043439108 >> 172.21.15.3:0/2043439108 conn(0x7f51dc0ff310 msgr2=0x7f51dc101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.789+0000 7f51e2000640 1 -- 172.21.15.3:0/2043439108 shutdown_connections 2024-03-21T08:42:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.789+0000 7f51e2000640 1 -- 172.21.15.3:0/2043439108 wait complete. 2024-03-21T08:42:51.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 Processor -- start 2024-03-21T08:42:51.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 -- start start 2024-03-21T08:42:51.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51dc103860 0x7f51dc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc104220 0x7f51dc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc113b80 con 0x7f51dc109870 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc113a00 con 0x7f51dc104220 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51e2000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc113d00 con 0x7f51dc103860 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40040/0 (socket says 172.21.15.3:40040) 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.790+0000 7f51dbe00640 1 -- 172.21.15.3:0/2833508758 learned_addr learned my addr 172.21.15.3:0/2833508758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51daa00640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc104220 0x7f51dc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51db400640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51dc103860 0x7f51dc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51daa00640 1 -- 172.21.15.3:0/2833508758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51dc103860 msgr2=0x7f51dc19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51daa00640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51dc103860 0x7f51dc19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51daa00640 1 -- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 msgr2=0x7f51dc19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51daa00640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51daa00640 1 -- 172.21.15.3:0/2833508758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc19f110 con 0x7f51dc104220 2024-03-21T08:42:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51dbe00640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:42:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.791+0000 7f51daa00640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc104220 0x7f51dc19a660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.792+0000 7f51d3e00640 1 -- 172.21.15.3:0/2833508758 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4009c30 con 0x7f51dc104220 2024-03-21T08:42:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.792+0000 7f51d3e00640 1 -- 172.21.15.3:0/2833508758 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51c4009dd0 con 0x7f51dc104220 2024-03-21T08:42:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.792+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc19f400 con 0x7f51dc104220 2024-03-21T08:42:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.792+0000 7f51d3e00640 1 -- 172.21.15.3:0/2833508758 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4003e40 con 0x7f51dc104220 2024-03-21T08:42:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.792+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc1a6c90 con 0x7f51dc104220 2024-03-21T08:42:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.793+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc068970 con 0x7f51dc104220 2024-03-21T08:42:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.796+0000 7f51d3e00640 1 -- 172.21.15.3:0/2833508758 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 26) v1 ==== 137984+0+0 (secure 0 0 0) 0x7f51c4004000 con 0x7f51dc104220 2024-03-21T08:42:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.797+0000 7f51d3e00640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51c00a3090 0x7f51c00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:42:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.797+0000 7f51d3e00640 1 -- 172.21.15.3:0/2833508758 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9670+0+0 (secure 0 0 0) 0x7f51c40cf860 con 0x7f51dc104220 2024-03-21T08:42:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.797+0000 7f51db400640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51c00a3090 0x7f51c00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:42:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.798+0000 7f51db400640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51c00a3090 0x7f51c00a5550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009900 tx=0x7f51cc005a20 comp rx=0 tx=0).ready entity=mgr.15654 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:42:51.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:51.810+0000 7f51d3e00640 1 -- 172.21.15.3:0/2833508758 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f51c4017020 con 0x7f51dc104220 2024-03-21T08:42:51.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:51 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/548629474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:51 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/645552250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.876 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:51 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/1034547845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:51 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/548629474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:51 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/645552250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:51 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/1034547845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:51 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/548629474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:51 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/645552250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:42:51.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:51 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/1034547845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T08:42:52.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.027+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51dc19b470 con 0x7f51dc104220 2024-03-21T08:42:52.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.028+0000 7f51d3e00640 1 -- 172.21.15.3:0/2833508758 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) v1 ==== 81+0+927 (secure 0 0 0) 0x7f51dc19b470 con 0x7f51dc104220 2024-03-21T08:42:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3880262252 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:52.029 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/62217793 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1599875703 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2117697938 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2293101955 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1520535032 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:52.030 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/2168061035 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1378601347 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1115678781 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/404577225 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1520535032 2024-03-22T08:31:50.557523+0000 2024-03-21T08:42:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1597279144 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:52.032 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4112229601 2024-03-22T08:18:30.424775+0000 2024-03-21T08:42:52.032 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2051354447 2024-03-22T08:19:05.157913+0000 2024-03-21T08:42:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:listed 16 entries 2024-03-21T08:42:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51c00a3090 msgr2=0x7f51c00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51c00a3090 0x7f51c00a5550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009900 tx=0x7f51cc005a20 comp rx=0 tx=0).stop 2024-03-21T08:42:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc104220 msgr2=0x7f51dc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:42:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc104220 0x7f51dc19a660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c400dde0 comp rx=0 tx=0).stop 2024-03-21T08:42:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 shutdown_connections 2024-03-21T08:42:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51dc109870 0x7f51dc19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51dc104220 0x7f51dc19a660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.3:6800/1070134681,v1:172.21.15.3:6801/1070134681] conn(0x7f51c00a3090 0x7f51c00a5550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 --2- 172.21.15.3:0/2833508758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51dc103860 0x7f51dc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:42:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 >> 172.21.15.3:0/2833508758 conn(0x7f51dc0ff310 msgr2=0x7f51dc0ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:42:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.031+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 shutdown_connections 2024-03-21T08:42:52.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-03-21T08:42:52.032+0000 7f51e2000640 1 -- 172.21.15.3:0/2833508758 wait complete. 2024-03-21T08:42:52.052 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-03-21T08:42:52.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:42:52.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-21T08:42:52.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[85526]: fuse finished with error 0 and tester_r 0 2024-03-21T08:42:52.204 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T08:42:52.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:52 smithi084 ceph-mon[35477]: pgmap v343: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:52.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:52 smithi084 ceph-mon[35477]: from='client.? 172.21.15.3:0/2833508758' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:42:52.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:52 smithi171 ceph-mon[35388]: pgmap v343: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:52.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:52 smithi171 ceph-mon[35388]: from='client.? 172.21.15.3:0/2833508758' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:42:52.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:52 smithi003 ceph-mon[31675]: pgmap v343: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:52.927 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:52 smithi003 ceph-mon[31675]: from='client.? 172.21.15.3:0/2833508758' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T08:42:54.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:54 smithi084 ceph-mon[35477]: pgmap v344: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:54.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:54 smithi171 ceph-mon[35388]: pgmap v344: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:54.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:54 smithi003 ceph-mon[31675]: pgmap v344: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T08:42:56.875 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:56 smithi084 ceph-mon[35477]: pgmap v345: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:56.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:56 smithi171 ceph-mon[35388]: pgmap v345: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:56.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:56 smithi003 ceph-mon[31675]: pgmap v345: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T08:42:58.306 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:42:58.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:42:58.307 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-21T08:42:58.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:42:58.372 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-03-21T08:42:58.428 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-21T08:42:58.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:42:58.429 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:42:58.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:42:58.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-03-21T08:42:58.649 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T08:42:58.660 INFO:tasks.cephadm:Teardown begin 2024-03-21T08:42:58.660 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/cephadm.py", line 1867, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-21T08:42:58.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:42:58.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:42:58.831 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:42:58 smithi084 ceph-mon[35477]: pgmap v346: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-21T08:42:58.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:42:58.882 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:42:58 smithi171 ceph-mon[35388]: pgmap v346: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-21T08:42:58.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:58 smithi003 ceph-mon[31675]: pgmap v346: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-21T08:42:58.934 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T08:42:58.934 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T08:42:58.995 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T08:42:59.039 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T08:42:59.095 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T08:42:59.095 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-21T08:42:59.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a 2024-03-21T08:42:59.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:59 smithi003 systemd[1]: Stopping Ceph mon.a for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:00.127 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31671]: 2024-03-21T08:42:59.730+0000 7efc92c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:00.128 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a[31671]: 2024-03-21T08:42:59.730+0000 7efc92c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T08:43:00.128 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:42:59 smithi003 podman[102346]: 2024-03-21 08:42:59.805928701 +0000 UTC m=+0.226951656 container died 0b9774172d25aec8d35683bbbba921245e6dac9379211ddf8365022e10c874f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T08:43:00.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:43:00 smithi003 podman[102346]: 2024-03-21 08:43:00.228703831 +0000 UTC m=+0.649726789 container cleanup 0b9774172d25aec8d35683bbbba921245e6dac9379211ddf8365022e10c874f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:43:00.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:43:00 smithi003 bash[102346]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a 2024-03-21T08:43:00.926 INFO:journalctl@ceph.mon.a.smithi003.stdout:Mar 21 08:43:00 smithi003 podman[102360]: 2024-03-21 08:43:00.463107504 +0000 UTC m=+0.642834935 container remove 0b9774172d25aec8d35683bbbba921245e6dac9379211ddf8365022e10c874f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T08:43:01.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.a.service' 2024-03-21T08:43:01.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:01.175 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-21T08:43:01.175 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-21T08:43:01.175 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.b 2024-03-21T08:43:01.624 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:43:01 smithi084 systemd[1]: Stopping Ceph mon.b for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:43:01 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b[35473]: 2024-03-21T08:43:01.806+0000 7f8dd4600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:43:01 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b[35473]: 2024-03-21T08:43:01.806+0000 7f8dd4600700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-21T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:43:01 smithi084 podman[71199]: 2024-03-21 08:43:01.869082932 +0000 UTC m=+0.254168669 container died 9f1b9919572d1ded4e001299cfd906db9a7e46637aaf18858ef3648525ccc14e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T08:43:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:43:02 smithi084 podman[71199]: 2024-03-21 08:43:02.432353595 +0000 UTC m=+0.817439383 container cleanup 9f1b9919572d1ded4e001299cfd906db9a7e46637aaf18858ef3648525ccc14e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T08:43:02.625 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:43:02 smithi084 bash[71199]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b 2024-03-21T08:43:03.014 INFO:journalctl@ceph.mon.b.smithi084.stdout:Mar 21 08:43:02 smithi084 podman[71212]: 2024-03-21 08:43:02.68402446 +0000 UTC m=+0.806620852 container remove 9f1b9919572d1ded4e001299cfd906db9a7e46637aaf18858ef3648525ccc14e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:03.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.b.service' 2024-03-21T08:43:03.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:03.152 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-21T08:43:03.152 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-21T08:43:03.153 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.c 2024-03-21T08:43:03.633 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:43:03 smithi171 systemd[1]: Stopping Ceph mon.c for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:04.082 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:43:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c[35384]: 2024-03-21T08:43:03.690+0000 7f6ab8000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:04.083 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:43:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c[35384]: 2024-03-21T08:43:03.690+0000 7f6ab8000700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T08:43:04.083 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:43:03 smithi171 podman[73454]: 2024-03-21 08:43:03.81487613 +0000 UTC m=+0.281190077 container died 5ae9e19f644ea5c5932a2549f4d4cce050ed8060f70831d2dc14c23d95bf9c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T08:43:04.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:43:04 smithi171 podman[73454]: 2024-03-21 08:43:04.284072192 +0000 UTC m=+0.750386153 container cleanup 5ae9e19f644ea5c5932a2549f4d4cce050ed8060f70831d2dc14c23d95bf9c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:43:04.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:43:04 smithi171 bash[73454]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c 2024-03-21T08:43:04.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 21 08:43:04 smithi171 podman[73468]: 2024-03-21 08:43:04.552638553 +0000 UTC m=+0.716578084 container remove 5ae9e19f644ea5c5932a2549f4d4cce050ed8060f70831d2dc14c23d95bf9c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mon-c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:05.025 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mon.c.service' 2024-03-21T08:43:05.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:05.158 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-21T08:43:05.158 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-21T08:43:05.158 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.x 2024-03-21T08:43:05.676 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:43:05 smithi003 systemd[1]: Stopping Ceph mgr.x for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:06.176 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:43:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:43:05.781+0000 7f7181800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:06.176 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:43:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x[31901]: 2024-03-21T08:43:05.781+0000 7f7181800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-21T08:43:06.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:43:05 smithi003 podman[102467]: 2024-03-21 08:43:05.876166182 +0000 UTC m=+0.260531708 container died 4a0ec4f047a25633a3bad7a7e56516352534808ca967d4e493f69eaddc22b992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:06.676 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:43:06 smithi003 podman[102467]: 2024-03-21 08:43:06.50844295 +0000 UTC m=+0.892808432 container cleanup 4a0ec4f047a25633a3bad7a7e56516352534808ca967d4e493f69eaddc22b992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2) 2024-03-21T08:43:06.676 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:43:06 smithi003 bash[102467]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x 2024-03-21T08:43:07.176 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Mar 21 08:43:06 smithi003 podman[102481]: 2024-03-21 08:43:06.718322492 +0000 UTC m=+0.830704445 container remove 4a0ec4f047a25633a3bad7a7e56516352534808ca967d4e493f69eaddc22b992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-21T08:43:07.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-21T08:43:07.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:07.432 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-21T08:43:07.433 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-21T08:43:07.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.y 2024-03-21T08:43:07.875 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:43:07 smithi084 systemd[1]: Stopping Ceph mgr.y for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:08.484 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:43:08 smithi084 podman[71330]: 2024-03-21 08:43:08.129149494 +0000 UTC m=+0.313478348 container died 8fa6f4e455c9e770e487c633613b7be5ddfa88d5cf93b194f1b07c40db44c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T08:43:08.875 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:43:08 smithi084 podman[71330]: 2024-03-21 08:43:08.654167624 +0000 UTC m=+0.838496493 container cleanup 8fa6f4e455c9e770e487c633613b7be5ddfa88d5cf93b194f1b07c40db44c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:43:08.875 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:43:08 smithi084 bash[71330]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y 2024-03-21T08:43:09.257 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:43:08 smithi084 podman[71345]: 2024-03-21 08:43:08.92586881 +0000 UTC m=+0.788142250 container remove 8fa6f4e455c9e770e487c633613b7be5ddfa88d5cf93b194f1b07c40db44c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T08:43:09.258 INFO:journalctl@ceph.mgr.y.smithi084.stdout:Mar 21 08:43:08 smithi084 systemd[1]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-21T08:43:09.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-21T08:43:09.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:09.377 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-21T08:43:09.377 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-21T08:43:09.378 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.z 2024-03-21T08:43:09.883 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:43:09 smithi171 systemd[1]: Stopping Ceph mgr.z for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:10.354 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:43:10 smithi171 podman[73588]: 2024-03-21 08:43:10.021230304 +0000 UTC m=+0.232978463 container died 2645ed24aaae1b1b9a49fb19b16532413b458e3002836d63b67c62b757f61fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-21T08:43:10.605 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:43:10 smithi171 podman[73588]: 2024-03-21 08:43:10.581574937 +0000 UTC m=+0.793323084 container cleanup 2645ed24aaae1b1b9a49fb19b16532413b458e3002836d63b67c62b757f61fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:43:10.606 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:43:10 smithi171 bash[73588]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z 2024-03-21T08:43:11.133 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:43:10 smithi171 podman[73602]: 2024-03-21 08:43:10.809126977 +0000 UTC m=+0.780607039 container remove 2645ed24aaae1b1b9a49fb19b16532413b458e3002836d63b67c62b757f61fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T08:43:11.133 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 21 08:43:10 smithi171 systemd[1]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-21T08:43:11.298 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-21T08:43:11.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:11.417 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-21T08:43:11.418 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T08:43:11.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.0 2024-03-21T08:43:11.925 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:11 smithi003 systemd[1]: Stopping Ceph osd.0 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:12.426 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0[46192]: 2024-03-21T08:43:12.020+0000 7f1a16600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:12.426 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0[46192]: 2024-03-21T08:43:12.020+0000 7f1a16600700 -1 osd.0 107 *** Got signal Terminated *** 2024-03-21T08:43:12.427 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0[46192]: 2024-03-21T08:43:12.020+0000 7f1a16600700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:43:17.926 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:17 smithi003 podman[102586]: 2024-03-21 08:43:17.602666671 +0000 UTC m=+5.732240134 container died cf4fe1fd78f6d84149f963868e397c859a4a1a88c20bccd5a44035db5e48471d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T08:43:18.676 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:18 smithi003 podman[102586]: 2024-03-21 08:43:18.271140167 +0000 UTC m=+6.400713677 container cleanup cf4fe1fd78f6d84149f963868e397c859a4a1a88c20bccd5a44035db5e48471d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:18.677 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:18 smithi003 bash[102586]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0 2024-03-21T08:43:18.677 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:18 smithi003 podman[102619]: 2024-03-21 08:43:18.480773855 +0000 UTC m=+0.864396242 container remove cf4fe1fd78f6d84149f963868e397c859a4a1a88c20bccd5a44035db5e48471d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T08:43:19.676 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:19 smithi003 podman[102707]: 2024-03-21 08:43:19.280725312 +0000 UTC m=+0.037608729 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:43:19.677 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:19 smithi003 podman[102707]: 2024-03-21 08:43:19.39015353 +0000 UTC m=+0.147036981 container create 74bedca7a47a681fa348925812a868d482e66133909e4dda3040c1d5e5757528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:20.150 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:19 smithi003 podman[102707]: 2024-03-21 08:43:19.685782148 +0000 UTC m=+0.442665605 container init 74bedca7a47a681fa348925812a868d482e66133909e4dda3040c1d5e5757528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-21T08:43:20.150 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:19 smithi003 podman[102707]: 2024-03-21 08:43:19.697618603 +0000 UTC m=+0.454502060 container start 74bedca7a47a681fa348925812a868d482e66133909e4dda3040c1d5e5757528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T08:43:20.150 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:19 smithi003 podman[102707]: 2024-03-21 08:43:19.801255369 +0000 UTC m=+0.558138873 container attach 74bedca7a47a681fa348925812a868d482e66133909e4dda3040c1d5e5757528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:43:20.151 INFO:journalctl@ceph.osd.0.smithi003.stdout:Mar 21 08:43:20 smithi003 podman[102707]: 2024-03-21 08:43:20.149343958 +0000 UTC m=+0.906227459 container died 74bedca7a47a681fa348925812a868d482e66133909e4dda3040c1d5e5757528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T08:43:21.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T08:43:21.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:21.223 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T08:43:21.223 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T08:43:21.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.1 2024-03-21T08:43:21.625 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:21 smithi084 systemd[1]: Stopping Ceph osd.1 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:22.125 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1[40070]: 2024-03-21T08:43:21.747+0000 7f7c39600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:22.125 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1[40070]: 2024-03-21T08:43:21.747+0000 7f7c39600700 -1 osd.1 107 *** Got signal Terminated *** 2024-03-21T08:43:22.125 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1[40070]: 2024-03-21T08:43:21.747+0000 7f7c39600700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:43:27.625 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:27 smithi084 podman[71460]: 2024-03-21 08:43:27.348215164 +0000 UTC m=+5.757754901 container died de968d5cfdee63ff4a8328a88251438a1d046fa5a68f390aae78d889a33a8625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T08:43:28.101 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:27 smithi084 podman[71460]: 2024-03-21 08:43:27.849348569 +0000 UTC m=+6.258888293 container cleanup de968d5cfdee63ff4a8328a88251438a1d046fa5a68f390aae78d889a33a8625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T08:43:28.101 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:27 smithi084 bash[71460]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1 2024-03-21T08:43:28.374 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:28 smithi084 podman[71478]: 2024-03-21 08:43:28.101165199 +0000 UTC m=+0.739746144 container remove de968d5cfdee63ff4a8328a88251438a1d046fa5a68f390aae78d889a33a8625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T08:43:29.124 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:28 smithi084 podman[71561]: 2024-03-21 08:43:28.783054979 +0000 UTC m=+0.042784616 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:43:29.125 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:28 smithi084 podman[71561]: 2024-03-21 08:43:28.935809743 +0000 UTC m=+0.195539404 container create 58fbfd75da9baa0986f98c7dbf39d8c430047971b868cf41bcfaae1da0374d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:29.625 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:29 smithi084 podman[71561]: 2024-03-21 08:43:29.224101852 +0000 UTC m=+0.483831471 container init 58fbfd75da9baa0986f98c7dbf39d8c430047971b868cf41bcfaae1da0374d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T08:43:29.625 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:29 smithi084 podman[71561]: 2024-03-21 08:43:29.237350423 +0000 UTC m=+0.497080037 container start 58fbfd75da9baa0986f98c7dbf39d8c430047971b868cf41bcfaae1da0374d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T08:43:29.625 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:29 smithi084 podman[71561]: 2024-03-21 08:43:29.313378188 +0000 UTC m=+0.573107804 container attach 58fbfd75da9baa0986f98c7dbf39d8c430047971b868cf41bcfaae1da0374d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:43:30.070 INFO:journalctl@ceph.osd.1.smithi084.stdout:Mar 21 08:43:29 smithi084 podman[71561]: 2024-03-21 08:43:29.7142429 +0000 UTC m=+0.973972507 container died 58fbfd75da9baa0986f98c7dbf39d8c430047971b868cf41bcfaae1da0374d54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-21T08:43:30.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T08:43:30.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:30.609 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T08:43:30.609 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T08:43:30.610 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.2 2024-03-21T08:43:31.133 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:30 smithi171 systemd[1]: Stopping Ceph osd.2 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:31.633 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2[40491]: 2024-03-21T08:43:31.203+0000 7f6722000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:31.633 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2[40491]: 2024-03-21T08:43:31.203+0000 7f6722000700 -1 osd.2 107 *** Got signal Terminated *** 2024-03-21T08:43:31.633 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2[40491]: 2024-03-21T08:43:31.203+0000 7f6722000700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:43:37.042 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:36 smithi171 podman[73711]: 2024-03-21 08:43:36.683214593 +0000 UTC m=+5.635888469 container died 1da8db4d81f7a4572522a3e7ef7463f76bb3ae41f17ff1203feb9d3638c48fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T08:43:37.373 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:37 smithi171 podman[73711]: 2024-03-21 08:43:37.196717181 +0000 UTC m=+6.149391097 container cleanup 1da8db4d81f7a4572522a3e7ef7463f76bb3ae41f17ff1203feb9d3638c48fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-21T08:43:37.374 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:37 smithi171 bash[73711]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2 2024-03-21T08:43:37.633 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:37 smithi171 podman[73731]: 2024-03-21 08:43:37.372915049 +0000 UTC m=+0.676358504 container remove 1da8db4d81f7a4572522a3e7ef7463f76bb3ae41f17ff1203feb9d3638c48fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:38.383 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:38 smithi171 podman[73817]: 2024-03-21 08:43:38.105593747 +0000 UTC m=+0.034669365 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:43:38.383 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:38 smithi171 podman[73817]: 2024-03-21 08:43:38.263865253 +0000 UTC m=+0.192940898 container create da30330108524abd69839c94ce996580064c0b1e0690848bfcd08cc978f0843c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:43:38.676 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:38 smithi171 podman[73817]: 2024-03-21 08:43:38.535365378 +0000 UTC m=+0.464440998 container init da30330108524abd69839c94ce996580064c0b1e0690848bfcd08cc978f0843c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:43:38.677 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:38 smithi171 podman[73817]: 2024-03-21 08:43:38.543851873 +0000 UTC m=+0.472927495 container start da30330108524abd69839c94ce996580064c0b1e0690848bfcd08cc978f0843c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T08:43:38.677 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:38 smithi171 podman[73817]: 2024-03-21 08:43:38.676376716 +0000 UTC m=+0.605452422 container attach da30330108524abd69839c94ce996580064c0b1e0690848bfcd08cc978f0843c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-21T08:43:38.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:38 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:38.608+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:39.367 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 21 08:43:38 smithi171 podman[73817]: 2024-03-21 08:43:38.963791052 +0000 UTC m=+0.892866734 container died da30330108524abd69839c94ce996580064c0b1e0690848bfcd08cc978f0843c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:43:39.765 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T08:43:39.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:39.881 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T08:43:39.881 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T08:43:39.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.3 2024-03-21T08:43:39.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:39 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:39.615+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:40.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:39.843+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:40.378 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:40 smithi003 systemd[1]: Stopping Ceph osd.3 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:40.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:40.315+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:40.675 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:40.624+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:40.676 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:40.377+0000 7f3051e00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.589607+0000 front 2024-03-21T08:43:15.589692+0000 (oldest deadline 2024-03-21T08:43:40.190021+0000) 2024-03-21T08:43:40.676 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:40.437+0000 7f305b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:40.676 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:40.437+0000 7f305b600700 -1 osd.3 107 *** Got signal Terminated *** 2024-03-21T08:43:40.676 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:40.437+0000 7f305b600700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:43:41.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:40 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:40.848+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:41.275 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:40.956+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:41.276 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:40.942+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:41.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:41.274+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:41.642 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:41 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:41.334+0000 7f3051e00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.589607+0000 front 2024-03-21T08:43:15.589692+0000 (oldest deadline 2024-03-21T08:43:40.190021+0000) 2024-03-21T08:43:41.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:41 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:41.641+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:41.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:41 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:41.674+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:42.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:41.810+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:42.230 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:41.943+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:42.231 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:41.899+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:42.593 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:42 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:42.285+0000 7f3051e00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.589607+0000 front 2024-03-21T08:43:15.589692+0000 (oldest deadline 2024-03-21T08:43:40.190021+0000) 2024-03-21T08:43:42.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:42.230+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:42.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:42 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:42.593+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:42.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:42 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:42.658+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:43.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:42.770+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:43.244 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:42.961+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:43.244 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:42.928+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:43.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:43.243+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:43.660 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:43.328+0000 7f3051e00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.589607+0000 front 2024-03-21T08:43:15.589692+0000 (oldest deadline 2024-03-21T08:43:40.190021+0000) 2024-03-21T08:43:43.661 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:43.566+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:43.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:43 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:43.659+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:44.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:43.811+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:44.241 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:43.920+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:44.241 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:43.941+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:44.613 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:44.312+0000 7f3051e00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.589607+0000 front 2024-03-21T08:43:15.589692+0000 (oldest deadline 2024-03-21T08:43:40.190021+0000) 2024-03-21T08:43:44.613 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:44.521+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:44.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:44.240+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:44.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:44 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:44.612+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:45.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:44.785+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:45.269 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:44.894+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:45.273 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:44.977+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:45.629 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:45 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3[53156]: 2024-03-21T08:43:45.297+0000 7f3051e00700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.589607+0000 front 2024-03-21T08:43:15.589692+0000 (oldest deadline 2024-03-21T08:43:40.190021+0000) 2024-03-21T08:43:45.630 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:45 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:45.524+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:45.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:45.269+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:45.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:45 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:45.626+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:46.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:45.820+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:46.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:45.901+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:46.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:46.022+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:46.397 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:46 smithi003 podman[102811]: 2024-03-21 08:43:46.072768985 +0000 UTC m=+5.749992061 container died 990ff090437affe6f560c7c6b1b2c67ed1d9f1e761cf62ca10df64e269b46d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-21T08:43:46.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:46.249+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:46.673 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:46 smithi003 podman[102811]: 2024-03-21 08:43:46.51465827 +0000 UTC m=+6.191881331 container cleanup 990ff090437affe6f560c7c6b1b2c67ed1d9f1e761cf62ca10df64e269b46d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:43:46.673 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:46 smithi003 bash[102811]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3 2024-03-21T08:43:46.674 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:46 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:46.574+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:46.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:46 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:46.672+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:46.926 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:46 smithi003 podman[102829]: 2024-03-21 08:43:46.807835895 +0000 UTC m=+0.724203876 container remove 990ff090437affe6f560c7c6b1b2c67ed1d9f1e761cf62ca10df64e269b46d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:43:47.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:46.830+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:47.298 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:46.949+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:47.298 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:47.005+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:47.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:47.297+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:47.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:47 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:47.581+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:47.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:47 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:47.709+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:47.926 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:47 smithi003 podman[102914]: 2024-03-21 08:43:47.594326313 +0000 UTC m=+0.040307103 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:43:47.926 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:47 smithi003 podman[102914]: 2024-03-21 08:43:47.716883703 +0000 UTC m=+0.162864505 container create bae3be1e13f51de42162d08afd95f759b864f1cc6b7888f5fdfc426e8642fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:48.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:47.865+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:48.176 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:48 smithi003 podman[102914]: 2024-03-21 08:43:48.04231978 +0000 UTC m=+0.488300619 container init bae3be1e13f51de42162d08afd95f759b864f1cc6b7888f5fdfc426e8642fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T08:43:48.176 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:48 smithi003 podman[102914]: 2024-03-21 08:43:48.054575484 +0000 UTC m=+0.500556262 container start bae3be1e13f51de42162d08afd95f759b864f1cc6b7888f5fdfc426e8642fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T08:43:48.176 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:48 smithi003 podman[102914]: 2024-03-21 08:43:48.136274108 +0000 UTC m=+0.582254951 container attach bae3be1e13f51de42162d08afd95f759b864f1cc6b7888f5fdfc426e8642fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T08:43:48.263 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:47.994+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:48.263 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:47.931+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:48.263 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:47.931+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:48.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:48.261+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:48.784 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:48 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:48.671+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:48.784 INFO:journalctl@ceph.osd.3.smithi003.stdout:Mar 21 08:43:48 smithi003 podman[102914]: 2024-03-21 08:43:48.504347707 +0000 UTC m=+0.950328553 container died bae3be1e13f51de42162d08afd95f759b864f1cc6b7888f5fdfc426e8642fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:43:48.785 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:48 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:48.570+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:48.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:48.502+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:49.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T08:43:49.285 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:49.027+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:49.286 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:49.027+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:49.286 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:48.887+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:49.286 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:48.887+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:49.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:49.290 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T08:43:49.291 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T08:43:49.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.4 2024-03-21T08:43:49.365 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:48.902+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:49.624 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:49.488+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:49.625 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:49 smithi084 systemd[1]: Stopping Ceph osd.4 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:49.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:49.284+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:49.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:49.284+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:49.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:49 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:49.560+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:49.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:49 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:49.676+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:50.071 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:49.842+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:50.072 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:49.842+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:50.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:49.934+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:50.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:49.934+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:50.125 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:49.768+0000 7ff3d5600700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:26.141762+0000 front 2024-03-21T08:43:26.141817+0000 (oldest deadline 2024-03-21T08:43:49.641995+0000) 2024-03-21T08:43:50.125 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:49.863+0000 7ff3d9e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:50.125 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:49.863+0000 7ff3d9e00700 -1 osd.4 107 *** Got signal Terminated *** 2024-03-21T08:43:50.126 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:49.863+0000 7ff3d9e00700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:43:50.382 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:50.263+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:50.383 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:50.263+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:50.383 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:50.071+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:50.383 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:50.071+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:50.817 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:50.495+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:50.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:50 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:50.574+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:50.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:50 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:50.641+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:50.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:50 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:50.641+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:51.090 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:50.836+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:51.091 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:50.836+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:51.124 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:50.816+0000 7ff3d5600700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:26.141762+0000 front 2024-03-21T08:43:26.141817+0000 (oldest deadline 2024-03-21T08:43:49.641995+0000) 2024-03-21T08:43:51.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:50.918+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:51.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:50.918+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:51.383 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:51.090+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:51.383 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:51.090+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:51.383 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:51.270+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:51.384 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:51.270+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:51.784 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:51 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:51.529+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:51.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:51.541+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:51.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:51.670+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:51.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:51 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:51.670+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:52.124 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:51 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:51.784+0000 7ff3d5600700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:26.141762+0000 front 2024-03-21T08:43:26.141817+0000 (oldest deadline 2024-03-21T08:43:49.641995+0000) 2024-03-21T08:43:52.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:51 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:51.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:52.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:51 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:51.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:52.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:51.803+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:52.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:51.803+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:52.632 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:52.136+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:52.633 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:52.136+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:52.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:52.277+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:52.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:52.277+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:52.821 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:52 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:52.542+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:52.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:52 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:52.537+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:52.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:52 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:52.712+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:52.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:52 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:52.712+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:53.124 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:52 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:52.820+0000 7ff3d5600700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:26.141762+0000 front 2024-03-21T08:43:26.141817+0000 (oldest deadline 2024-03-21T08:43:49.641995+0000) 2024-03-21T08:43:53.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:52 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:52.958+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:53.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:52 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:52.958+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:53.130 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:52.784+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:53.131 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:52.784+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:53.382 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:53.281+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:53.383 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:53.281+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:53.383 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:53.130+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:53.383 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:53.130+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:53.787 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:53 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:53.523+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:53.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:53 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:53.555+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:53.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:53 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:53.555+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:43:53.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:53 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:53.734+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:53.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:53 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:53.734+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:54.124 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:53 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:53.786+0000 7ff3d5600700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:26.141762+0000 front 2024-03-21T08:43:26.141817+0000 (oldest deadline 2024-03-21T08:43:49.641995+0000) 2024-03-21T08:43:54.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:54 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:54.000+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:54.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:54 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:54.000+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:54.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:53.797+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:54.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:53.797+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:54.632 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:54.149+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:54.633 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:54.149+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:54.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:54.325+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:54.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:54.325+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:54.802 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:54 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:54.483+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:54.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:54 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:54.548+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:54.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:54 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:54.548+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:43:54.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:54 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:54.749+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:54.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:54 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:54.749+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:55.102 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:55.102+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:55.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:54.760+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:55.103 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:54.760+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:55.124 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:54 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4[46545]: 2024-03-21T08:43:54.801+0000 7ff3d5600700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:26.141762+0000 front 2024-03-21T08:43:26.141817+0000 (oldest deadline 2024-03-21T08:43:49.641995+0000) 2024-03-21T08:43:55.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:55 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:55.032+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:55.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:55 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:55.032+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:55.366 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:55.102+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:55.624 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:55 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:55.443+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:55.625 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:55 smithi084 podman[71674]: 2024-03-21 08:43:55.337047179 +0000 UTC m=+5.626476273 container died 396fff485347519cf9d140912fb881d9f99fbbd7b2bc4f49fe9a233c2a14ba58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T08:43:55.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:55.366+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:55.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:55.366+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:55.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:55 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:55.522+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:55.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:55 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:55.522+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:43:55.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:55 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:55.764+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:55.927 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:55 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:55.764+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:56.005 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:55 smithi084 podman[71674]: 2024-03-21 08:43:55.801947022 +0000 UTC m=+6.091376164 container cleanup 396fff485347519cf9d140912fb881d9f99fbbd7b2bc4f49fe9a233c2a14ba58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:43:56.006 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:55 smithi084 bash[71674]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4 2024-03-21T08:43:56.123 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:55.805+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:56.123 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:55.805+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:56.374 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:56 smithi084 podman[71691]: 2024-03-21 08:43:56.056785767 +0000 UTC m=+0.701399242 container remove 396fff485347519cf9d140912fb881d9f99fbbd7b2bc4f49fe9a233c2a14ba58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-21T08:43:56.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:56 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:56.005+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:56.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:56 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:56.005+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:56.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:56.122+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:56.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:56.122+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:56.750 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:56 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:56.478+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:56.781 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:56.412+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:56.781 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:56.412+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:56.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:56.506+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:56.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:56.506+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:43:56.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:56.743+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:56.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:56 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:56.743+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:57.094 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:56.781+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:57.094 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:56.781+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:57.099 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:56 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:56.981+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:57.099 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:56 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:56.981+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:57.099 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:56 smithi084 podman[71777]: 2024-03-21 08:43:56.654747439 +0000 UTC m=+0.040091557 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:43:57.099 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:56 smithi084 podman[71777]: 2024-03-21 08:43:56.796003752 +0000 UTC m=+0.181347901 container create 7f536a17a7f93b6c29dfb7d625d7820df31b028b697e32dccb1c110157460fad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T08:43:57.100 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:57 smithi084 podman[71777]: 2024-03-21 08:43:57.099776679 +0000 UTC m=+0.485120830 container init 7f536a17a7f93b6c29dfb7d625d7820df31b028b697e32dccb1c110157460fad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-21T08:43:57.374 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:57 smithi084 podman[71777]: 2024-03-21 08:43:57.110720631 +0000 UTC m=+0.496064756 container start 7f536a17a7f93b6c29dfb7d625d7820df31b028b697e32dccb1c110157460fad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:43:57.375 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:57 smithi084 podman[71777]: 2024-03-21 08:43:57.207359926 +0000 UTC m=+0.592704045 container attach 7f536a17a7f93b6c29dfb7d625d7820df31b028b697e32dccb1c110157460fad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-21T08:43:57.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:57.093+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:57.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:57.093+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:57.761 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:57 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:57.470+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:57.761 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:57 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:57.470+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:43:57.812 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:57.400+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:57.812 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:57.400+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:57.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:57 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:57.491+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:57.874 INFO:journalctl@ceph.osd.4.smithi084.stdout:Mar 21 08:43:57 smithi084 podman[71777]: 2024-03-21 08:43:57.73966807 +0000 UTC m=+1.125012199 container died 7f536a17a7f93b6c29dfb7d625d7820df31b028b697e32dccb1c110157460fad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-21T08:43:58.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:58.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:58.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:58.051+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:58.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:58.051+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:58.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:57 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:57.760+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:58.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:57 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:57.760+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:58.196 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:57 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:57.934+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:58.197 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:57 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:57.934+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:58.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T08:43:58.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:58.350+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:58.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:58.350+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:58.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:43:58.662 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T08:43:58.662 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T08:43:58.662 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.5 2024-03-21T08:43:58.783 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:58.479+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:58.783 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:58.479+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:43:58.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:58 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:58.474+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:59.028 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:58 smithi171 systemd[1]: Stopping Ceph osd.5 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:43:59.029 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:58.818+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:43:59.029 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:58.818+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:43:59.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:58.783+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:43:59.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:58 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:58.783+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:43:59.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:58 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:58.937+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:43:59.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:58 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:58.937+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:43:59.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:58 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:58.937+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:43:59.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.027+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:43:59.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.027+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:43:59.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.215+0000 7f5989000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:43:59.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.215+0000 7f5989000700 -1 osd.5 107 *** Got signal Terminated *** 2024-03-21T08:43:59.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.215+0000 7f5989000700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:43:59.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:59.381+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:43:59.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:43:59.381+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:43:59.808 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:59.503+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:43:59.809 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:59.503+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:43:59.809 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:43:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:43:59.503+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:43:59.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:59 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:59.452+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:43:59.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:43:59 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:43:59.452+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:00.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:59.781+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:00.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:43:59.781+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:00.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.982+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:44:00.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.982+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:44:00.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:43:59 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:43:59.982+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.852299+0000 front 2024-03-21T08:43:35.852477+0000 (oldest deadline 2024-03-21T08:43:59.253220+0000) 2024-03-21T08:44:00.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:59.807+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:00.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:59.807+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:00.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:43:59 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:43:59.807+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:00.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:59 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:59.976+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:00.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:59 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:59.976+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:00.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:43:59 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:43:59.976+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:00.810 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:00.545+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:00.810 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:00.545+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:00.811 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:00.545+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:00.813 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:00 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:00.408+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:00.813 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:00 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:00.408+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:00.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:00 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:00.468+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:00.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:00 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:00.468+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:01.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:00 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:00.812+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:01.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:00 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:00.812+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:01.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:00 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:00.934+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:44:01.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:00 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:00.934+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:44:01.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:00 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:00.934+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.852299+0000 front 2024-03-21T08:43:35.852477+0000 (oldest deadline 2024-03-21T08:43:59.253220+0000) 2024-03-21T08:44:01.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:00.810+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:01.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:00.810+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:01.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:00 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:00.810+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:01.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:00 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:00.930+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:01.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:00 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:00.930+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:01.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:00 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:00.930+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:01.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:01.381+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:01.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:01.381+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:01.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:01 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:01.448+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:01.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:01 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:01.448+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:01.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:01.790+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:01.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:01.790+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:01.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:01.790+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:01.927 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:01.547+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:01.927 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:01.547+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:01.927 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:01 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:01.547+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:02.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:01.794+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:02.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:01.794+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:02.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:01.794+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:02.134 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:01.967+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:44:02.134 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:01.967+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:44:02.134 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:01 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:01.967+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.852299+0000 front 2024-03-21T08:43:35.852477+0000 (oldest deadline 2024-03-21T08:43:59.253220+0000) 2024-03-21T08:44:02.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:01 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:01.953+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:02.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:01 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:01.953+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:02.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:01 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:01.953+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:02.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:02.380+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:02.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:02.380+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:02.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:02.380+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:02.839 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:02.534+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:02.840 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:02.534+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:02.840 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:02.534+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:02.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:02 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:02.446+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:02.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:02 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:02.446+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:03.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:02.754+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:03.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:02.754+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:03.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:02.754+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:03.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:02.995+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:44:03.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:02.995+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:44:03.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:02 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:02.995+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.852299+0000 front 2024-03-21T08:43:35.852477+0000 (oldest deadline 2024-03-21T08:43:59.253220+0000) 2024-03-21T08:44:03.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:02.839+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:03.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:02.839+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:03.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:02 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:02.839+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:03.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:02 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:02.976+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:03.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:02 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:02.976+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:03.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:02 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:02.976+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:03.772 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:03.423+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:03.773 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:03.423+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:03.773 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:03.423+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:03.812 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:03.545+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:03.813 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:03.545+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:03.813 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:03.545+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:03.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:03 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:03.460+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:03.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:03 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:03.460+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:04.132 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:03.999+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.446475+0000 front 2024-03-21T08:43:12.446246+0000 (oldest deadline 2024-03-21T08:43:40.647515+0000) 2024-03-21T08:44:04.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:03.999+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.949882+0000 front 2024-03-21T08:43:25.949705+0000 (oldest deadline 2024-03-21T08:43:48.249302+0000) 2024-03-21T08:44:04.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5[47279]: 2024-03-21T08:44:03.999+0000 7f5984800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.852299+0000 front 2024-03-21T08:43:35.852477+0000 (oldest deadline 2024-03-21T08:43:59.253220+0000) 2024-03-21T08:44:04.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:03.771+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:04.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:03.771+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:04.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:03 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:03.771+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:04.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:03.812+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:04.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:03.812+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:04.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:03.812+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:04.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:03 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:03.968+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:04.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:03 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:03.968+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:04.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:03 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:03.968+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:04.771 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:04.456+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:04.771 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:04.456+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:04.771 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:04.456+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:04.832 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:04.565+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:04.832 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:04.565+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:04.832 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:04.565+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:04.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:04 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:04.509+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:04.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:04 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:04.509+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:05.107 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:04 smithi171 podman[73930]: 2024-03-21 08:44:04.771667046 +0000 UTC m=+5.706664205 container died 6c11021182b65fda2967109026cc2e42ece0d88968570df8cebf6c93edb952ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T08:44:05.107 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:04.812+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:05.107 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:04.812+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:05.108 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:04 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:04.812+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:05.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:04.831+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:05.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:04.831+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:05.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:04 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:04.831+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:05.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:04 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:04.925+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:05.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:04 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:04.925+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:05.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:04 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:04.925+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:05.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:05 smithi171 podman[73930]: 2024-03-21 08:44:05.285136023 +0000 UTC m=+6.220133242 container cleanup 6c11021182b65fda2967109026cc2e42ece0d88968570df8cebf6c93edb952ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T08:44:05.382 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:05 smithi171 bash[73930]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5 2024-03-21T08:44:05.814 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:05 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:05.473+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:05.814 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:05 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:05.473+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:05.814 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:05 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:05.473+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:05.815 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:05 smithi171 podman[73950]: 2024-03-21 08:44:05.553420485 +0000 UTC m=+0.770424624 container remove 6c11021182b65fda2967109026cc2e42ece0d88968570df8cebf6c93edb952ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-21T08:44:05.850 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:05.577+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:05.851 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:05.577+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:05.851 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:05.577+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:05.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:05 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:05.493+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:05.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:05 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:05.493+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:06.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:05 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:05.813+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:06.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:05 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:05.813+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:06.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:05 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:05.813+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:06.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:05.850+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:06.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:05.850+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:06.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:05 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:05.850+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:06.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:05 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:05.921+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:06.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:05 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:05.921+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:06.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:05 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:05.921+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:06.632 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:06 smithi171 podman[74039]: 2024-03-21 08:44:06.273136903 +0000 UTC m=+0.036459480 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:44:06.633 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:06 smithi171 podman[74039]: 2024-03-21 08:44:06.438480715 +0000 UTC m=+0.201803328 container create a565ce7a7c900fa8e0b51abfd2a2401f1d5510a8c4b4d72bcb3c313f58ed6fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T08:44:06.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:06 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:06.442+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:06.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:06 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:06.442+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:06.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:06 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:06.442+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:06.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:06 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:06.488+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:06.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:06 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:06.488+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:06.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:06 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:06.488+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:06.882 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:06 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:06.612+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:06.882 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:06 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:06.612+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:06.882 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:06 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:06.612+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:07.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:06 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:06.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:07.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:06 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:06.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:07.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:06 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:06.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:07.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:06 smithi171 podman[74039]: 2024-03-21 08:44:06.717431368 +0000 UTC m=+0.480753991 container init a565ce7a7c900fa8e0b51abfd2a2401f1d5510a8c4b4d72bcb3c313f58ed6fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:44:07.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:06 smithi171 podman[74039]: 2024-03-21 08:44:06.728118922 +0000 UTC m=+0.491441542 container start a565ce7a7c900fa8e0b51abfd2a2401f1d5510a8c4b4d72bcb3c313f58ed6fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T08:44:07.133 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:06 smithi171 podman[74039]: 2024-03-21 08:44:06.858069665 +0000 UTC m=+0.621392266 container attach a565ce7a7c900fa8e0b51abfd2a2401f1d5510a8c4b4d72bcb3c313f58ed6fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-21T08:44:07.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:06 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:06.881+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:07.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:06 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:06.881+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:07.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:06 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:06.881+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:07.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:06 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:06.922+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:07.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:06 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:06.922+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:07.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:06 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:06.922+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:07.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:06 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:06.922+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:07.440 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 21 08:44:07 smithi171 podman[74039]: 2024-03-21 08:44:07.163549595 +0000 UTC m=+0.926872186 container died a565ce7a7c900fa8e0b51abfd2a2401f1d5510a8c4b4d72bcb3c313f58ed6fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T08:44:07.804 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:07 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:07.439+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:07.805 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:07 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:07.439+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:07.805 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:07 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:07.439+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:07.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:07.479+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:07.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:07.479+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:07.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:07.479+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:07.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:07.658+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:07.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:07.658+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:07.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:07.658+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:07.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:07.883+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:07.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:07.883+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:07.926 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:07 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:07.883+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:07.934 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T08:44:08.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:44:08.040 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T08:44:08.040 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-21T08:44:08.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.6 2024-03-21T08:44:08.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:07 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:07.803+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:08.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:07 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:07.803+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:08.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:07 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:07.803+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:08.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:07.926+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:08.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:07.926+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:08.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:07.926+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:08.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:07 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:07.926+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:08.425 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 systemd[1]: Stopping Ceph osd.6 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:44:08.721 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:08 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:08.438+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:08.722 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:08 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:08.438+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:08.722 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:08 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:08.438+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:08.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:08 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:08.485+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:08.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:08 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:08.485+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:08.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:08 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:08.485+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:08.922 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:08.631+0000 7fa381800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:44:08.923 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:08.631+0000 7fa381800700 -1 osd.6 107 *** Got signal Terminated *** 2024-03-21T08:44:08.923 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:08.631+0000 7fa381800700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:44:08.923 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:08.696+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:08.923 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:08.696+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:08.923 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:08.696+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:08.924 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:08.696+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:39.623730+0000 front 2024-03-21T08:43:39.624311+0000 (oldest deadline 2024-03-21T08:44:07.825250+0000) 2024-03-21T08:44:09.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:08 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:08.787+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:09.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:08 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:08.787+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:09.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:08 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:08.787+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:09.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:08.921+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:09.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:08.921+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:09.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:08 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:08.921+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:09.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:08 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:08.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:09.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:08 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:08.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:09.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:08 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:08.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:09.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:08 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:08.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:09.740 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:09 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:09.469+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:09.740 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:09 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:09.470+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:09.740 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:09 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:09.470+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:09.740 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:09 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:09.470+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:09.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:09 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:09.514+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:09.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:09 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:09.514+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:09.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:09 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:09.514+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:09.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:09.653+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:09.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:09.653+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:09.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:09.653+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:09.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:09.653+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:39.623730+0000 front 2024-03-21T08:43:39.624311+0000 (oldest deadline 2024-03-21T08:44:07.825250+0000) 2024-03-21T08:44:10.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:09 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:09.739+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:10.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:09 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:09.739+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:10.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:09 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:09.739+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:10.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:09 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:09.962+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:10.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:09 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:09.962+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:10.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:09 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:09.962+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:10.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:09 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:09.962+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:10.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:09.962+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:10.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:09.962+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:10.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:09.962+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:10.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:09.962+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:10.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:10 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:10.507+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:10.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:10 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:10.507+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:10.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:10 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:10.507+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:10.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:10 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:10.730+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:10.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:10 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:10.730+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:10.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:10 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:10.730+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:10.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:10 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:10.489+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:10.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:10 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:10.489+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:10.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:10 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:10.489+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:10.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:10 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:10.489+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:10.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:10.637+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:10.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:10.637+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:10.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:10.637+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:10.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:10.637+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:39.623730+0000 front 2024-03-21T08:43:39.624311+0000 (oldest deadline 2024-03-21T08:44:07.825250+0000) 2024-03-21T08:44:11.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:10 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:10.959+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:11.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:10 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:10.959+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:11.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:10 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:10.959+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:11.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:10 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:10.959+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:11.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:10.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:11.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:10.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:11.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:10.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:11.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:10 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:10.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:11.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:11.536+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:11.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:11.536+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:11.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:11.537+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:11.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:11.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:11.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:11.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:11.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:11.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:11.886 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:11.470+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:11.886 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:11.470+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:11.886 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:11.470+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:11.886 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:11 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:11.470+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:11.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:11.666+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:11.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:11.666+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:11.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:11.666+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:11.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:11 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:11.666+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:39.623730+0000 front 2024-03-21T08:43:39.624311+0000 (oldest deadline 2024-03-21T08:44:07.825250+0000) 2024-03-21T08:44:12.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:11.954+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:12.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:11.954+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:12.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:11.954+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:12.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:11 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:11.954+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:12.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:12.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:12.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:12.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:12.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:12.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:12.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:12.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:12.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:12 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:12.490+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:12.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:12 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:12.490+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:12.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:12 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:12.490+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:12.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:12 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:12.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:12.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:12 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:12.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:12.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:12 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:12.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:12.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:12 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:12.519+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:12.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:12 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:12.519+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:12.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:12 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:12.519+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:12.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:12 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:12.519+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:12.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:12.672+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:12.016204+0000 front 2024-03-21T08:43:12.018652+0000 (oldest deadline 2024-03-21T08:43:37.916238+0000) 2024-03-21T08:44:12.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:12.673+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:21.418468+0000 front 2024-03-21T08:43:21.419040+0000 (oldest deadline 2024-03-21T08:43:53.220063+0000) 2024-03-21T08:44:12.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:12.673+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.221087+0000 front 2024-03-21T08:43:33.221033+0000 (oldest deadline 2024-03-21T08:43:59.121296+0000) 2024-03-21T08:44:12.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:12 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6[59791]: 2024-03-21T08:44:12.673+0000 7fa378000700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:39.623730+0000 front 2024-03-21T08:43:39.624311+0000 (oldest deadline 2024-03-21T08:44:07.825250+0000) 2024-03-21T08:44:13.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:12 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:12.933+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:13.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:12 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:12.933+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:13.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:12 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:12.933+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:13.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:12 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:12.933+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:13.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:13 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:13.028+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:13.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:13 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:13.028+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:13.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:13 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:13.028+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:13.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:13 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:13.028+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:13.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:13 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:13.466+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:13.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:13 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:13.466+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:13.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:13 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:13.466+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:13.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:13 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:13.705+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:13.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:13 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:13.705+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:13.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:13 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:13.705+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:13.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:13 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:13.492+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:13.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:13 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:13.492+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:13.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:13 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:13.492+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:13.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:13 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:13.492+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:14.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:13 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:13.972+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:14.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:13 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:13.972+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:14.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:13 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:13.972+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:14.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:13 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:13.972+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:14.426 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:14 smithi003 podman[103031]: 2024-03-21 08:44:14.250123381 +0000 UTC m=+5.769358258 container died 8f5d2df0d519503093677d2081b519d8ce3546ee443f259d5fa6a6883b3ec221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:44:14.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:14 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:14.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:14 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:14.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:14 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:14.427 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:14 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:14.817 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:14 smithi003 podman[103031]: 2024-03-21 08:44:14.582374099 +0000 UTC m=+6.101608986 container cleanup 8f5d2df0d519503093677d2081b519d8ce3546ee443f259d5fa6a6883b3ec221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T08:44:14.817 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:14 smithi003 bash[103031]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6 2024-03-21T08:44:14.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:14 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:14.455+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:14.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:14 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:14.455+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:14.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:14 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:14.455+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:14.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:14.746+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:14.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:14.746+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:14.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:14.746+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:14.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:14.514+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:14.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:14.514+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:14.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:14.514+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:14.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:14 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:14.515+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:15.175 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:14 smithi003 podman[103050]: 2024-03-21 08:44:14.816811764 +0000 UTC m=+0.555159487 container remove 8f5d2df0d519503093677d2081b519d8ce3546ee443f259d5fa6a6883b3ec221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T08:44:15.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.999+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:15.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.999+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:15.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.999+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:15.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:15 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:14.999+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:15.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:14 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:14.930+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:15.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:14 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:14.930+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:15.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:14 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:14.930+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:15.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:14 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:14.930+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:15.770 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:15.456+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:15.770 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:15.456+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:15.771 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:15.456+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:15.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:15 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:15.736+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:15.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:15 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:15.736+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:15.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:15 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:15.736+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:15.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:15 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:15.522+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:15.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:15 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:15.522+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:15.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:15 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:15.522+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:15.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:15 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:15.522+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:15.925 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:15 smithi003 podman[103138]: 2024-03-21 08:44:15.650909685 +0000 UTC m=+0.130947211 container create 083cdcc82fba5d76fe8e413869688b302fdfff04ed043393b822143f6480310f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-21T08:44:15.926 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:15 smithi003 podman[103138]: 2024-03-21 08:44:15.561353504 +0000 UTC m=+0.041391000 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:44:16.083 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:15.963+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:16.083 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:15.963+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:16.084 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:15.963+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:16.084 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:15 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:15.963+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:16.421 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:15 smithi003 podman[103138]: 2024-03-21 08:44:15.956893722 +0000 UTC m=+0.436931255 container init 083cdcc82fba5d76fe8e413869688b302fdfff04ed043393b822143f6480310f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:44:16.421 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:15 smithi003 podman[103138]: 2024-03-21 08:44:15.969209174 +0000 UTC m=+0.449246684 container start 083cdcc82fba5d76fe8e413869688b302fdfff04ed043393b822143f6480310f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2) 2024-03-21T08:44:16.421 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:16 smithi003 podman[103138]: 2024-03-21 08:44:16.061625895 +0000 UTC m=+0.541663471 container attach 083cdcc82fba5d76fe8e413869688b302fdfff04ed043393b822143f6480310f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-21T08:44:16.422 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.014+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:16.422 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.014+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:16.422 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.014+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:16.422 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.014+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:16.422 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.014+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:16.675 INFO:journalctl@ceph.osd.6.smithi003.stdout:Mar 21 08:44:16 smithi003 podman[103138]: 2024-03-21 08:44:16.422256443 +0000 UTC m=+0.902293999 container died 083cdcc82fba5d76fe8e413869688b302fdfff04ed043393b822143f6480310f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:44:16.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:16.428+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:16.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:16.428+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:16.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:16.428+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:16.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:16.428+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:52.073192+0000 front 2024-03-21T08:43:52.073116+0000 (oldest deadline 2024-03-21T08:44:15.573152+0000) 2024-03-21T08:44:16.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:16.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:16.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:16.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:16.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:16.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:16.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:16.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:16.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:16.514+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:16.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:16.514+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:16.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:16.514+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:16.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:16 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:16.514+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:17.119 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.987+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:17.120 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.987+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:17.120 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.987+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:17.120 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.987+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:17.120 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:16 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:16.987+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:17.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.6.service' 2024-03-21T08:44:17.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:44:17.260 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-21T08:44:17.260 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-21T08:44:17.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.7 2024-03-21T08:44:17.338 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:16.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:17.338 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:16.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:17.338 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:16.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:17.338 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:16 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:16.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:17.625 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 systemd[1]: Stopping Ceph osd.7 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:44:17.626 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:17.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:17.626 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:17.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:17.626 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:17.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:17.626 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:17.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:52.073192+0000 front 2024-03-21T08:43:52.073116+0000 (oldest deadline 2024-03-21T08:44:15.573152+0000) 2024-03-21T08:44:17.725 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:17.724+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:17.725 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:17.724+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:17.725 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:17.474+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:17.726 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:17.474+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:17.726 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:17.474+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:17.726 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:17.474+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:18.124 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:17.791+0000 7f1d5ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:44:18.124 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:17.791+0000 7f1d5ca00700 -1 osd.7 107 *** Got signal Terminated *** 2024-03-21T08:44:18.124 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:17 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:17.791+0000 7f1d5ca00700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:44:18.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:18.008+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:18.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:18.008+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:18.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:18.008+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:18.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:18.008+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:18.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:18.008+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:18.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:17.724+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:18.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:17 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:17.724+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:18.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:18 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:18.007+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:18.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:18 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:18.007+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:18.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:18 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:18.007+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:18.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:18 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:18.007+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:18.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:18 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:18.007+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:18.835 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:18.499+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:18.836 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:18.499+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:18.836 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:18.499+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:18.836 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:18.499+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:18.836 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:18.731+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:18.836 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:18.731+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:18.836 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:18.731+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:18.836 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:18 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:18.731+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:18.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:18.472+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:18.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:18.472+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:18.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:18.472+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:18.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:18 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:18.472+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:52.073192+0000 front 2024-03-21T08:43:52.073116+0000 (oldest deadline 2024-03-21T08:44:15.573152+0000) 2024-03-21T08:44:19.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.025+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:19.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.025+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:19.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.025+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:19.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.025+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:19.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.025+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:19.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:19.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:19.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:19.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:19.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.021+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:19.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:19.425+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:19.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:19.425+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:19.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:19.425+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:19.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:19.425+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:52.073192+0000 front 2024-03-21T08:43:52.073116+0000 (oldest deadline 2024-03-21T08:44:15.573152+0000) 2024-03-21T08:44:19.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:19.718+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:19.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:19.718+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:19.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:19.718+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:19.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:19.718+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:19.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:19.525+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:19.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:19.525+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:19.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:19.525+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:19.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:19 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:19.525+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:20.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.978+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:20.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.978+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:20.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.978+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:20.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.978+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:20.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:19 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:19.978+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:20.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.981+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:20.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.981+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:20.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.981+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:20.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.981+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:20.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:19 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:19.981+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:20.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:20.464+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:20.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:20.464+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:20.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:20.464+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:20.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:20.464+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:52.073192+0000 front 2024-03-21T08:43:52.073116+0000 (oldest deadline 2024-03-21T08:44:15.573152+0000) 2024-03-21T08:44:20.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:20.730+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:20.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:20.730+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:20.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:20.730+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:20.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:20.730+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:20.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:20.563+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:20.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:20.563+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:20.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:20.563+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:20.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:20.563+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:20.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:20 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:20.563+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:21.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:20.954+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:21.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:20.954+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:21.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:20.955+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:21.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:20.955+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:21.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:20 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:20.955+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:21.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:20 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:20.939+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:21.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:20 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:20.939+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:21.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:20 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:20.939+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:21.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:20 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:20.939+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:21.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:20 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:20.939+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:21.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:21.468+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:21.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:21.468+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:21.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:21.468+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:21.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:21.468+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:52.073192+0000 front 2024-03-21T08:43:52.073116+0000 (oldest deadline 2024-03-21T08:44:15.573152+0000) 2024-03-21T08:44:21.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:21.778+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:21.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:21.778+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:21.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:21.778+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:21.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:21.778+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:21.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:21.569+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:21.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:21.569+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:21.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:21.569+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:21.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:21.569+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:21.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:21 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:21.569+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:22.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:21.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:22.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:21.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:22.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:21.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:22.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:21.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:22.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:21.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:22.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:21 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:21.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:22.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:21 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:21.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:22.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:21 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:21.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:22.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:21 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:21.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:22.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:21 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:21.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:22.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:22.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:22.366156+0000 front 2024-03-21T08:43:22.366219+0000 (oldest deadline 2024-03-21T08:43:47.666259+0000) 2024-03-21T08:44:22.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:22.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.068597+0000 front 2024-03-21T08:43:34.068143+0000 (oldest deadline 2024-03-21T08:43:59.367845+0000) 2024-03-21T08:44:22.874 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:22.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:45.171527+0000 front 2024-03-21T08:43:45.171484+0000 (oldest deadline 2024-03-21T08:44:06.270973+0000) 2024-03-21T08:44:22.875 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7[53647]: 2024-03-21T08:44:22.430+0000 7f1d53200700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:52.073192+0000 front 2024-03-21T08:43:52.073116+0000 (oldest deadline 2024-03-21T08:44:15.573152+0000) 2024-03-21T08:44:22.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:22.744+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:22.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:22.744+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:22.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:22.744+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:22.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:22.744+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:22.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:22.556+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:22.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:22.556+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:22.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:22.556+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:22.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:22.556+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:22.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:22 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:22.556+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:23.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:22.967+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:23.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:22.967+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:23.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:22.967+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:23.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:22.967+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:23.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:22 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:22.967+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:23.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:22 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:22.955+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:23.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:22 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:22.955+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:23.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:22 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:22.955+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:23.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:22 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:22.955+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:23.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:22 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:22.955+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:23.766 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:23 smithi084 podman[71893]: 2024-03-21 08:44:23.424908387 +0000 UTC m=+5.767198994 container died db476597cfd84427f5198075b1a69e7d8146b929930029e61d7ce041b5961c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:44:23.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:23.759+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:23.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:23.759+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:23.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:23.759+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:23.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:23.759+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:23.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:23.553+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:23.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:23.553+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:23.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:23.553+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:23.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:23.553+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:23.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:23 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:23.553+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:24.016 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:24 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:24.016+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:24.017 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:23 smithi084 podman[71893]: 2024-03-21 08:44:23.945628034 +0000 UTC m=+6.287918662 container cleanup db476597cfd84427f5198075b1a69e7d8146b929930029e61d7ce041b5961c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T08:44:24.017 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:23 smithi084 bash[71893]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7 2024-03-21T08:44:24.374 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:24 smithi084 podman[71911]: 2024-03-21 08:44:24.163745778 +0000 UTC m=+0.725520619 container remove db476597cfd84427f5198075b1a69e7d8146b929930029e61d7ce041b5961c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T08:44:24.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:24 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:24.016+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:24.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:24 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:24.016+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:24.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:24 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:24.016+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:24.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:24 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:24.016+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:24.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:23.932+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:24.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:23.932+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:24.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:23.932+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:24.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:23.932+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:24.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:23 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:23.932+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:24.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:24.785+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:24.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:24.785+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:24.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:24.785+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:24.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:24.785+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:24.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:24.595+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:24.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:24.595+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:24.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:24.595+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:24.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:24.595+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:24.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:24 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:24.595+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:25.218 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.011+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:25.218 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.011+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:25.218 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.011+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:25.219 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.011+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:25.219 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.011+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:25.219 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:24 smithi084 podman[71993]: 2024-03-21 08:44:24.78764311 +0000 UTC m=+0.045968810 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:44:25.219 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:24 smithi084 podman[71993]: 2024-03-21 08:44:24.90643901 +0000 UTC m=+0.164764734 container create 242d8c26042de4b4410bc97e6b031484592ae6c21e2b9f7baf7d173d99ca3110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T08:44:25.219 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:25 smithi084 podman[71993]: 2024-03-21 08:44:25.218650995 +0000 UTC m=+0.476976718 container init 242d8c26042de4b4410bc97e6b031484592ae6c21e2b9f7baf7d173d99ca3110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-21T08:44:25.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:24.947+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:25.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:24.947+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:25.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:24.947+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:25.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:24.947+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:25.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:24 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:24.947+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:25.624 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:25 smithi084 podman[71993]: 2024-03-21 08:44:25.232715682 +0000 UTC m=+0.491041383 container start 242d8c26042de4b4410bc97e6b031484592ae6c21e2b9f7baf7d173d99ca3110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-21T08:44:25.625 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:25 smithi084 podman[71993]: 2024-03-21 08:44:25.309220605 +0000 UTC m=+0.567546319 container attach 242d8c26042de4b4410bc97e6b031484592ae6c21e2b9f7baf7d173d99ca3110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:44:25.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:25.748+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:25.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:25.748+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:25.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:25.748+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:25.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:25.748+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:25.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:25.560+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:25.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:25.560+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:25.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:25.560+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:25.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:25.560+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:25.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:25 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:25.560+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:25.971 INFO:journalctl@ceph.osd.7.smithi084.stdout:Mar 21 08:44:25 smithi084 podman[71993]: 2024-03-21 08:44:25.643558507 +0000 UTC m=+0.901884207 container died 242d8c26042de4b4410bc97e6b031484592ae6c21e2b9f7baf7d173d99ca3110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-21T08:44:26.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:25 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:25.905+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:26.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:25 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:25.905+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:26.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:25 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:25.905+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:26.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:25 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:25.905+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:26.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:25 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:25.905+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:26.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:26.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:26.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:26.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:26.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:25.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:26.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.7.service' 2024-03-21T08:44:26.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:44:26.549 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-21T08:44:26.549 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-21T08:44:26.549 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.8 2024-03-21T08:44:26.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:26.603+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:26.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:26.603+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:26.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:26.603+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:26.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:26.603+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:26.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:26.603+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:26.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:26 smithi171 systemd[1]: Stopping Ceph osd.8 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:44:26.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:26.717+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:26.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:26.717+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:26.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:26.717+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:26.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:26.717+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:26.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:26 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:26.717+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:27.334 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:26 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:26.986+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:27.334 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:26 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:26.986+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:27.334 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:26 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:26.986+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:27.334 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:26 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:26.986+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:27.334 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:26 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:26.986+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:27.382 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.103+0000 7f491b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:44:27.382 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.103+0000 7f491b800700 -1 osd.8 107 *** Got signal Terminated *** 2024-03-21T08:44:27.383 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.103+0000 7f491b800700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:44:27.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:26.937+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:27.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:26.937+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:27.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:26.937+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:27.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:26.937+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:27.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:26 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:26.937+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:27.882 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.598+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:27.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.598+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:27.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.598+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:27.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.598+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:27.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.598+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:27.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:27.598+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.912153+0000 front 2024-03-21T08:44:01.912282+0000 (oldest deadline 2024-03-21T08:44:26.612361+0000) 2024-03-21T08:44:27.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:27.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:27.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:27.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:27.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:27.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:27.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:27.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:27.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:27 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:27.715+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:28.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:27 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:27.992+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:28.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:27 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:27.992+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:28.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:27 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:27.992+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:28.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:27 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:27.992+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:28.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:27 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:27.992+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:28.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:27 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:27.992+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:28.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:27.985+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:28.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:27.985+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:28.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:27.985+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:28.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:27.985+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:28.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:27.985+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:28.427 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:27 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:27.985+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:28.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:28.693+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:28.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:28.693+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:28.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:28.693+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:28.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:28.693+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:28.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:28.693+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:28.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:28.597+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:28.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:28.597+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:28.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:28.597+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:28.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:28.597+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:28.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:28.597+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:28.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:28 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:28.597+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.912153+0000 front 2024-03-21T08:44:01.912282+0000 (oldest deadline 2024-03-21T08:44:26.612361+0000) 2024-03-21T08:44:29.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:28 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:28.990+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:29.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:28 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:28.990+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:29.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:28 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:28.990+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:29.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:28 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:28.990+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:29.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:28 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:28.990+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:29.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:28 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:28.990+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:29.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:28.983+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:29.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:28.983+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:29.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:28.983+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:29.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:28.983+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:29.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:28.983+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:29.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:28 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:28.983+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:29.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:29.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:29.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:29.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:29.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:29.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:29.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:29.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:29.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:29.697+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:29.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:29.608+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:29.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:29.608+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:29.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:29.608+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:29.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:29.609+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:29.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:29.609+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:29.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:29 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:29.609+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.912153+0000 front 2024-03-21T08:44:01.912282+0000 (oldest deadline 2024-03-21T08:44:26.612361+0000) 2024-03-21T08:44:30.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:29 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:29.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:30.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:29 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:29.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:30.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:29 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:29.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:30.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:29 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:29.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:30.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:29 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:29.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:30.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:29 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:29.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:30.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:29.949+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:30.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:29.949+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:30.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:29.949+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:30.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:29.949+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:30.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:29.949+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:30.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:29 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:29.949+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:30.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:30.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:30.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:30.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:30.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:30.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:30.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:30.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:30.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:30.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:30.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:30.625+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:30.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:30.625+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:30.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:30.625+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:30.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:30.625+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:30.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:30.625+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:30.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:30 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:30.625+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.912153+0000 front 2024-03-21T08:44:01.912282+0000 (oldest deadline 2024-03-21T08:44:26.612361+0000) 2024-03-21T08:44:31.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:30.910+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:31.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:30.910+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:31.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:30.910+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:31.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:30.910+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:31.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:30.910+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:31.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:30 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:30.910+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:31.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.006+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:31.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.006+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:31.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.006+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:31.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.006+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:31.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.006+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:31.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.007+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:31.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:31.630+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:31.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:31.630+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:31.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:31.630+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:31.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:31.630+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:31.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:31.630+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:31.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:31.647+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.790800+0000 front 2024-03-21T08:43:15.790532+0000 (oldest deadline 2024-03-21T08:43:39.890408+0000) 2024-03-21T08:44:31.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:31.647+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.495693+0000 front 2024-03-21T08:43:25.495761+0000 (oldest deadline 2024-03-21T08:43:48.995770+0000) 2024-03-21T08:44:31.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:31.647+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:34.001391+0000 front 2024-03-21T08:43:34.001536+0000 (oldest deadline 2024-03-21T08:44:01.603138+0000) 2024-03-21T08:44:31.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:31.647+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:41.604730+0000 front 2024-03-21T08:43:41.604425+0000 (oldest deadline 2024-03-21T08:44:08.605061+0000) 2024-03-21T08:44:31.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:31.647+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.208854+0000 front 2024-03-21T08:43:53.208917+0000 (oldest deadline 2024-03-21T08:44:20.210319+0000) 2024-03-21T08:44:31.884 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:31 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8[54121]: 2024-03-21T08:44:31.647+0000 7f4912000700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.912153+0000 front 2024-03-21T08:44:01.912282+0000 (oldest deadline 2024-03-21T08:44:26.612361+0000) 2024-03-21T08:44:32.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:32.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:32.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:32.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:32.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:32.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:31 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:31.970+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:32.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:31 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:31.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:32.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:31 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:31.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:32.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:31 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:31.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:32.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:31 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:31.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:32.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:31 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:31.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:32.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:31 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:31.940+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:32.823 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:32 smithi171 podman[74157]: 2024-03-21 08:44:32.506653612 +0000 UTC m=+5.547290240 container died 5fb019aff6ae5f0c57130310dac0cec1c051a06673a0b6f152087494b745d91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T08:44:32.823 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:32 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:32.617+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:32.823 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:32 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:32.617+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:32.823 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:32 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:32.617+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:32.823 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:32 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:32.617+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:32.824 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:32 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:32.617+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:33.132 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:32 smithi171 podman[74157]: 2024-03-21 08:44:32.98802031 +0000 UTC m=+6.028656958 container cleanup 5fb019aff6ae5f0c57130310dac0cec1c051a06673a0b6f152087494b745d91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T08:44:33.132 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:32 smithi171 bash[74157]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8 2024-03-21T08:44:33.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.007+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:33.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.007+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:33.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.007+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:33.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.007+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:33.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.007+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:33.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.007+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:33.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:32 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:32.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:33.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:32 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:32.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:33.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:32 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:32.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:33.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:32 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:32.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:33.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:32 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:32.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:33.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:32 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:32.977+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:33.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:33 smithi171 podman[74176]: 2024-03-21 08:44:33.240803483 +0000 UTC m=+0.724869657 container remove 5fb019aff6ae5f0c57130310dac0cec1c051a06673a0b6f152087494b745d91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-21T08:44:34.005 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:33 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:33.662+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:34.005 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:33 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:33.662+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:34.006 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:33 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:33.662+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:34.006 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:33 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:33.662+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:34.006 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:33 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:33.662+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:34.319 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:34 smithi171 podman[74264]: 2024-03-21 08:44:33.911684225 +0000 UTC m=+0.035025627 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:44:34.319 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:34 smithi171 podman[74264]: 2024-03-21 08:44:34.032896023 +0000 UTC m=+0.156237450 container create e2ae0a2230c2c9ddb346b6125eae7bbac3cd1b4858157b441529de034172a874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:44:34.320 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:34 smithi171 podman[74264]: 2024-03-21 08:44:34.319728674 +0000 UTC m=+0.443070141 container init e2ae0a2230c2c9ddb346b6125eae7bbac3cd1b4858157b441529de034172a874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-21T08:44:34.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.988+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:34.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.988+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:34.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.988+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:34.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.988+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:34.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.988+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:34.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:33 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:33.988+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:34.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:34 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:34.020+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:34.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:34 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:34.020+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:34.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:34 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:34.020+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:34.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:34 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:34.020+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:34.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:34 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:34.020+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:34.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:34 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:34.020+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:34.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:34 smithi171 podman[74264]: 2024-03-21 08:44:34.330238145 +0000 UTC m=+0.453579605 container start e2ae0a2230c2c9ddb346b6125eae7bbac3cd1b4858157b441529de034172a874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:44:34.632 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:34 smithi171 podman[74264]: 2024-03-21 08:44:34.401975976 +0000 UTC m=+0.525317409 container attach e2ae0a2230c2c9ddb346b6125eae7bbac3cd1b4858157b441529de034172a874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T08:44:35.111 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 21 08:44:34 smithi171 podman[74264]: 2024-03-21 08:44:34.76228875 +0000 UTC m=+0.885630169 container died e2ae0a2230c2c9ddb346b6125eae7bbac3cd1b4858157b441529de034172a874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:44:35.111 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:34 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:34.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:35.111 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:34 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:34.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:35.111 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:34 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:34.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:35.111 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:34 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:34.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:35.112 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:34 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:34.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:35.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:34 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:34.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:35.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:34 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:34.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:35.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:34 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:34.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:35.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:34 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:34.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:35.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:34 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:34.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:35.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:34 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:34.994+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:35.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:35.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:35.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:35.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:35.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:35.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.030+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:35.538 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.8.service' 2024-03-21T08:44:35.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:44:35.659 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-21T08:44:35.659 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-21T08:44:35.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.9 2024-03-21T08:44:35.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:35 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:35.677+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:35.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:35 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:35.677+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:35.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:35 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:35.677+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:35.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:35 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:35.677+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:35.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:35 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:35.677+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:36.175 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 systemd[1]: Stopping Ceph osd.9 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:44:36.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.988+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:36.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.988+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:36.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.988+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:36.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.988+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:36.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.988+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:36.176 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:35 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:35.988+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:36.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:36 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:36.032+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:36.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:36 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:36.033+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:36.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:36 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:36.033+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:36.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:36 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:36.033+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:36.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:36 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:36.033+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:36.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:36 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:36.033+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:36.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:36 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:36.033+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:36.675 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.237+0000 7fa327e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:44:36.902 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.238+0000 7fa327e00700 -1 osd.9 107 *** Got signal Terminated *** 2024-03-21T08:44:36.903 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.238+0000 7fa327e00700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:44:37.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:36 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:36.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:37.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:36 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:36.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:37.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:36 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:36.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:37.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:36 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:36.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:37.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:36 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:36.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:37.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.029+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:37.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.029+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:37.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.029+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:37.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.029+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:37.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.029+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:37.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.029+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:37.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.029+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:37.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.991+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:37.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.991+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:37.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.991+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:37.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.991+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:37.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.991+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:37.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:36 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:36.991+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:38.034 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:37 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:37.646+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:38.034 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:37 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:37.646+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:38.035 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:37 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:37.646+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:38.035 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:37 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:37.646+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:38.035 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:37 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:37.646+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:38.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.987+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:38.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.987+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:38.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.987+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:38.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.987+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:38.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.987+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:38.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.987+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:38.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:37 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:37.987+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:38.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:38 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:38.025+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:38.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:38 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:38.025+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:38.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:38 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:38.025+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:38.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:38 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:38.025+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:38.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:38 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:38.025+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:38.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:38 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:38.025+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:39.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:38 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:38.651+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:39.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:38 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:38.651+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:39.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:38 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:38.651+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:39.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:38 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:38.651+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:39.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:38 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:38.651+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:39.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:38 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:38.651+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:39.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:38 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:38.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:39.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:38 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:38.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:39.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:38 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:38.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:39.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:38 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:38.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:39.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:38 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:38.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:39.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:38 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:38.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:39.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:38 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:38.965+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:39.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:39 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:39.026+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:39.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:39 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:39.026+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:39.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:39 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:39.026+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:39.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:39 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:39.026+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:39.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:39 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:39.026+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:39.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:39 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:39.026+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:39.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:39.632+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:39.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:39.632+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:39.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:39.632+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:39.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:39.632+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:39.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:39.632+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:39.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:39 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:39.632+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:40.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:39.974+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:40.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:39.974+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:40.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:39.974+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:40.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:39.974+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:40.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:39.974+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:40.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:39.974+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:40.376 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:39.974+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:40.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.019+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:40.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.019+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:40.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.019+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:40.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.019+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:40.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.019+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:40.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.019+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:41.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:40.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:41.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:40.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:41.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:40.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:41.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:40.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:41.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:40.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:41.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:40 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:40.675+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:41.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:41.012+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:41.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:41.012+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:41.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:41.012+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:41.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:41.012+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:41.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:41.012+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:41.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:41.012+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:41.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:41 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:41.012+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:41.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.990+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:16.431064+0000 front 2024-03-21T08:43:16.431203+0000 (oldest deadline 2024-03-21T08:43:41.131281+0000) 2024-03-21T08:44:41.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.990+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:25.734401+0000 front 2024-03-21T08:43:25.734343+0000 (oldest deadline 2024-03-21T08:43:49.834602+0000) 2024-03-21T08:44:41.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.990+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:33.336635+0000 front 2024-03-21T08:43:33.336952+0000 (oldest deadline 2024-03-21T08:43:59.236741+0000) 2024-03-21T08:44:41.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.990+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:43.839300+0000 front 2024-03-21T08:43:43.839263+0000 (oldest deadline 2024-03-21T08:44:09.138893+0000) 2024-03-21T08:44:41.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.990+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:51.942447+0000 front 2024-03-21T08:43:51.942623+0000 (oldest deadline 2024-03-21T08:44:15.442598+0000) 2024-03-21T08:44:41.426 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:40 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9[66689]: 2024-03-21T08:44:40.990+0000 7fa31e600700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:01.846824+0000 front 2024-03-21T08:44:01.846782+0000 (oldest deadline 2024-03-21T08:44:27.048091+0000) 2024-03-21T08:44:42.087 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:41 smithi003 podman[103255]: 2024-03-21 08:44:41.767359793 +0000 UTC m=+5.661834179 container stop cc321b34bc33c281805a5b745ae8382e5f43e67b10a4e12745c10b2331fae40a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T08:44:42.087 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:41 smithi003 podman[103255]: 2024-03-21 08:44:41.794647141 +0000 UTC m=+5.689121528 container died cc321b34bc33c281805a5b745ae8382e5f43e67b10a4e12745c10b2331fae40a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-21T08:44:42.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:41.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:42.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:41.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:42.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:41.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:42.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:41.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:42.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:41.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:42.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:41 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:41.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:42.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:42.059+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:42.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:42.059+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:42.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:42.059+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:42.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:42.059+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:42.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:42.059+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:42.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:42.059+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:42.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:42 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:42.059+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:42.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:42 smithi003 podman[103255]: 2024-03-21 08:44:42.18934189 +0000 UTC m=+6.083816286 container cleanup cc321b34bc33c281805a5b745ae8382e5f43e67b10a4e12745c10b2331fae40a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T08:44:42.425 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:42 smithi003 bash[103255]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9 2024-03-21T08:44:42.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:42 smithi003 podman[103273]: 2024-03-21 08:44:42.431918837 +0000 UTC m=+0.645828104 container remove cc321b34bc33c281805a5b745ae8382e5f43e67b10a4e12745c10b2331fae40a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T08:44:43.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:42.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:43.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:42.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:43.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:42.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:43.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:42.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:43.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:42.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:43.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:42 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:42.688+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:43.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:43.077+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:43.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:43.077+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:43.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:43.077+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:43.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:43.077+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:43.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:43.077+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:43.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:43.077+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:43.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:43 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:43.077+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:43.652 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:43 smithi003 podman[103361]: 2024-03-21 08:44:43.248215881 +0000 UTC m=+0.034854535 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:44:43.652 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:43 smithi003 podman[103361]: 2024-03-21 08:44:43.357717185 +0000 UTC m=+0.144355865 container create 3ec0b6b028444437128b6fb80c0c4c159e9084d4b231169a2dbc75035b376e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:44:43.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:43 smithi003 podman[103361]: 2024-03-21 08:44:43.652907754 +0000 UTC m=+0.439546438 container init 3ec0b6b028444437128b6fb80c0c4c159e9084d4b231169a2dbc75035b376e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T08:44:43.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:43 smithi003 podman[103361]: 2024-03-21 08:44:43.664052703 +0000 UTC m=+0.450691347 container start 3ec0b6b028444437128b6fb80c0c4c159e9084d4b231169a2dbc75035b376e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T08:44:43.925 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:43 smithi003 podman[103361]: 2024-03-21 08:44:43.73474056 +0000 UTC m=+0.521379223 container attach 3ec0b6b028444437128b6fb80c0c4c159e9084d4b231169a2dbc75035b376e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T08:44:44.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:43.701+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:44.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:43.701+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:44.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:43.701+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:44.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:43.701+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:44.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:43.701+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:44.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:43.701+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:44.368 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:44 smithi003 podman[103361]: 2024-03-21 08:44:44.111551452 +0000 UTC m=+0.898190136 container died 3ec0b6b028444437128b6fb80c0c4c159e9084d4b231169a2dbc75035b376e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:44:44.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:44.106+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:44.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:44.106+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:44.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:44.106+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:44.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:44.106+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:44.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:44.106+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:44.376 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:44.106+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:44.376 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:44.106+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:44.622 INFO:journalctl@ceph.osd.9.smithi003.stdout:Mar 21 08:44:44 smithi003 podman[103387]: 2024-03-21 08:44:44.612850299 +0000 UTC m=+0.480645909 container remove 3ec0b6b028444437128b6fb80c0c4c159e9084d4b231169a2dbc75035b376e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T08:44:44.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.9.service' 2024-03-21T08:44:44.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:44:44.795 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-21T08:44:44.795 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-21T08:44:44.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.10 2024-03-21T08:44:45.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:44 smithi084 systemd[1]: Stopping Ceph osd.10 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:44:45.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.078+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:45.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.078+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:45.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.078+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:45.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.078+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:45.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.078+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:45.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.078+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:45.125 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.078+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:45.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:44.674+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:45.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:44.674+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:45.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:44.674+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:45.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:44.674+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:45.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:44.674+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:45.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:44.674+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:45.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:44 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:44.674+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:45.624 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.336+0000 7f8ff1e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:44:45.624 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.336+0000 7f8ff1e00700 -1 osd.10 107 *** Got signal Terminated *** 2024-03-21T08:44:45.624 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:45 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:45.336+0000 7f8ff1e00700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:44:46.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:45.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:46.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:45.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:46.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:45.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:46.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:45.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:46.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:45.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:46.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:45.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:46.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:45 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:45.678+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:46.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:46.104+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:46.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:46.104+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:46.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:46.104+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:46.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:46.104+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:46.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:46.104+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:46.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:46.104+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:46.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:46 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:46.104+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:47.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:46.680+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:47.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:46.680+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:47.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:46.680+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:47.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:46.680+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:47.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:46.680+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:47.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:46.680+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:47.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:46 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:46.680+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:47.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:47.113+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:47.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:47.113+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:47.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:47.113+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:47.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:47.113+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:47.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:47.113+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:47.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:47.113+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:47.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:47 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:47.113+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:48.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:47.695+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:48.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:47.695+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:48.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:47.695+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:48.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:47.695+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:48.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:47.695+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:48.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:47.695+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:48.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:47 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:47.695+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:48.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:48.074+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:48.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:48.074+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:48.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:48.074+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:48.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:48.074+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:48.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:48.074+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:48.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:48.074+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:48.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:48 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:48.074+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:49.067 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:48.740+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:49.068 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:48.740+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:49.068 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:48.740+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:49.068 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:48.740+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:49.068 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:48.740+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:49.068 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:48.740+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:49.068 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:48 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:48.740+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:49.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:49.062+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:49.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:49.062+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:49.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:49.062+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:49.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:49.062+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:49.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:49.062+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:49.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:49.062+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:49.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:49 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:49.062+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:50.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:49.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:50.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:49.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:50.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:49.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:50.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:49.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:50.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:49.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:50.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:49.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:50.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:49.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:50.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:50.031+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:14.573452+0000 front 2024-03-21T08:43:14.573246+0000 (oldest deadline 2024-03-21T08:43:39.273420+0000) 2024-03-21T08:44:50.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:50.031+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:24.476713+0000 front 2024-03-21T08:43:24.476471+0000 (oldest deadline 2024-03-21T08:43:49.776390+0000) 2024-03-21T08:44:50.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:50.031+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:36.080206+0000 front 2024-03-21T08:43:36.080696+0000 (oldest deadline 2024-03-21T08:43:58.379995+0000) 2024-03-21T08:44:50.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:50.031+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-03-21T08:43:42.482027+0000 front 2024-03-21T08:43:42.482130+0000 (oldest deadline 2024-03-21T08:44:06.581742+0000) 2024-03-21T08:44:50.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:50.031+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:53.584756+0000 front 2024-03-21T08:43:53.585003+0000 (oldest deadline 2024-03-21T08:44:17.084952+0000) 2024-03-21T08:44:50.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:50.031+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:02.887991+0000 front 2024-03-21T08:44:02.887606+0000 (oldest deadline 2024-03-21T08:44:27.587541+0000) 2024-03-21T08:44:50.375 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10[60232]: 2024-03-21T08:44:50.031+0000 7f8fed600700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:12.189857+0000 front 2024-03-21T08:44:12.190189+0000 (oldest deadline 2024-03-21T08:44:35.689747+0000) 2024-03-21T08:44:51.034 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:50 smithi084 podman[72107]: 2024-03-21 08:44:50.689293262 +0000 UTC m=+5.492003850 container died b7c2cfd6e7cc711b01389a1d494bbb70c955ea6c7ce0ae25000fcded0419771b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T08:44:51.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:50.737+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:51.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:50.737+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:51.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:50.737+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:51.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:50.737+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:51.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:50.737+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:51.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:50.737+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:51.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:50 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:50.737+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:51.373 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:51 smithi084 podman[72107]: 2024-03-21 08:44:51.138841483 +0000 UTC m=+5.941552095 container cleanup b7c2cfd6e7cc711b01389a1d494bbb70c955ea6c7ce0ae25000fcded0419771b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T08:44:51.374 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:51 smithi084 bash[72107]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10 2024-03-21T08:44:51.624 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:51 smithi084 podman[72122]: 2024-03-21 08:44:51.374569158 +0000 UTC m=+0.667485948 container remove b7c2cfd6e7cc711b01389a1d494bbb70c955ea6c7ce0ae25000fcded0419771b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T08:44:52.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:51.761+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:52.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:51.761+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:52.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:51.761+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:52.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:51.761+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:52.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:51.761+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:52.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:51.761+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:52.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:51 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:51.761+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:52.402 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:52 smithi084 podman[72206]: 2024-03-21 08:44:52.047685769 +0000 UTC m=+0.042992818 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:44:52.402 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:52 smithi084 podman[72206]: 2024-03-21 08:44:52.151155851 +0000 UTC m=+0.146462909 container create 064442c96a327085b263b0348451e0f9273c6a384fae8e1f53ab2b9d3dec2861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:44:52.402 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:52 smithi084 podman[72206]: 2024-03-21 08:44:52.401968716 +0000 UTC m=+0.397275792 container init 064442c96a327085b263b0348451e0f9273c6a384fae8e1f53ab2b9d3dec2861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-21T08:44:52.835 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:52 smithi084 podman[72206]: 2024-03-21 08:44:52.411695622 +0000 UTC m=+0.407002683 container start 064442c96a327085b263b0348451e0f9273c6a384fae8e1f53ab2b9d3dec2861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:44:52.836 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:52 smithi084 podman[72206]: 2024-03-21 08:44:52.495486617 +0000 UTC m=+0.490793690 container attach 064442c96a327085b263b0348451e0f9273c6a384fae8e1f53ab2b9d3dec2861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:44:53.124 INFO:journalctl@ceph.osd.10.smithi084.stdout:Mar 21 08:44:52 smithi084 podman[72206]: 2024-03-21 08:44:52.837456321 +0000 UTC m=+0.832763385 container died 064442c96a327085b263b0348451e0f9273c6a384fae8e1f53ab2b9d3dec2861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T08:44:53.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:52.747+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:53.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:52.747+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:53.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:52.747+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:53.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:52.747+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:53.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:52.747+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:53.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:52.747+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:53.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:52 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:52.747+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:53.601 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.10.service' 2024-03-21T08:44:53.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:44:53.707 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-21T08:44:53.707 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-21T08:44:53.707 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.11 2024-03-21T08:44:54.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:54.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:54.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:54.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:54.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:54.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:54.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:54.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:53.770+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-03-21T08:44:28.070310+0000 front 2024-03-21T08:44:28.070041+0000 (oldest deadline 2024-03-21T08:44:52.769864+0000) 2024-03-21T08:44:54.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:53 smithi171 systemd[1]: Stopping Ceph osd.11 for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:44:54.632 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.270+0000 7f4206800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:44:54.632 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.270+0000 7f4206800700 -1 osd.11 107 *** Got signal Terminated *** 2024-03-21T08:44:54.633 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.270+0000 7f4206800700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T08:44:55.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:55.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:55.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:55.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:55.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:55.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:55.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:55.134 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:54.774+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-03-21T08:44:28.070310+0000 front 2024-03-21T08:44:28.070041+0000 (oldest deadline 2024-03-21T08:44:52.769864+0000) 2024-03-21T08:44:56.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.815+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:56.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.815+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:56.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.815+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:56.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.815+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:56.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.815+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:56.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.815+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:56.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.816+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:56.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:55 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:55.816+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-03-21T08:44:28.070310+0000 front 2024-03-21T08:44:28.070041+0000 (oldest deadline 2024-03-21T08:44:52.769864+0000) 2024-03-21T08:44:57.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:57.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:57.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:57.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:57.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:57.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:57.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:57.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:56 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:56.798+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-03-21T08:44:28.070310+0000 front 2024-03-21T08:44:28.070041+0000 (oldest deadline 2024-03-21T08:44:52.769864+0000) 2024-03-21T08:44:58.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:58.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:58.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:58.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:58.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:58.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:58.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:58.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:57.811+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-03-21T08:44:28.070310+0000 front 2024-03-21T08:44:28.070041+0000 (oldest deadline 2024-03-21T08:44:52.769864+0000) 2024-03-21T08:44:59.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-03-21T08:43:15.749019+0000 front 2024-03-21T08:43:15.749422+0000 (oldest deadline 2024-03-21T08:43:40.449315+0000) 2024-03-21T08:44:59.132 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-03-21T08:43:23.252729+0000 front 2024-03-21T08:43:23.252610+0000 (oldest deadline 2024-03-21T08:43:47.352810+0000) 2024-03-21T08:44:59.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-21T08:43:35.556690+0000 front 2024-03-21T08:43:35.556044+0000 (oldest deadline 2024-03-21T08:44:00.856089+0000) 2024-03-21T08:44:59.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6812 osd.4 since back 2024-03-21T08:43:54.260565+0000 front 2024-03-21T08:43:54.260665+0000 (oldest deadline 2024-03-21T08:44:16.560697+0000) 2024-03-21T08:44:59.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-21T08:44:04.064996+0000 front 2024-03-21T08:44:04.065227+0000 (oldest deadline 2024-03-21T08:44:25.764975+0000) 2024-03-21T08:44:59.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-03-21T08:44:13.368063+0000 front 2024-03-21T08:44:13.368331+0000 (oldest deadline 2024-03-21T08:44:38.067664+0000) 2024-03-21T08:44:59.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.84:6820 osd.7 since back 2024-03-21T08:44:18.068490+0000 front 2024-03-21T08:44:18.068436+0000 (oldest deadline 2024-03-21T08:44:43.968385+0000) 2024-03-21T08:44:59.133 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:58 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11[61258]: 2024-03-21T08:44:58.822+0000 7f4202000700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-03-21T08:44:28.070310+0000 front 2024-03-21T08:44:28.070041+0000 (oldest deadline 2024-03-21T08:44:52.769864+0000) 2024-03-21T08:45:00.062 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:44:59 smithi171 podman[74382]: 2024-03-21 08:44:59.802571299 +0000 UTC m=+5.630420515 container died 0862fe05f1d826b1119514cf1b4cf62608b6562808f03e66f75a1ef7794a54c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T08:45:00.382 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:00 smithi171 podman[74382]: 2024-03-21 08:45:00.156970141 +0000 UTC m=+5.984819348 container cleanup 0862fe05f1d826b1119514cf1b4cf62608b6562808f03e66f75a1ef7794a54c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:45:00.383 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:00 smithi171 bash[74382]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11 2024-03-21T08:45:00.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:00 smithi171 podman[74401]: 2024-03-21 08:45:00.433644486 +0000 UTC m=+0.612808990 container remove 0862fe05f1d826b1119514cf1b4cf62608b6562808f03e66f75a1ef7794a54c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:45:01.562 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:01 smithi171 podman[74491]: 2024-03-21 08:45:01.162253054 +0000 UTC m=+0.038951927 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:45:01.562 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:01 smithi171 podman[74491]: 2024-03-21 08:45:01.292611911 +0000 UTC m=+0.169310823 container create 2c494dc686c11430c44496a72db0c80331c6b2df179a9455598ab9cf67a25b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T08:45:01.562 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:01 smithi171 podman[74491]: 2024-03-21 08:45:01.562697391 +0000 UTC m=+0.439396286 container init 2c494dc686c11430c44496a72db0c80331c6b2df179a9455598ab9cf67a25b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:45:01.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:01 smithi171 podman[74491]: 2024-03-21 08:45:01.572656486 +0000 UTC m=+0.449355347 container start 2c494dc686c11430c44496a72db0c80331c6b2df179a9455598ab9cf67a25b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:45:01.882 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:01 smithi171 podman[74491]: 2024-03-21 08:45:01.669956219 +0000 UTC m=+0.546655164 container attach 2c494dc686c11430c44496a72db0c80331c6b2df179a9455598ab9cf67a25b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:45:02.382 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 21 08:45:02 smithi171 podman[74491]: 2024-03-21 08:45:02.001679152 +0000 UTC m=+0.878378004 container died 2c494dc686c11430c44496a72db0c80331c6b2df179a9455598ab9cf67a25b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-21T08:45:02.757 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@osd.11.service' 2024-03-21T08:45:02.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:02.859 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-21T08:45:02.860 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-21T08:45:02.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.a 2024-03-21T08:45:03.425 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:45:02 smithi003 systemd[1]: Stopping Ceph mds.a for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:03.799 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:45:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a[73343]: 2024-03-21T08:45:03.461+0000 7f35c1800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:03.799 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:45:03 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a[73343]: 2024-03-21T08:45:03.461+0000 7f35c1800700 -1 mds.a *** got signal Terminated *** 2024-03-21T08:45:07.810 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:45:07 smithi003 podman[103465]: 2024-03-21 08:45:07.478195972 +0000 UTC m=+4.149840537 container died 04c79969f28d0a32a1b11d37f58c5737e38f28076689c2d28c61811b57498059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:45:08.175 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:45:07 smithi003 podman[103465]: 2024-03-21 08:45:07.92935468 +0000 UTC m=+4.600999246 container cleanup 04c79969f28d0a32a1b11d37f58c5737e38f28076689c2d28c61811b57498059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-21T08:45:08.175 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:45:07 smithi003 bash[103465]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a 2024-03-21T08:45:08.674 INFO:journalctl@ceph.mds.a.smithi003.stdout:Mar 21 08:45:08 smithi003 podman[103491]: 2024-03-21 08:45:08.222267217 +0000 UTC m=+0.720053564 container remove 04c79969f28d0a32a1b11d37f58c5737e38f28076689c2d28c61811b57498059 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T08:45:08.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.a.service' 2024-03-21T08:45:08.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:08.868 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-21T08:45:08.868 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-21T08:45:08.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.d 2024-03-21T08:45:09.423 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:45:08 smithi003 systemd[1]: Stopping Ceph mds.d for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:09.674 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:45:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d[72266]: 2024-03-21T08:45:09.423+0000 7eff3e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:09.675 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:45:09 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d[72266]: 2024-03-21T08:45:09.423+0000 7eff3e200700 -1 mds.d *** got signal Terminated *** 2024-03-21T08:45:12.366 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:45:12 smithi003 podman[103597]: 2024-03-21 08:45:12.049349748 +0000 UTC m=+2.746676655 container died 217b9b8b110b09647c7dbfd3a073c0b8a1a9687964ac030672b161d580bce282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T08:45:12.366 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:45:12 smithi003 podman[103597]: 2024-03-21 08:45:12.366685987 +0000 UTC m=+3.064012901 container cleanup 217b9b8b110b09647c7dbfd3a073c0b8a1a9687964ac030672b161d580bce282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-21T08:45:12.674 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:45:12 smithi003 bash[103597]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d 2024-03-21T08:45:12.675 INFO:journalctl@ceph.mds.d.smithi003.stdout:Mar 21 08:45:12 smithi003 podman[103611]: 2024-03-21 08:45:12.517630246 +0000 UTC m=+0.452545569 container remove 217b9b8b110b09647c7dbfd3a073c0b8a1a9687964ac030672b161d580bce282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-d, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-03-21T08:45:13.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.d.service' 2024-03-21T08:45:13.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:13.162 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-21T08:45:13.162 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-21T08:45:13.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.g 2024-03-21T08:45:13.674 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:45:13 smithi003 systemd[1]: Stopping Ceph mds.g for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:14.174 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:45:13 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g[72952]: 2024-03-21T08:45:13.754+0000 7fca3de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:14.175 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:45:13 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g[72952]: 2024-03-21T08:45:13.754+0000 7fca3de00700 -1 mds.g *** got signal Terminated *** 2024-03-21T08:45:16.340 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:45:16 smithi003 podman[103718]: 2024-03-21 08:45:16.064186494 +0000 UTC m=+2.458708465 container died 01c0afe2d76de61515fe8108b2c467ae55a4df48987f0a2d23ee356ae84d47fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T08:45:16.675 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:45:16 smithi003 podman[103718]: 2024-03-21 08:45:16.478068705 +0000 UTC m=+2.872590642 container cleanup 01c0afe2d76de61515fe8108b2c467ae55a4df48987f0a2d23ee356ae84d47fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T08:45:16.675 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:45:16 smithi003 bash[103718]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g 2024-03-21T08:45:17.174 INFO:journalctl@ceph.mds.g.smithi003.stdout:Mar 21 08:45:16 smithi003 podman[103732]: 2024-03-21 08:45:16.712408427 +0000 UTC m=+0.630393229 container remove 01c0afe2d76de61515fe8108b2c467ae55a4df48987f0a2d23ee356ae84d47fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T08:45:17.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.g.service' 2024-03-21T08:45:17.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:17.350 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-21T08:45:17.350 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-21T08:45:17.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.j 2024-03-21T08:45:17.924 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:45:17 smithi003 systemd[1]: Stopping Ceph mds.j for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:18.424 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:45:17 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j[71881]: 2024-03-21T08:45:17.938+0000 7f2d99e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:18.425 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:45:17 smithi003 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j[71881]: 2024-03-21T08:45:17.938+0000 7f2d99e00700 -1 mds.j *** got signal Terminated *** 2024-03-21T08:45:20.424 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:45:20 smithi003 podman[103837]: 2024-03-21 08:45:20.280909418 +0000 UTC m=+2.461669016 container died a0523c19653a5c796648ceed99ac5ddcadcd97da980e05da43021102e74e2ec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T08:45:20.924 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:45:20 smithi003 podman[103837]: 2024-03-21 08:45:20.564931154 +0000 UTC m=+2.745690768 container cleanup a0523c19653a5c796648ceed99ac5ddcadcd97da980e05da43021102e74e2ec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T08:45:20.925 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:45:20 smithi003 bash[103837]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j 2024-03-21T08:45:20.925 INFO:journalctl@ceph.mds.j.smithi003.stdout:Mar 21 08:45:20 smithi003 podman[103851]: 2024-03-21 08:45:20.69889969 +0000 UTC m=+0.400977956 container remove a0523c19653a5c796648ceed99ac5ddcadcd97da980e05da43021102e74e2ec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T08:45:21.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.j.service' 2024-03-21T08:45:21.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:21.305 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-21T08:45:21.305 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-21T08:45:21.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.b 2024-03-21T08:45:21.800 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:45:21 smithi084 systemd[1]: Stopping Ceph mds.b for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:22.123 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:45:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b[65441]: 2024-03-21T08:45:21.817+0000 7f70f1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:22.124 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:45:21 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b[65441]: 2024-03-21T08:45:21.818+0000 7f70f1000700 -1 mds.b *** got signal Terminated *** 2024-03-21T08:45:24.315 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:45:23 smithi084 podman[72307]: 2024-03-21 08:45:23.981013398 +0000 UTC m=+2.328426287 container died 8f5c231aee30c77f823c39a23e14268ff23b21e2bb8f8e0be85c2a751c4d2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-21T08:45:24.623 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:45:24 smithi084 podman[72307]: 2024-03-21 08:45:24.43841148 +0000 UTC m=+2.785824378 container cleanup 8f5c231aee30c77f823c39a23e14268ff23b21e2bb8f8e0be85c2a751c4d2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T08:45:24.623 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:45:24 smithi084 bash[72307]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b 2024-03-21T08:45:25.002 INFO:journalctl@ceph.mds.b.smithi084.stdout:Mar 21 08:45:24 smithi084 podman[72319]: 2024-03-21 08:45:24.673193142 +0000 UTC m=+0.682969288 container remove 8f5c231aee30c77f823c39a23e14268ff23b21e2bb8f8e0be85c2a751c4d2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:45:25.027 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.b.service' 2024-03-21T08:45:25.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:25.119 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-21T08:45:25.119 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-21T08:45:25.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.e 2024-03-21T08:45:25.373 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:45:25 smithi084 systemd[1]: Stopping Ceph mds.e for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:25.873 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:45:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e[64828]: 2024-03-21T08:45:25.618+0000 7f3817200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:25.873 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:45:25 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e[64828]: 2024-03-21T08:45:25.618+0000 7f3817200700 -1 mds.e *** got signal Terminated *** 2024-03-21T08:45:28.922 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:45:28 smithi084 podman[72424]: 2024-03-21 08:45:28.627489266 +0000 UTC m=+3.159649514 container died 88eb30fefa9eb0385edd9a1d1680bacf0330683af1f0e142c086c9839201769e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:45:29.318 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:45:29 smithi084 podman[72424]: 2024-03-21 08:45:29.026804966 +0000 UTC m=+3.558965255 container cleanup 88eb30fefa9eb0385edd9a1d1680bacf0330683af1f0e142c086c9839201769e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:45:29.318 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:45:29 smithi084 bash[72424]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e 2024-03-21T08:45:29.623 INFO:journalctl@ceph.mds.e.smithi084.stdout:Mar 21 08:45:29 smithi084 podman[72438]: 2024-03-21 08:45:29.319787165 +0000 UTC m=+0.678369790 container remove 88eb30fefa9eb0385edd9a1d1680bacf0330683af1f0e142c086c9839201769e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T08:45:29.703 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.e.service' 2024-03-21T08:45:29.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:29.794 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-21T08:45:29.794 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-21T08:45:29.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.h 2024-03-21T08:45:30.123 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:45:29 smithi084 systemd[1]: Stopping Ceph mds.h for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:30.623 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:45:30 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h[65129]: 2024-03-21T08:45:30.361+0000 7f8c83c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:30.623 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:45:30 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h[65129]: 2024-03-21T08:45:30.361+0000 7f8c83c00700 -1 mds.h *** got signal Terminated *** 2024-03-21T08:45:37.757 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:45:37 smithi084 podman[72541]: 2024-03-21 08:45:37.429309117 +0000 UTC m=+7.233189810 container died 83c96345e0c4e79986d1c530ed4cae21dcd07c1e3d06d1e5c9d657fcf39f780b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T08:45:38.123 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:45:37 smithi084 podman[72541]: 2024-03-21 08:45:37.871734219 +0000 UTC m=+7.675614913 container cleanup 83c96345e0c4e79986d1c530ed4cae21dcd07c1e3d06d1e5c9d657fcf39f780b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T08:45:38.124 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:45:37 smithi084 bash[72541]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h 2024-03-21T08:45:38.596 INFO:journalctl@ceph.mds.h.smithi084.stdout:Mar 21 08:45:38 smithi084 podman[72554]: 2024-03-21 08:45:38.151211709 +0000 UTC m=+0.701081757 container remove 83c96345e0c4e79986d1c530ed4cae21dcd07c1e3d06d1e5c9d657fcf39f780b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-h, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:45:38.631 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.h.service' 2024-03-21T08:45:38.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:38.731 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-21T08:45:38.732 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-21T08:45:38.732 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.k 2024-03-21T08:45:39.255 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:45:38 smithi084 systemd[1]: Stopping Ceph mds.k for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:39.623 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:45:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k[64508]: 2024-03-21T08:45:39.255+0000 7f8e74e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:39.624 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:45:39 smithi084 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k[64508]: 2024-03-21T08:45:39.255+0000 7f8e74e00700 -1 mds.k *** got signal Terminated *** 2024-03-21T08:45:41.770 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:45:41 smithi084 podman[72659]: 2024-03-21 08:45:41.480100007 +0000 UTC m=+2.334589459 container died 0498498dd34531742d8d31142a39f1c1a9784edcd274f6d54b92f0e61f3b8ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-21T08:45:42.123 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:45:41 smithi084 podman[72659]: 2024-03-21 08:45:41.917476967 +0000 UTC m=+2.771966390 container cleanup 0498498dd34531742d8d31142a39f1c1a9784edcd274f6d54b92f0e61f3b8ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T08:45:42.124 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:45:41 smithi084 bash[72659]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k 2024-03-21T08:45:42.623 INFO:journalctl@ceph.mds.k.smithi084.stdout:Mar 21 08:45:42 smithi084 podman[72673]: 2024-03-21 08:45:42.160920484 +0000 UTC m=+0.663213318 container remove 0498498dd34531742d8d31142a39f1c1a9784edcd274f6d54b92f0e61f3b8ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T08:45:42.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.k.service' 2024-03-21T08:45:42.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:42.783 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-21T08:45:42.784 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-21T08:45:42.784 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.c 2024-03-21T08:45:43.131 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:45:42 smithi171 systemd[1]: Stopping Ceph mds.c for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:43.631 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:45:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c[66591]: 2024-03-21T08:45:43.361+0000 7f3f4ac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:43.632 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:45:43 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c[66591]: 2024-03-21T08:45:43.361+0000 7f3f4ac00700 -1 mds.c *** got signal Terminated *** 2024-03-21T08:45:47.842 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:45:47 smithi171 podman[74615]: 2024-03-21 08:45:47.536672462 +0000 UTC m=+4.313812610 container died a817d127d543d4a3b559fa60c449930c42dc043b451f5093389b7efa78058813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-21T08:45:48.133 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:45:47 smithi171 podman[74615]: 2024-03-21 08:45:47.946934214 +0000 UTC m=+4.724074379 container cleanup a817d127d543d4a3b559fa60c449930c42dc043b451f5093389b7efa78058813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:45:48.133 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:45:47 smithi171 bash[74615]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c 2024-03-21T08:45:48.631 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 21 08:45:48 smithi171 podman[74627]: 2024-03-21 08:45:48.239895375 +0000 UTC m=+0.692867996 container remove a817d127d543d4a3b559fa60c449930c42dc043b451f5093389b7efa78058813 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:45:48.710 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.c.service' 2024-03-21T08:45:48.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:48.825 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-21T08:45:48.825 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-21T08:45:48.825 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.f 2024-03-21T08:45:49.381 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:48 smithi171 systemd[1]: Stopping Ceph mds.f for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:49.881 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f[66279]: 2024-03-21T08:45:49.414+0000 7f816bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:49.881 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:49 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f[66279]: 2024-03-21T08:45:49.414+0000 7f816bc00700 -1 mds.f *** got signal Terminated *** 2024-03-21T08:45:52.513 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:52 smithi171 podman[74738]: 2024-03-21 08:45:52.145808359 +0000 UTC m=+2.847394690 container stop c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T08:45:52.514 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:52 smithi171 podman[74738]: 2024-03-21 08:45:52.172832008 +0000 UTC m=+2.874418332 container died c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:45:52.851 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:52 smithi171 podman[74738]: 2024-03-21 08:45:52.626073012 +0000 UTC m=+3.327659358 container cleanup c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:45:52.851 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:52 smithi171 bash[74738]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f 2024-03-21T08:45:53.131 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 21 08:45:52 smithi171 podman[74753]: 2024-03-21 08:45:52.85180885 +0000 UTC m=+0.681256320 container remove c2edf7ea5ad6b44346adac186bcd3e1b99ddd850419075381c24ce179f8355db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T08:45:53.354 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.f.service' 2024-03-21T08:45:53.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:53.469 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-21T08:45:53.470 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-21T08:45:53.470 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.i 2024-03-21T08:45:53.631 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:45:53 smithi171 systemd[1]: Stopping Ceph mds.i for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:54.381 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:45:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i[66897]: 2024-03-21T08:45:54.010+0000 7f2c4be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:54.381 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:45:54 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i[66897]: 2024-03-21T08:45:54.010+0000 7f2c4be00700 -1 mds.i *** got signal Terminated *** 2024-03-21T08:45:56.282 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:45:55 smithi171 podman[74860]: 2024-03-21 08:45:55.999335428 +0000 UTC m=+2.124007231 container died 5339140889dd01af6b1d23b3d98f4dbb238ec37480412eb587ba364b69b61029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:45:56.631 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:45:56 smithi171 podman[74860]: 2024-03-21 08:45:56.395869625 +0000 UTC m=+2.520541424 container cleanup 5339140889dd01af6b1d23b3d98f4dbb238ec37480412eb587ba364b69b61029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-03-21T08:45:56.631 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:45:56 smithi171 bash[74860]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i 2024-03-21T08:45:56.973 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 21 08:45:56 smithi171 podman[74874]: 2024-03-21 08:45:56.72195939 +0000 UTC m=+0.704032451 container remove 5339140889dd01af6b1d23b3d98f4dbb238ec37480412eb587ba364b69b61029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T08:45:56.993 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.i.service' 2024-03-21T08:45:57.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:45:57.101 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-21T08:45:57.102 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-21T08:45:57.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.l 2024-03-21T08:45:57.634 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:45:57 smithi171 systemd[1]: Stopping Ceph mds.l for 544fc91e-e75b-11ee-95cd-87774f69a715... 2024-03-21T08:45:58.131 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:45:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l[65962]: 2024-03-21T08:45:57.664+0000 7f0503600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:45:58.132 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:45:57 smithi171 ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l[65962]: 2024-03-21T08:45:57.664+0000 7f0503600700 -1 mds.l *** got signal Terminated *** 2024-03-21T08:46:00.254 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:45:59 smithi171 podman[74980]: 2024-03-21 08:45:59.942658991 +0000 UTC m=+2.399938552 container died 378e40e0af04376a4f0040091aefdb339ee0ef08acf0ed5b65a0ab6c6b882f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-21T08:46:00.626 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:46:00 smithi171 podman[74980]: 2024-03-21 08:46:00.368001607 +0000 UTC m=+2.825281189 container cleanup 378e40e0af04376a4f0040091aefdb339ee0ef08acf0ed5b65a0ab6c6b882f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:46:00.626 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:46:00 smithi171 bash[74980]: ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l 2024-03-21T08:46:00.881 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 21 08:46:00 smithi171 podman[74994]: 2024-03-21 08:46:00.627403597 +0000 UTC m=+0.671043186 container remove 378e40e0af04376a4f0040091aefdb339ee0ef08acf0ed5b65a0ab6c6b882f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-544fc91e-e75b-11ee-95cd-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:46:01.171 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-544fc91e-e75b-11ee-95cd-87774f69a715@mds.l.service' 2024-03-21T08:46:01.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T08:46:01.284 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-21T08:46:01.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T08:46:01.627 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:46:06.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T08:46:06.896 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:46:11.538 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T08:46:11.863 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:46:16.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:46:16.965 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:46:17.067 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:46:17.162 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T08:46:17.165 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi003/crash 2024-03-21T08:46:17.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash -- . 2024-03-21T08:46:17.287 INFO:teuthology.orchestra.run.smithi003.stderr:tar: /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T08:46:17.288 INFO:teuthology.orchestra.run.smithi003.stderr:tar: Error is not recoverable: exiting now 2024-03-21T08:46:17.292 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi084/crash 2024-03-21T08:46:17.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash -- . 2024-03-21T08:46:17.406 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T08:46:17.406 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now 2024-03-21T08:46:17.411 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi171/crash 2024-03-21T08:46:17.412 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash -- . 2024-03-21T08:46:17.511 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T08:46:17.511 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-03-21T08:46:17.515 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T08:46:17.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T08:46:17.634 INFO:teuthology.orchestra.run.smithi003.stdout:2024-03-21T08:20:24.490360+0000 mon.a (mon.0) 182 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T08:46:17.635 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T08:46:17.635 INFO:tasks.cephadm:Compressing logs... 2024-03-21T08:46:17.635 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T08:46:17.640 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T08:46:17.647 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T08:46:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T08:46:17.710 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T08:46:17.710 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-21T08:46:17.710 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-21T08:46:17.710 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-21T08:46:17.711 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T08:46:17.713 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-21T08:46:17.714 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-21T08:46:17.716 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T08:46:17.719 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.j.log: /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.3.log: 90.1% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T08:46:17.719 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-21T08:46:17.721 INFO:teuthology.orchestra.run.smithi003.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T08:46:17.721 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T08:46:17.723 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-21T08:46:17.724 INFO:teuthology.orchestra.run.smithi003.stderr: 94.4% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-21T08:46:17.724 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T08:46:17.725 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log 2024-03-21T08:46:17.727 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T08:46:17.727 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T08:46:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-21T08:46:17.730 INFO:teuthology.orchestra.run.smithi084.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T08:46:17.731 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T08:46:17.732 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-21T08:46:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log: 87.2% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T08:46:17.733 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T08:46:17.734 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.10.log: 92.0%gzip -- replaced with /var/log/ceph/cephadm.log.gz -5 2024-03-21T08:46:17.734 INFO:teuthology.orchestra.run.smithi084.stderr: --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-21T08:46:17.734 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T08:46:17.735 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T08:46:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-21T08:46:17.737 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T08:46:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-21T08:46:17.738 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-21T08:46:17.739 INFO:teuthology.orchestra.run.smithi084.stderr: 90.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T08:46:17.739 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T08:46:17.739 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-21T08:46:17.739 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T08:46:17.742 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-21T08:46:17.742 INFO:teuthology.orchestra.run.smithi171.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T08:46:17.742 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-21T08:46:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-21T08:46:17.743 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-21T08:46:17.743 INFO:teuthology.orchestra.run.smithi084.stderr: 94.6% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-21T08:46:17.743 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-21T08:46:17.743 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-21T08:46:17.745 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-21T08:46:17.745 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-21T08:46:17.745 INFO:teuthology.orchestra.run.smithi171.stderr: 90.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T08:46:17.745 INFO:teuthology.orchestra.run.smithi171.stderr: 94.5% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-21T08:46:17.746 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T08:46:17.746 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-21T08:46:17.747 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-21T08:46:17.748 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.e.log: /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T08:46:17.748 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T08:46:17.749 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log 2024-03-21T08:46:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.log: 92.1% 90.8% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T08:46:17.752 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log 2024-03-21T08:46:17.753 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T08:46:17.753 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T08:46:17.754 INFO:teuthology.orchestra.run.smithi003.stderr: 94.4% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-21T08:46:17.754 INFO:teuthology.orchestra.run.smithi003.stderr: 94.5% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-21T08:46:17.754 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-21T08:46:17.756 INFO:teuthology.orchestra.run.smithi084.stderr: 83.6%gzip -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T08:46:17.756 INFO:teuthology.orchestra.run.smithi084.stderr: -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T08:46:17.757 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-21T08:46:17.757 INFO:teuthology.orchestra.run.smithi171.stderr: 87.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T08:46:17.757 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T08:46:17.758 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.c.log: 92.1% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-21T08:46:17.759 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-21T08:46:17.760 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-21T08:46:17.762 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log: 87.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T08:46:17.765 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-21T08:46:17.775 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T08:46:17.791 INFO:teuthology.orchestra.run.smithi084.stderr: 90.8% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T08:46:18.113 INFO:teuthology.orchestra.run.smithi003.stderr: 89.7% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-21T08:46:18.296 INFO:teuthology.orchestra.run.smithi171.stderr: 91.8% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-21T08:46:18.377 INFO:teuthology.orchestra.run.smithi084.stderr: 91.9% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-21T08:46:19.177 INFO:teuthology.orchestra.run.smithi003.stderr: 90.5% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-21T08:46:19.801 INFO:teuthology.orchestra.run.smithi171.stderr: 95.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-21T08:46:20.706 INFO:teuthology.orchestra.run.smithi171.stderr: 95.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-21T08:46:20.946 INFO:teuthology.orchestra.run.smithi171.stderr: 95.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-21T08:46:22.685 INFO:teuthology.orchestra.run.smithi171.stderr: 91.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-21T08:46:22.888 INFO:teuthology.orchestra.run.smithi003.stderr: 91.9% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T08:46:24.150 INFO:teuthology.orchestra.run.smithi084.stderr: 91.5% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-21T08:46:24.189 INFO:teuthology.orchestra.run.smithi171.stderr: 91.6% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-21T08:46:24.538 INFO:teuthology.orchestra.run.smithi171.stderr: 91.6% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T08:46:24.666 INFO:teuthology.orchestra.run.smithi003.stderr: 91.7% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T08:46:24.704 INFO:teuthology.orchestra.run.smithi084.stderr: 91.6% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T08:46:25.574 INFO:teuthology.orchestra.run.smithi171.stderr: 91.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T08:46:25.577 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-03-21T08:46:25.577 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m7.879s 2024-03-21T08:46:25.578 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m32.338s 2024-03-21T08:46:25.578 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m2.492s 2024-03-21T08:46:25.679 INFO:teuthology.orchestra.run.smithi003.stderr: 91.6% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-21T08:46:25.918 INFO:teuthology.orchestra.run.smithi084.stderr: 91.7% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-21T08:46:25.991 INFO:teuthology.orchestra.run.smithi003.stderr: 91.6% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-21T08:46:26.052 INFO:teuthology.orchestra.run.smithi084.stderr: 91.5% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T08:46:27.027 INFO:teuthology.orchestra.run.smithi003.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-21T08:46:27.034 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-03-21T08:46:27.034 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m9.360s 2024-03-21T08:46:27.034 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m36.702s 2024-03-21T08:46:27.034 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m2.701s 2024-03-21T08:46:33.887 INFO:teuthology.orchestra.run.smithi084.stderr: 93.3% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-21T08:46:34.055 INFO:teuthology.orchestra.run.smithi084.stderr: 93.4% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-21T08:46:43.587 INFO:teuthology.orchestra.run.smithi084.stderr: 92.7% -- replaced with /var/log/ceph/544fc91e-e75b-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-21T08:46:43.590 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-03-21T08:46:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m25.899s 2024-03-21T08:46:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:user 1m22.702s 2024-03-21T08:46:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m6.113s 2024-03-21T08:46:43.592 INFO:tasks.cephadm:Archiving logs... 2024-03-21T08:46:43.593 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi003/log 2024-03-21T08:46:43.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T08:46:48.923 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi084/log 2024-03-21T08:46:48.924 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T08:47:00.177 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi171/log 2024-03-21T08:47:00.178 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T08:47:04.764 INFO:tasks.cephadm:Removing cluster... 2024-03-21T08:47:04.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 --force 2024-03-21T08:47:05.111 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:47:05.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 --force 2024-03-21T08:47:06.193 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:47:07.049 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 544fc91e-e75b-11ee-95cd-87774f69a715 --force 2024-03-21T08:47:07.370 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 544fc91e-e75b-11ee-95cd-87774f69a715 2024-03-21T08:47:08.091 INFO:tasks.cephadm:Removing cephadm ... 2024-03-21T08:47:08.092 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T08:47:08.153 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T08:47:08.209 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T08:47:08.267 INFO:tasks.cephadm:Teardown complete 2024-03-21T08:47:08.267 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T08:47:08.279 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-21T08:47:08.281 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T08:47:08.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T08:47:08.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T08:47:08.296 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T08:47:08.478 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T08:47:08.489 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T08:47:08.490 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:47:08.498 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:47:08.505 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:47:08.546 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:47:08.552 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-03-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 66 -752us[ -715us] +/- 106ms 2024-03-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -73us[ -37us] +/- 53ms 2024-03-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 135 +741us[ +725us] +/- 94ms 2024-03-21T08:47:08.556 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:47:08.561 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:47:08.562 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:47:08.562 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-03-21T08:47:08.562 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 -795us[ -785us] +/- 107ms 2024-03-21T08:47:08.562 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 -68us[ -58us] +/- 53ms 2024-03-21T08:47:08.562 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +642us[ +652us] +/- 65ms 2024-03-21T08:47:08.562 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:47:08.571 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:47:08.571 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-03-21T08:47:08.571 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 -797us[ -802us] +/- 107ms 2024-03-21T08:47:08.572 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -60us[ -65us] +/- 53ms 2024-03-21T08:47:08.572 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +632us[ +627us] +/- 65ms 2024-03-21T08:47:08.572 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:47:08.573 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T08:47:08.585 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T08:47:08.586 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T08:47:08.597 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T08:47:08.853 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T08:47:09.073 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T08:47:09.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:47:09.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:47:09.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:47:09.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:47:09.512 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:47:09.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:47:09.615 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T08:47:09.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T08:47:09.644 INFO:teuthology.task.internal:Duration was 2388.918914 seconds 2024-03-21T08:47:09.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T08:47:09.660 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T08:47:09.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T08:47:09.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T08:47:09.678 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T08:47:09.828 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:47:09.836 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:47:09.848 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:47:10.253 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T08:47:10.253 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-03-21T08:47:10.254 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T08:47:10.320 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-03-21T08:47:10.320 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T08:47:10.385 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-03-21T08:47:10.385 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T08:47:10.465 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T08:47:10.466 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T08:47:10.474 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T08:47:10.478 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T08:47:10.574 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T08:47:10.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T08:47:10.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T08:47:10.584 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T08:47:11.094 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T08:47:11.106 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T08:47:11.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T08:47:11.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T08:47:11.305 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T08:47:11.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T08:47:11.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T08:47:11.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T08:47:11.435 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T08:47:11.536 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-03-21T08:47:11.539 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-03-21T08:47:11.543 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-03-21T08:47:11.628 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T08:47:11.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:47:11.686 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T08:47:11.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:47:11.741 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T08:47:11.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:47:11.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T08:47:11.808 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T08:47:11.809 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi003 2024-03-21T08:47:11.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T08:47:11.933 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi084 2024-03-21T08:47:11.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T08:47:12.046 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117/remote/smithi171 2024-03-21T08:47:12.046 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T08:47:12.157 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T08:47:12.157 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T08:47:12.166 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T08:47:12.173 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T08:47:12.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T08:47:12.244 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T08:47:12.244 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T08:47:12.258 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T08:47:12.258 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T08:47:12.266 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T08:47:12.275 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T08:47:12.318 INFO:teuthology.orchestra.run.smithi003.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 08:47 /home/ubuntu/cephtest 2024-03-21T08:47:12.322 INFO:teuthology.orchestra.run.smithi084.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 08:47 /home/ubuntu/cephtest 2024-03-21T08:47:12.332 INFO:teuthology.orchestra.run.smithi171.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 08:47 /home/ubuntu/cephtest 2024-03-21T08:47:12.336 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T08:47:12.348 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T08:47:12.361 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-21T08:47:12.361 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T08:47:12.390 DEBUG:teuthology.nuke:shortname: smithi003 2024-03-21T08:47:12.390 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T08:47:12.416 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 07:56:43.417094', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJDj96CaTm2/xZqJDkdoZNnUKpmGwfZz2kpHXR/Y21IrtkN6OGzeD4VzXbZ63tXBerS9FgBv3ayeab/WJ+/FpNE='} 2024-03-21T08:47:12.421 DEBUG:teuthology.nuke:shortname: smithi084 2024-03-21T08:47:12.421 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T08:47:12.448 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 07:56:43.418897', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPP0yP7mX4zsSOqBOxp68QobEtnme1Xc9wLUGtwZ2Yl7Za9u+FacaCqBcA5eMm9++w8axt+NJ46jUJYIxo9RShA='} 2024-03-21T08:47:12.449 DEBUG:teuthology.nuke:shortname: smithi171 2024-03-21T08:47:12.449 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T08:47:12.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 07:56:43.420329', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBXpGC40u8YdbIVz5SEG8QaUJr7u3b/aEXSvg9CYeSrYV+uWi7WnDuwILoBGgeERzSUEVTeQnVM7tqs7eJMJotE='} 2024-03-21T08:47:12.484 INFO:teuthology.orchestra.console.smithi003:Power off 2024-03-21T08:47:12.484 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T08:47:12.510 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2024-03-21T08:47:12.511 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:12.527 INFO:teuthology.orchestra.console.smithi084:Power off 2024-03-21T08:47:12.527 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T08:47:12.544 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2024-03-21T08:47:12.552 DEBUG:teuthology.orchestra.console.smithi084:power off output: Chassis Power Control: Down/Off 2024-03-21T08:47:12.552 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:12.571 INFO:teuthology.orchestra.console.smithi171:Power off 2024-03-21T08:47:12.571 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T08:47:12.582 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-03-21T08:47:12.596 DEBUG:teuthology.orchestra.console.smithi171:power off output: Chassis Power Control: Down/Off 2024-03-21T08:47:12.596 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:12.625 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-03-21T08:47:16.545 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:16.583 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:16.626 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:16.657 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2024-03-21T08:47:16.701 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-03-21T08:47:16.738 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-03-21T08:47:20.662 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:20.702 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:20.739 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:20.774 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2024-03-21T08:47:20.815 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-03-21T08:47:20.850 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is off 2024-03-21T08:47:20.917 INFO:teuthology.orchestra.console.smithi084:Power off completed 2024-03-21T08:47:20.951 INFO:teuthology.orchestra.console.smithi171:Power off completed 2024-03-21T08:47:24.778 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:47:24.890 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2024-03-21T08:47:24.991 INFO:teuthology.orchestra.console.smithi003:Power off completed 2024-03-21T08:47:25.099 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} duration: 2388.918914079666 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi003 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default mon.a-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 mon.b-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 mon.c-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0877337574d84672afd3a07a001c742d status: fail success: false 2024-03-21T08:47:25.099 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:47:25.203 INFO:teuthology.run:FAIL